Subject: Re: [PATCH net-next v2] rocker: move netevent neigh update to processes context

From: Toshiaki Makita <[email-masked]>
Date: Fri, 05 Jun 2015 00:34:50 +0900

On 15/06/04 (木) 18:07, Simon Horman wrote:
> Hi Dave,
>
> On Thu, Jun 04, 2015 at 01:34:09AM -0700, David Miller wrote:
>> From: Simon Horman <[email-masked]>
>> Date: Thu, 4 Jun 2015 17:20:48 +0900
>>
>>> What I was seeing is as follows:
>>>
>>> 1. rocker_port_ipv4_nh() is called via switchdev_port_obj_add()
>>>     with trans = SWITCHDEV_TRANS_PREPARE
>>>
>>> 2. rocker_port_ipv4_neigh() is called by rocker_neigh_update()
>>>     with trans = SWITCHDEV_TRANS_NONE.
>>>
>>>     The call chain goes up to arp_process() via neigh_update().
>>>
>>> 3. rocker_port_ipv4_nh() is called via switchdev_port_obj_add()
>>>     with trans = SWITCHDEV_TRANS_COMMIT
>>>
>>> #1 and #2 are guarded by rtl across those calls but
>>> #2 is not guarded by rtnl.
>>>
>>> Inside both rocker_port_ipv4_nh() and rocker_port_ipv4_neigh()
>>> neigh_tbl_lock lock is taken but it is not held across the
>>> two calls to rocker_port_ipv4_nh within a single prepare->commit transaction.
>>>
>>> I can double check that the above still occurs, but I'm not aware of any
>>> recent changes that would cause it not to occur any more.
>>
>> Ok, thanks for explaining.
>>
>> I still don't want this to be moved asynchronosly from the ARP neigh
>> update event code path.
>>
>> And therefore I'd like folks to look into another mechanism to fix
>> this.
>>
>> If that means the prepare/commit engine is given a spinlock by the
>> driver to be held across the two calls, so be it.
>
> I'm not opposed to such a scheme. But I'd like to note that it seems to me
> that for it to resolve the problem above the lock would be need be held
> both for "none" and "prepare->commit" transactions.  I think the former may
> be a little tricky as it may occur in IRQ context but perhaps I am missing
> something obvious.

I'm thinking IRQ context does not match the prepare-commit model, and 
Scott's fix is needed. There are more critical problems Scott's patch fixes.
(I shortly explained it before, although it is not clearly stated in the 
commitlog. http://marc.info/?l=linux-netdev&m=143219842420093&w=2)

1. Operations from IRQ context could change the state of rocker, like 
hash tables. This could cause inconsistent states between prepare-commit 
(for example, prepare phase cannot find an entry but commit phase can 
find it), and leads to memory corruption (unreserved memory could be 
used or reserved memory could not be used in commit phase).

2. rocker_wait_event_timeout() can sleep, but it can be called from IRQ 
context.

   rocker_event_neigh_update()
    rocker_port_ipv4_neigh()
     rocker_group_l3_unicast()
      rocker_group_tbl_do()
       rocker_group_tbl_add()
        rocker_cmd_exec()
         rocker_wait_event_timeout()
          wait_event_timeout()
           might_sleep()

3. Currently memory allocation is always done with GFP_KERNEL in 
__rocker_port_mem_alloc(), but it can be called from IRQ context.

   rocker_event_neigh_update()
    rocker_port_ipv4_neigh()
     rocker_port_kzalloc()
      __rocker_port_mem_alloc()

Toshiaki Makita