Subject: [flow_cache_new_hashrnd] WARNING: at kernel/smp.c:328 smp_call_function_single()

From: Fengguang Wu <[email-masked]>
Date: Mon, 11 Feb 2013 16:56:00 +0800

Greetings,

I got the below oops and the first bad commit is

commit b29f39c7c3e75a741a7da88244ec707f293ec04c
Author: Chuansheng Liu <[email-masked]>
Date:   Wed Feb 6 23:18:21 2013 +0800

    smp: Give WARN()ing if in_interrupt() when calling smp_call_function_many()/single()
    
    Currently the functions smp_call_function_many()/single() will
    give a WARN()ing only in the case of irqs_disabled(), but that
    check is not enough to guarantee execution of the SMP
    cross-calls.
    
    In many other cases such as softirq handling/interrupt handling,
    the two APIs still can not be called, just as the
    smp_call_function_many() comments say:
    
      * You must not call this function with disabled interrupts or from a
      * hardware interrupt handler or from a bottom half handler. Preemption
      * must be disabled when calling this function.
    
    There is a real case for softirq DEADLOCK case:
    
    CPUA                            CPUB
                                    spin_lock(&spinlock)
                                    Any irq coming, call the irq handler
                                    irq_exit()
    spin_lock_irq(&spinlock)
    <== Blocking here due to
    CPUB hold it
                                      __do_softirq()
                                        run_timer_softirq()
                                          timer_cb()
                                            call smp_call_function_many()
                                              send IPI interrupt to CPUA
                                                wait_csd()
    
    Then both CPUA and CPUB will be deadlocked here.
    
    So we should give a warning in the in_interrupt() case as well.
    
    Signed-off-by: liu chuansheng <[email-masked]>
    Cc: [email-masked]
    Cc: [email-masked]
    Cc: [email-masked]
    Cc: [email-masked]
    Cc: [email-masked]
    Cc: [email-masked]
    Cc: Linus Torvalds <[email-masked]>
    Cc: Andrew Morton <[email-masked]>
    Cc: Peter Zijlstra <[email-masked]>
    Link: http://lkml.kernel.org/r/1360163901.24670.13.camel@cliu38-desktop-build
    Signed-off-by: Ingo Molnar <[email-masked]>

[  222.203606] nfsd: last server has exited, flushing export cache
[  229.330068] ------------[ cut here ]------------
[  229.331757] WARNING: at /c/kernel-tests/src/stable/kernel/smp.c:328 smp_call_function_single+0xbd/0x1c7()
[  229.335003] Hardware name: Bochs
[  229.336378] Modules linked in:
[  229.337825] Pid: 6740, comm: reboot Not tainted 3.8.0-rc6-00548-gc479495 #40
[  229.339945] Call Trace:
[  229.341176]  [<ffffffff810353b4>] warn_slowpath_common+0x83/0x9c
[  229.343091]  [<ffffffff817089d6>] ? flow_cache_new_hashrnd+0x98/0x98
[  229.345105]  [<ffffffff810353e7>] warn_slowpath_null+0x1a/0x1c
[  229.346978]  [<ffffffff81090793>] smp_call_function_single+0xbd/0x1c7
[  229.349017]  [<ffffffff81090afe>] smp_call_function_many+0x121/0x23e
[  229.350996]  [<ffffffff817089d6>] ? flow_cache_new_hashrnd+0x98/0x98
[  229.353005]  [<ffffffff81090e09>] smp_call_function+0x37/0x40
[  229.354860]  [<ffffffff8170907e>] flow_cache_flush+0x72/0xa0
[  229.356735]  [<ffffffff81759e33>] xfrm_dev_event+0x14/0x20
[  229.358545]  [<ffffffff817ff8b0>] notifier_call_chain+0x65/0x95
[  229.360469]  [<ffffffff8105ce16>] __raw_notifier_call_chain+0xe/0x10
[  229.362453]  [<ffffffff8105ce2c>] raw_notifier_call_chain+0x14/0x16
[  229.364453]  [<ffffffff816f63d6>] call_netdevice_notifiers+0x4a/0x4f
[  229.366434]  [<ffffffff816fa2ab>] __dev_notify_flags+0x37/0x5b
[  229.368342]  [<ffffffff816fa318>] dev_change_flags+0x49/0x54
[  229.370184]  [<ffffffff8174574a>] devinet_ioctl+0x24f/0x542
[  229.372036]  [<ffffffff81746975>] inet_ioctl+0x97/0xb1
[  229.373774]  [<ffffffff816e5042>] sock_do_ioctl.constprop.42+0x18/0x37
[  229.375791]  [<ffffffff816e5480>] sock_ioctl+0x1fd/0x20a
[  229.377648]  [<ffffffff810bb9cd>] ? trace_buffer_lock_reserve+0x41/0x56
[  229.379701]  [<ffffffff8112b7c6>] vfs_ioctl+0x26/0x39
[  229.381459]  [<ffffffff8112c0d2>] do_vfs_ioctl+0x41b/0x45e
[  229.383269]  [<ffffffff8100bc3a>] ? ftrace_raw_event_sys_enter+0x10b/0x11a
[  229.385404]  [<ffffffff817fc118>] ? retint_swapgs+0x13/0x1b
[  229.387227]  [<ffffffff8112c15a>] sys_ioctl+0x45/0x73
[  229.388975]  [<ffffffff818034be>] tracesys+0xd0/0xd5
[  229.390665] ---[ end trace c47a5dd12e99cb04 ]---

git bisect start c4794952da9f0395101682925a4117d0a6a01ae5 1589a3e7777631ff56dd58cd7dcdf275185e62b5 --
git bisect good 89bb67ff935d461544fed87174bb13dcc4bac673  #    11  2013-02-08 17:23:50  perf tools: Fix perf_evsel::exclude_GH handling
git bisect good cacafec82e2dbbfbeb5c86c1f31ff17d0983b11b  #    10  2013-02-08 21:24:30  arch/mips/jazz: remove depends on CONFIG_EXPERIMENTAL
git bisect good 42f0e2b23ea9c97e276b32e7a7849bed37e5c47f  #    18  2013-02-09 01:25:42  final removal of CONFIG_EXPERIMENTAL
git bisect  bad 9197b357ed90fb43488a67b30a6d7cf7edce34c9  #     0  2013-02-09 02:46:24  Merge remote-tracking branch 'kees/devtmpfs-safe' into devel-bee-x86_64-2013-02-07-05-59
git bisect  bad a31151b33053ee9c2f91d53f2f8b8b1f36c00115  #     0  2013-02-09 02:49:49  Merge branch 'core/urgent' into core/locking
git bisect good 0ebd64ee0f225028c5a97581cdfeeb15931ec35c  #    16  2013-02-09 05:33:05  liblockdep: Add public headers for pthread_mutex_t implementation
git bisect good 3279bccc7ff663ef1ccd8b9e8d4ac197c0e041e7  #    13  2013-02-09 09:33:45  liblockdep: Add a MAINTAINERS entry
git bisect  bad b29f39c7c3e75a741a7da88244ec707f293ec04c  #     0  2013-02-09 10:55:01  smp: Give WARN()ing if in_interrupt() when calling smp_call_function_many()/single()
git bisect good b2e7c77a379071318b2a9c9e500e3b8b47084391  #    11  2013-02-09 13:36:55  perf: Integrate liblockdep support into perf
git bisect good b2e7c77a379071318b2a9c9e500e3b8b47084391  #    34  2013-02-09 17:38:01  perf: Integrate liblockdep support into perf
git bisect good 1587f71ebbf5aedf754062baa11fcc9e9b49ecf0  #    33  2013-02-10 00:19:24  Add linux-next specific files for 20130125

Thanks,
Fengguang