* Kernel 2.6.38.3 panic and hangs on network traffic
@ 2011-07-21 15:20 Rustam Afanasyev
0 siblings, 0 replies; only message in thread
From: Rustam Afanasyev @ 2011-07-21 15:20 UTC (permalink / raw)
To: netdev
[-- Attachment #1: Type: text/plain, Size: 617 bytes --]
Hi!
I starting up new server for the test. It's task is terminate L2TP&PPTP
users. And when i have opened server (NAS) for users - its crush quickly :(
What it's could be?
Hardware is Intel Q6600 with 4GB, Intel nic (igb from inel, latest, RSS
enabled 4 queue per RX,TX on both nic). Software is accel-ppp and
openl2tpd, ppp redirects to ifb(no ip is up on it, just shaping), ipset,
iptables.
If need more info - i can give it. Strange is - sometimes (mostly) i see
panic, but sometime it's just hangs. Only reset helps...
All IF'ses are IPV4. What on the users side - must be ipv4, but i think
not the only...
[-- Attachment #2: trap.txt --]
[-- Type: text/plain, Size: 6900 bytes --]
[ 6089.358502] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 6089.362482] IP: [<ffffffffa0264e24>] __nf_conntrack_confirm+0x2b4/0x480 [nf_conntrack]
[ 6089.362482] PGD 117822067 PUD 117823067 PMD 0
[ 6089.362482] Oops: 0002 [#1] SMP
[ 6089.362482] last sysfs file: /sys/devices/virtual/net/ppp6/uevent
[ 6089.362482] CPU 1
[ 6089.362482] Modules linked in: act_mirred act_skbedit cls_u32 sch_ingress arc4 ecb ppp_mppe l2tp_ppp l2tp_netlink l2tpd
[ 6089.362482]
[ 6089.362482] Pid: 0, comm: kworker/0:0 Not tainted 2.6.39-std-def-alt3 #1 ASUS RS100-E4/PI2/P5M2-M/RS100-E4
[ 6089.362482] RIP: 0010:[<ffffffffa0264e24>] [<ffffffffa0264e24>] __nf_conntrack_confirm+0x2b4/0x480 [nf_conntrack]
[ 6089.362482] RSP: 0018:ffff88011fc83a80 EFLAGS: 00010202
[ 6089.362482] RAX: 00000000000047cd RBX: ffff88011fc94620 RCX: 0000000000000000
[ 6089.362482] RDX: 000000000000fa10 RSI: 00000000d00dd719 RDI: ffffffffa026f680
[ 6089.362482] RBP: ffff88011fc83ac0 R08: 00000000e753847a R09: ffff880117f00000
[ 6089.362482] R10: 0000000000004000 R11: 0000000000000001 R12: 0000000000000000
[ 6089.362482] R13: ffff880116a95080 R14: ffffffff81a4e1c0 R15: 00000000000114d8
[ 6089.362482] FS: 0000000000000000(0000) GS:ffff88011fc80000(0000) knlGS:0000000000000000
[ 6089.362482] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 6089.362482] CR2: 0000000000000000 CR3: 0000000117821000 CR4: 00000000000006e0
[ 6089.362482] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 6089.362482] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 6089.362482] Process kworker/0:0 (pid: 0, threadinfo ffff880118ef6000, task ffff880118ef42c0)
[ 6089.362482] Stack:
[ 6089.362482] 000000007d09da80 0000400016bd4400 0000000116ea2000 ffff880116a95080
[ 6089.362482] 0000000000000001 ffff88011fc94620 0000000000000002 ffff880116ea2000
[ 6089.362482] ffff88011fc83b30 ffffffffa02b1dd8 ffff880116ea2000 0000000000000000
[ 6089.362482] Call Trace:
[ 6089.362482] <IRQ>
[ 6089.362482] [<ffffffffa02b1dd8>] ipv4_confirm+0x188/0x1c0 [nf_conntrack_ipv4]
[ 6089.362482] [<ffffffff81382004>] nf_iterate+0x84/0xa0
[ 6089.362482] [<ffffffff81389390>] ? ip_rcv_finish+0x390/0x390
[ 6089.362482] [<ffffffff81382096>] nf_hook_slow+0x76/0x130
[ 6089.362482] [<ffffffff81389390>] ? ip_rcv_finish+0x390/0x390
[ 6089.362482] [<ffffffff813897d7>] ip_local_deliver+0x67/0x90
[ 6089.362482] [<ffffffff81389135>] ip_rcv_finish+0x135/0x390
[ 6089.362482] [<ffffffff81389a1c>] ip_rcv+0x21c/0x2e0
[ 6089.362482] [<ffffffff81356f5a>] __netif_receive_skb+0x52a/0x690
[ 6089.362482] [<ffffffff813572d0>] netif_receive_skb+0x60/0x90
[ 6089.362482] [<ffffffff8124de9c>] ? is_swiotlb_buffer+0x3c/0x50
[ 6089.362482] [<ffffffff81357440>] napi_skb_finish+0x50/0x70
[ 6089.362482] [<ffffffff813579bd>] napi_gro_receive+0xbd/0xd0
[ 6089.362482] [<ffffffffa012129b>] igb_poll+0x6fb/0xae0 [igb]
[ 6089.362482] [<ffffffff8107fb61>] ? enqueue_hrtimer+0x31/0x80
[ 6090.302391] [<ffffffff81357be5>] net_rx_action+0x135/0x270
[ 6090.302391] [<ffffffff81062705>] __do_softirq+0xa5/0x1d0
[ 6090.302391] [<ffffffff8141301c>] call_softirq+0x1c/0x30
[ 6090.302391] [<ffffffff8100d355>] do_softirq+0x65/0xa0
[ 6090.302391] [<ffffffff81062a96>] irq_exit+0x86/0xa0
[ 6090.302391] [<ffffffff8100cf01>] do_IRQ+0x61/0xe0
[ 6090.302391] [<ffffffff8140a593>] common_interrupt+0x13/0x13
[ 6090.302391] <EOI>
[ 6090.302391] [<ffffffff81012deb>] ? mwait_idle+0x9b/0x1d0
[ 6090.302391] [<ffffffff8140de35>] ? atomic_notifier_call_chain+0x15/0x20
[ 6090.302391] [<ffffffff8100a1e6>] cpu_idle+0x56/0xa0
[ 6090.302391] [<ffffffff81403729>] start_secondary+0x197/0x19c
[ 6090.302391] Code: ff 74 09 40 0f b6 ff 41 0f b7 0c 38 66 41 39 cc 0f 84 f1 fe ff ff 48 8b 00 a8 01 0f 84 51 ff ff ff 4
[ 6090.302391] 89 01 75 04 48 89 48 08 48 8b 05 4c 4a 60 e1 48 01 83 98 00
[ 6090.302391] RIP [<ffffffffa0264e24>] __nf_conntrack_confirm+0x2b4/0x480 [nf_conntrack]
[ 6090.302391] RSP <ffff88011fc83a80>
[ 6090.302391] CR2: 0000000000000000
[ 6090.611999] ---[ end trace ded67c8afb62f164 ]---
[ 6090.625910] Kernel panic - not syncing: Fatal exception in interrupt
[ 6090.645022] Pid: 0, comm: kworker/0:0 Tainted: G D 2.6.39-std-def-alt3 #1
[ 6090.667822] Call Trace:
[ 6090.675232] <IRQ> [<ffffffff81407284>] panic+0x8c/0x197
[ 6090.691563] [<ffffffff8140b4d2>] oops_end+0xe2/0xf0
[ 6090.706514] [<ffffffff81039ad0>] no_context+0xf0/0x260
[ 6090.722242] [<ffffffff81382096>] ? nf_hook_slow+0x76/0x130
[ 6090.739013] [<ffffffff8138e0f0>] ? ip_fragment+0x960/0x960
[ 6090.755784] [<ffffffff81039d65>] __bad_area_nosemaphore+0x125/0x1e0
[ 6090.774894] [<ffffffff81039e2e>] bad_area_nosemaphore+0xe/0x10
[ 6090.792703] [<ffffffff8140dbf6>] do_page_fault+0x306/0x4b0
[ 6090.809473] [<ffffffff8106b48e>] ? mod_timer+0x15e/0x2c0
[ 6090.825725] [<ffffffff81345b47>] ? sk_reset_timer+0x17/0x30
[ 6090.842754] [<ffffffff81393eeb>] ? inet_csk_reset_keepalive_timer+0x1b/0x20
[ 6090.863945] [<ffffffff81393fe7>] ? inet_csk_reqsk_queue_hash_add+0xf7/0x110
[ 6090.885135] [<ffffffff8140a855>] page_fault+0x25/0x30
[ 6090.900608] [<ffffffffa0264e24>] ? __nf_conntrack_confirm+0x2b4/0x480 [nf_conntrack]
[ 6090.924189] [<ffffffffa02b1dd8>] ipv4_confirm+0x188/0x1c0 [nf_conntrack_ipv4]
[ 6090.945950] [<ffffffff81382004>] nf_iterate+0x84/0xa0
[ 6090.961420] [<ffffffff81389390>] ? ip_rcv_finish+0x390/0x390
[ 6090.978710] [<ffffffff81382096>] nf_hook_slow+0x76/0x130
[ 6090.994959] [<ffffffff81389390>] ? ip_rcv_finish+0x390/0x390
[ 6091.012250] [<ffffffff813897d7>] ip_local_deliver+0x67/0x90
[ 6091.029281] [<ffffffff81389135>] ip_rcv_finish+0x135/0x390
[ 6091.046052] [<ffffffff81389a1c>] ip_rcv+0x21c/0x2e0
[ 6091.061000] [<ffffffff81356f5a>] __netif_receive_skb+0x52a/0x690
[ 6091.079332] [<ffffffff813572d0>] netif_receive_skb+0x60/0x90
[ 6091.096622] [<ffffffff8124de9c>] ? is_swiotlb_buffer+0x3c/0x50
[ 6091.114431] [<ffffffff81357440>] napi_skb_finish+0x50/0x70
[ 6091.131201] [<ffffffff813579bd>] napi_gro_receive+0xbd/0xd0
[ 6091.148234] [<ffffffffa012129b>] igb_poll+0x6fb/0xae0 [igb]
[ 6091.165262] [<ffffffff8107fb61>] ? enqueue_hrtimer+0x31/0x80
[ 6091.182553] [<ffffffff81357be5>] net_rx_action+0x135/0x270
[ 6091.199323] [<ffffffff81062705>] __do_softirq+0xa5/0x1d0
[ 6091.215573] [<ffffffff8141301c>] call_softirq+0x1c/0x30
[ 6091.231562] [<ffffffff8100d355>] do_softirq+0x65/0xa0
[ 6091.247035] [<ffffffff81062a96>] irq_exit+0x86/0xa0
[ 6091.261983] [<ffffffff8100cf01>] do_IRQ+0x61/0xe0
[ 6091.276413] [<ffffffff8140a593>] common_interrupt+0x13/0x13
[ 6091.293442] <EOI> [<ffffffff81012deb>] ? mwait_idle+0x9b/0x1d0
[ 6091.311593] [<ffffffff8140de35>] ? atomic_notifier_call_chain+0x15/0x20
[ 6091.331742] [<ffffffff8100a1e6>] cpu_idle+0x56/0xa0
[ 6091.346692] [<ffffffff81403729>] start_secondary+0x197/0x19c
[ 6091.363986] Rebooting in 30 seconds..
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2011-07-21 15:30 UTC | newest]
Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-07-21 15:20 Kernel 2.6.38.3 panic and hangs on network traffic Rustam Afanasyev
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox