From: Tariq Toukan <ttoukan.linux@gmail•com>
To: Eric Dumazet <edumazet@google•com>,
"David S . Miller" <davem@davemloft•net>
Cc: netdev <netdev@vger•kernel.org>,
Tariq Toukan <tariqt@mellanox•com>,
Martin KaFai Lau <kafai@fb•com>,
Willem de Bruijn <willemb@google•com>,
Jesper Dangaard Brouer <brouer@redhat•com>,
Brenden Blanco <bblanco@plumgrid•com>,
Alexei Starovoitov <ast@kernel•org>,
Eric Dumazet <eric.dumazet@gmail•com>
Subject: Re: [PATCH net-next 0/9] mlx4: order-0 allocations and page recycling
Date: Wed, 8 Feb 2017 11:02:01 +0200 [thread overview]
Message-ID: <029a2e2e-eb06-e452-e00e-56dc9c4f3a47@gmail.com> (raw)
In-Reply-To: <3b996d14-0471-1fa3-e686-15cf48db4c19@gmail.com>
On 07/02/2017 5:50 PM, Tariq Toukan wrote:
> Hi Eric,
>
> Thanks for your series.
>
> On 07/02/2017 5:02 AM, Eric Dumazet wrote:
>> As mentioned half a year ago, we better switch mlx4 driver to order-0
>> allocations and page recycling.
>>
>> This reduces vulnerability surface thanks to better skb->truesize
>> tracking
>> and provides better performance in most cases.
> The series makes significant change in the RX data-path, that requires
> deeper checks, in addition to code review.
> We applied your series and started running both our functional and
> performance regression.
> We will have results by tomorrow morning, and will analyze them during
> the day. I'll update about that.
We hit a kernel panic when running traffic after configuring a large MTU
(9000).
I will take deeper look into this soon and will keep you updated.
[56136.982183] BUG: unable to handle kernel paging request at
000000022f9e7020
[56136.990426] IP: mlx4_en_complete_rx_desc+0x130/0x2e0 [mlx4_en]
[56136.995303] PGD 220b7c067
[56136.995304] PUD 0
[56136.997941]
[56137.001807] Oops: 0000 [#1] SMP
[56137.004540] Modules linked in: netconsole mlx4_ib mlx4_en(E)
mlx4_core(E) nfsv3 nfs fscache rpcrdma ib_isert iscsi_target_mod ib_iser
libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp
scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm
dm_mirror dm_region_hash dm_log ib_cm dm_mod iw_cm ppdev parport_pc
i2c_piix4 sg virtio_balloon parport pcspkr acpi_cpufreq nfsd auth_rpcgss
nfs_acl lockd grace sunrpc ip_tables mlx5_ib sd_mod ata_generic
pata_acpi ib_core mlx5_core cirrus drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix floppy libata ptp
e1000 crc32c_intel virtio_pci pps_core serio_raw virtio_ring i2c_core
virtio [last unloaded: netconsole]
[56137.046028] CPU: 1 PID: 16 Comm: ksoftirqd/1 Tainted: G
E 4.10.0-rc6+ #26
[56137.051501] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[56137.055817] task: ffff880236245200 task.stack: ffffc90000d04000
[56137.060154] RIP: 0010:mlx4_en_complete_rx_desc+0x130/0x2e0 [mlx4_en]
[56137.064712] RSP: 0018:ffffc90000d07c90 EFLAGS: 00010282
[56137.068646] RAX: 0000000000000003 RBX: 000000022f9e7000 RCX:
ffff880234988880
[56137.073588] RDX: ffff8802349888e0 RSI: 0000000000000000 RDI:
ffff880235dad0a0
[56137.078563] RBP: ffffc90000d07ce0 R08: 0000000000000000 R09:
ffff8802225a08c0
[56137.083370] R10: ffff8802335c7800 R11: 0000000000000000 R12:
ffffc90001da1048
[56137.088123] R13: 0000000000000b36 R14: ffff8802225af040 R15:
0000000000000b36
[56137.092837] FS: 0000000000000000(0000) GS:ffff88023fc40000(0000)
knlGS:0000000000000000
[56137.098495] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[56137.102626] CR2: 000000022f9e7020 CR3: 000000023441f000 CR4:
00000000000006e0
[56137.107581] Call Trace:
[56137.109955] mlx4_en_process_rx_cq+0x35c/0xda0 [mlx4_en]
[56137.113894] ? mlx4_en_free_tx_desc+0x14e/0x350 [mlx4_en]
[56137.117992] ? load_balance+0x1ac/0x900
[56137.121285] mlx4_en_poll_rx_cq+0x30/0xa0 [mlx4_en]
[56137.125023] net_rx_action+0x23d/0x3a0
[56137.128146] __do_softirq+0xd1/0x2a2
[56137.131178] run_ksoftirqd+0x29/0x50
[56137.134180] smpboot_thread_fn+0x110/0x160
[56137.137530] kthread+0x101/0x140
[56137.140330] ? sort_range+0x30/0x30
[56137.143255] ? kthread_park+0x90/0x90
[56137.146304] ? __kthread_parkme+0x50/0x70
[56137.149466] ret_from_fork+0x2c/0x40
[56137.152426] Code: c0 8b 45 cc 41 8b 8a cc 00 00 00 48 63 d0 49 03 8a
d0 00 00 00 48 83 c2 03 48 c1 e2 04 48 01 ca 48 89 1a 44 89 5a 08 44 89
7a 0c <48> 8b 53 20 f6 c2 01 0f 85 76 01 00 00 48 89 da 48 83 7a 10 ff
[56137.164855] RIP: mlx4_en_complete_rx_desc+0x130/0x2e0 [mlx4_en] RSP:
ffffc90000d07c90
[56137.170211] CR2: 000000022f9e7020
[56137.175430] ---[ end trace 6a259f16967a0cff ]---
>>
>> Worth noting this patch series deletes more than 100 lines of code ;)
>>
>> Eric Dumazet (9):
>> mlx4: use __skb_fill_page_desc()
>> mlx4: dma_dir is a mlx4_en_priv attribute
>> mlx4: remove order field from mlx4_en_frag_info
>> mlx4: get rid of frag_prefix_size
>> mlx4: rx_headroom is a per port attribute
>> mlx4: reduce rx ring page_cache size
>> mlx4: removal of frag_sizes[]
>> mlx4: use order-0 pages for RX
>> mlx4: add page recycling in receive path
>>
>> drivers/net/ethernet/mellanox/mlx4/en_rx.c | 350
>> +++++++++------------------
>> drivers/net/ethernet/mellanox/mlx4/en_tx.c | 4 +-
>> drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 28 +--
>> 3 files changed, 129 insertions(+), 253 deletions(-)
>>
> Thanks,
> Tariq
next prev parent reply other threads:[~2017-02-08 9:12 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-02-07 3:02 [PATCH net-next 0/9] mlx4: order-0 allocations and page recycling Eric Dumazet
2017-02-07 3:02 ` [PATCH net-next 1/9] mlx4: use __skb_fill_page_desc() Eric Dumazet
2017-02-07 3:02 ` [PATCH net-next 2/9] mlx4: dma_dir is a mlx4_en_priv attribute Eric Dumazet
2017-02-07 3:02 ` [PATCH net-next 3/9] mlx4: remove order field from mlx4_en_frag_info Eric Dumazet
2017-02-07 3:02 ` [PATCH net-next 4/9] mlx4: get rid of frag_prefix_size Eric Dumazet
2017-02-09 12:28 ` Tariq Toukan
2017-02-09 14:06 ` Eric Dumazet
2017-02-07 3:02 ` [PATCH net-next 5/9] mlx4: rx_headroom is a per port attribute Eric Dumazet
2017-02-07 3:02 ` [PATCH net-next 6/9] mlx4: reduce rx ring page_cache size Eric Dumazet
2017-02-07 3:02 ` [PATCH net-next 7/9] mlx4: removal of frag_sizes[] Eric Dumazet
2017-02-07 3:02 ` [PATCH net-next 8/9] mlx4: use order-0 pages for RX Eric Dumazet
2017-02-07 3:02 ` [PATCH net-next 9/9] mlx4: add page recycling in receive path Eric Dumazet
2017-02-07 16:20 ` Tariq Toukan
2017-02-07 16:34 ` Eric Dumazet
2017-02-08 10:27 ` Tariq Toukan
2017-02-07 15:50 ` [PATCH net-next 0/9] mlx4: order-0 allocations and page recycling Tariq Toukan
2017-02-07 16:06 ` Eric Dumazet
2017-02-07 16:26 ` Eric Dumazet
2017-02-07 16:28 ` Eric Dumazet
2017-02-07 19:05 ` Alexei Starovoitov
2017-02-07 19:18 ` Eric Dumazet
2017-02-08 9:02 ` Tariq Toukan [this message]
2017-02-08 10:29 ` Tariq Toukan
2017-02-08 15:52 ` Eric Dumazet
2017-02-09 12:00 ` Tariq Toukan
2017-02-09 13:31 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=029a2e2e-eb06-e452-e00e-56dc9c4f3a47@gmail.com \
--to=ttoukan.linux@gmail$(echo .)com \
--cc=ast@kernel$(echo .)org \
--cc=bblanco@plumgrid$(echo .)com \
--cc=brouer@redhat$(echo .)com \
--cc=davem@davemloft$(echo .)net \
--cc=edumazet@google$(echo .)com \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=kafai@fb$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
--cc=tariqt@mellanox$(echo .)com \
--cc=willemb@google$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox