From: Daniel Axtens <dja@axtens•net>
To: "Chopra\, Manish" <Manish.Chopra@cavium•com>,
"netdev\@vger.kernel.org" <netdev@vger•kernel.org>
Cc: Eric Dumazet <eric.dumazet@gmail•com>,
Jason Wang <jasowang@redhat•com>, Pravin Shelar <pshelar@ovn•org>,
Marcelo Ricardo Leitner <marcelo.leitner@gmail•com>
Subject: RE: [PATCH v4 2/2] bnx2x: disable GSO where gso_size is too big for hardware
Date: Thu, 01 Feb 2018 08:11:27 +1100 [thread overview]
Message-ID: <87vafhag3k.fsf@linkitivity.dja.id.au> (raw)
In-Reply-To: <DM2PR07MB685966ACC7E64731C185B5F89FB0@DM2PR07MB685.namprd07.prod.outlook.com>
"Chopra, Manish" <Manish.Chopra@cavium•com> writes:
>> -----Original Message-----
>> From: Daniel Axtens [mailto:dja@axtens•net]
>> Sent: Wednesday, January 31, 2018 8:46 AM
>> To: netdev@vger•kernel.org
>> Cc: Daniel Axtens <dja@axtens•net>; Eric Dumazet <eric.dumazet@gmail•com>;
>> Chopra, Manish <Manish.Chopra@cavium•com>; Jason Wang
>> <jasowang@redhat•com>; Pravin Shelar <pshelar@ovn•org>; Marcelo Ricardo
>> Leitner <marcelo.leitner@gmail•com>
>> Subject: [PATCH v4 2/2] bnx2x: disable GSO where gso_size is too big for
>> hardware
>>
>> If a bnx2x card is passed a GSO packet with a gso_size larger than
>> ~9700 bytes, it will cause a firmware error that will bring the card
>> down:
>>
>> bnx2x: [bnx2x_attn_int_deasserted3:4323(enP24p1s0f0)]MC assert!
>> bnx2x: [bnx2x_mc_assert:720(enP24p1s0f0)]XSTORM_ASSERT_LIST_INDEX 0x2
>> bnx2x: [bnx2x_mc_assert:736(enP24p1s0f0)]XSTORM_ASSERT_INDEX 0x0 =
>> 0x00000000 0x25e43e47 0x00463e01 0x00010052
>> bnx2x: [bnx2x_mc_assert:750(enP24p1s0f0)]Chip Revision: everest3, FW
>> Version: 7_13_1 ... (dump of values continues) ...
>>
>> Detect when the mac length of a GSO packet is greater than the maximum
>> packet size (9700 bytes) and disable GSO.
>>
>> Signed-off-by: Daniel Axtens <dja@axtens•net>
>>
>> ---
>>
>> v4: Only call the slow check if the gso_size is large.
>> Eric - I think this is what you had in mind?
>> Manish - do you think this is an acceptable performance trade-off?
>> GSO will work for any packet size, and only jumbo frames will
>> have to do the slower test.
>>
>> Again, only build-tested.
>> ---
>> drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 18
>> ++++++++++++++++++
>> 1 file changed, 18 insertions(+)
>>
>> diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
>> b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
>> index 7b08323e3f3d..74fc9af4aadb 100644
>> --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
>> +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c
>> @@ -12934,6 +12934,24 @@ static netdev_features_t
>> bnx2x_features_check(struct sk_buff *skb,
>> struct net_device *dev,
>> netdev_features_t features)
>> {
>> + /*
>> + * A skb with gso_size + header length > 9700 will cause a
>> + * firmware panic. Drop GSO support.
>> + *
>> + * Eventually the upper layer should not pass these packets down.
>> + *
>> + * For speed, if the gso_size is <= 9000, assume there will
>> + * not be 700 bytes of headers and pass it through. Only do a
>> + * full (slow) validation if the gso_size is > 9000.
>> + *
>> + * (Due to the way SKB_BY_FRAGS works this will also do a full
>> + * validation in that case.)
>> + */
>> + if (unlikely(skb_is_gso(skb) &&
>> + (skb_shinfo(skb)->gso_size > 9000) &&
>> + !skb_gso_validate_mac_len(skb, 9700)))
>> + features &= ~NETIF_F_GSO_MASK;
>
> Hi Daniel,
>
> Obviously, it could be bad from performance perspective since every gso packet has to do these check.
> When running iperf/netperf performance benchmark, where GSO is likely to occur.
>
> Why do you have to put two checks for skb_is_gso() and gso_size ? Isn't gso_size > anything means GSO skb ?
Yes, the check is redundant. I do it to make my logic clearer.
The compiler will optimise it into one check. I compiled with gcc-7.2
and gcc-4.8, both targeting amd64, and in both cases I could only find
one relevant cmp:
skb_is_gso():
/home/dja/dev/linux/linux/include/linux/skbuff.h:4032
3686: 48 8b 8f d0 00 00 00 mov 0xd0(%rdi),%rcx
bnx2x_features_check():
/home/dja/dev/linux/linux/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c:12950
368d: 66 81 7c 01 04 28 23 cmpw $0x2328,0x4(%rcx,%rax,1)
3694: 0f 87 ea 01 00 00 ja 3884 <bnx2x_features_check+0x214>
0x2328 is decimal 9000.
> I assume it won't cause disabling the offload if "headers [L2 + L3 + L4] + gso_size" is still <= 9700. ?
That is correct. The flow is:
If the gso_size is less than or equal to 9000, offload is enabled with
no further checks. This is the fast path.
If the gso_size is greater than 9000, then the "headers [L2 + L3 + L4] +
gso_size" is checked. This is an out-of-line check done once per GSO skb.
If headers + gso_size is <= 9700, offload is enabled.
If headers + gso_size > 9700, offload is disabled.
Regards,
Daniel
>
> Thanks,
> Manish
next prev parent reply other threads:[~2018-01-31 21:11 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-31 3:15 [PATCH v4 0/2] bnx2x: disable GSO on too-large packets Daniel Axtens
2018-01-31 3:15 ` [PATCH v4 1/2] net: create skb_gso_validate_mac_len() Daniel Axtens
2018-01-31 3:15 ` [PATCH v4 2/2] bnx2x: disable GSO where gso_size is too big for hardware Daniel Axtens
2018-01-31 9:00 ` Chopra, Manish
2018-01-31 21:11 ` Daniel Axtens [this message]
2018-01-31 21:43 ` Eric Dumazet
2018-02-01 14:36 ` [PATCH v4 0/2] bnx2x: disable GSO on too-large packets David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87vafhag3k.fsf@linkitivity.dja.id.au \
--to=dja@axtens$(echo .)net \
--cc=Manish.Chopra@cavium$(echo .)com \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=jasowang@redhat$(echo .)com \
--cc=marcelo.leitner@gmail$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
--cc=pshelar@ovn$(echo .)org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox