Subject: RE: [PATCH V1 net-next 0/5] Add devlink support to ena

From: "Arinzon, David" <[email-masked]>
Date: Wed, 11 Jan 2023 21:21:16 +0000

> -----Original Message-----
> From: Jakub Kicinski <[email-masked]>
> Sent: Wednesday, January 11, 2023 10:00 PM
> To: Arinzon, David <[email-masked]>
> Cc: David Miller <[email-masked]>; [email-masked];
> Machulsky, Zorik <[email-masked]>; Matushevsky, Alexander
> <[email-masked]>; Bshara, Saeed <[email-masked]>; Bshara,
> Nafea <[email-masked]>; Saidi, Ali <[email-masked]>;
> Kiyanovski, Arthur <[email-masked]>; Dagan, Noam
> <[email-masked]>; Agroskin, Shay <[email-masked]>; Itzko,
> Shahar <[email-masked]>; Abboud, Osama <[email-masked]>
> Subject: RE: [EXTERNAL][PATCH V1 net-next 0/5] Add devlink support to
> ena
> 
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you can confirm the sender and
> know the content is safe.
> 
> 
> 
> On Wed, 11 Jan 2023 19:31:39 +0000 Arinzon, David wrote:
> > If the packet network headers are not within the size of the LLQ
> > entry, then the packet will be dropped. So I'll say that c) describes
> > the impact the best given that certain types of traffic will not succeed or
> have disruptions due to dropped TX packets.
> 
> I see. Sounds like it could be a fit for
> DEVLINK_ATTR_ESWITCH_INLINE_MODE ? But that one configures the
> depth of the headers copied inline, rather than bytes. We could add a value
> for "tunneled" and have that imply 256B LLQ in case of ena.
> 
> The other option is to introduce the concept of "max length of inline data"
> to ethtool, and add a new netlink attribute to ethtool -g.
> 
> Either way - the length of "inline|push" data is not a ena-specific concept,
> so using private knobs (devlink params or ethtool flags) is not appropriate
> upstream. We should add a bona fide uAPI for it.

I've looked into the original commit of DEVLINK_ATTR_ESWITCH_INLINE_MODE
and it seems to be more related to encapsulation and adding a tunneling concept
here would be misleading, as it doesn't cover all the use-cases affected here.
It is true that in case of tunneling and if there's a need to access the encapsulated
headers, the length of the headers might be larger than the default value in LLQ,
but this may be also the case for non-tunneled traffic, for example,
IPv6 with TCP SACK option.

I'll note again that this is not a configurable value, meaning, the only option is
to have 128B (standard) or 256B (large) LLQ, there's no option to set other values,
but only choose between the modes, therefore, I don't see how having such an
option through ethtool, as you suggested (setting the max length) can be
beneficial in our use-case (might be good overall, as you noted, it's a more
generic concept). Will put more thought into it.