Subject: Re: Hardware Offload discussion WAS(Re: [PATCH net-next v12 00/15] Introducing P4TC (series 1)

From: Jakub Kicinski <[email-masked]>
Date: Sat, 2 Mar 2024 19:27:47 -0800

On Sat, 2 Mar 2024 09:36:53 -0500 Jamal Hadi Salim wrote:
> 2) Your point on:  "integrate later", or at least "fill in the gaps"
> This part i am probably going to mumble on. I am going to consider
> more than just doing ACLs/MAT via flower/u32 for the sake of
> discussion.
> True, "fill the gaps" has been our model so far. It requires kernel
> changes, user space code changes etc justifiably so because most of
> the time such datapaths are subject to standardization via IETF, IEEE,
> etc and new extensions come in on a regular basis.  And sometimes we
> do add features that one or two users or a single vendor has need for
> at the cost of kernel and user/control extension. Given our work
> process, any features added this way take a long time to make it to
> the end user.

What I had in mind was more of a DDP model. The device loads it binary
blob FW in whatever way it does, then it tells the kernel its parser
graph, and tables. The kernel exposes those tables to user space.
All dynamic, no need to change the kernel for each new protocol.

But that's different in two ways:
 1. the device tells kernel the tables, no "dynamic reprogramming"
 2. you don't need the SW side, the only use of the API is to interact
    with the device

User can still do BPF kfuncs to look up in the tables (like in FIB), 
but call them from cls_bpf.

I think in P4 terms that may be something more akin to only providing
the runtime API? I seem to recall they had some distinction...

> At the cost of this sounding controversial, i am going
> to call things like fdb, fib, etc which have fixed datapaths in the
> kernel "legacy". These "legacy" datapaths almost all the time have

The cynic in me sometimes thinks that the biggest problem with "legacy"
protocols is that it's hard to make money on them :)

> very strong user bases with strong infra tooling which took years to
> get in shape. So they must be supported. I see two approaches:
> -  you can leave those "legacy" ndo ops alone and not go via the tc
> ndo ops used by P4TC.
> -  or write a P4 program that looks _exactly_ like what current
> bridging looks like and add helpers to allow existing tools to
> continue to work via tc ndo and then phase out the "fixed datapath"
> ndos. This will take a long long time but it could be a goal.
> 
> There is another caveat: Often different vendor hardware has slightly
> different features which cant be exposed because either they are very
> specific to the vendor or it's just very hard to express with existing
> "legacy" without making intrusive changes. So we are going to be able
> to allow these vendors/users to expose as much or as little as is
> needed for a specific deployment without affecting anyone else with
> new kernel/user code.
> 
> On the "integrate later" aspect: That is probably because most of the
> times we want to avoid doing intrusive niche changes (which is
> resolvable with the above).