Subject: Re: [PATCH v14 bpf-next 00/18] mvneta: introduce XDP multi-buffer support

From: Toke Høiland-Jørgensen <[email-masked]>
Date: Mon, 20 Sep 2021 23:01:48 +0200

Jakub Kicinski <[email-masked]> writes:

> On Sat, 18 Sep 2021 13:53:35 +0200 Toke Høiland-Jørgensen wrote:
>> I'm OK with a bpf_header_pointer()-type helper - I quite like the
>> in-kernel version of this for SKBs, so replicating it as a BPF helper
>> would be great. But I'm a little worried about taking a performance hit.
>> 
>> I.e., if you do:
>> 
>> ptr = bpf_header_pointer(pkt, offset, len, stack_ptr)
>> *ptr = xxx;
>> 
>> then, if the helper ended up copying the data into the stack pointer,
>> you didn't actually change anything in the packet, so you need to do a
>> writeback.
>> 
>> Jakub suggested up-thread that this should be done with some kind of
>> flush() helper. But you don't know whether the header_pointer()-helper
>> copied the data, so you always need to call the flush() helper, which
>> will incur overhead. If the verifier can in-line the helpers that will
>> lower it, but will it be enough to make it negligible?
>
> Depends on the assumptions the program otherwise makes, right?
>
> For reading I'd expect a *layout-independent* TC program would 
> replace approximately:
>
> ptr = <some_ptr>;
> if (ptr + CONST >= md->ptr_end)
> 	if (bpf_pull_data(md, off + CONST))
> 		return DROP;
> 	ptr = <some_ptr>;
> 	if (ptr + CONST >= md->ptr_end)
> 		return DROP; /* da hell? */
> }
>
> With this (pre-inlining):
>
> ptr = bpf_header_pointer(md, offset, len, stack);
> if (!ptr)
> 	return DROP;
>
> Post-inlining (assuming static validation of args to prevent wraps):
>
> if (md->ptr + args->off + args->len < md->ptr_end)
> 	ptr = md->ptr + args->off;
> else
> 	ptr = __bpf_header_pointer(md, offset, len, stack);
> if (!ptr)
> 	return DROP;
>
> But that's based on guesswork so perhaps I'm off base.

Yeah, that's more or less what I was thinking...

> Regarding the flush() I was expecting that most progs will not modify
> the packet (or at least won't modify most headers they load) so no
> point paying the price of tracking changes auto-magically.

... but I guess my mental model assumed that packet writes would be just
as frequent as reads, in which case it would be a win if you could amend
your example like:

> In fact I don't think there is anything infra can do better for
> flushing than the prog itself:
>
> 	bool mod = false;
>
> 	ptr = bpf_header_pointer(...);
> 	...
> 	if (some_cond(...)) {
> 		change_packet(...);
> 		mod = true;
> 	}
> 	...
> 	if (mod)

to have an additional check like:

if (mod && ptr == stack)

(or something to that effect). No?

-Toke

> 		bpf_header_pointer_flush();
>
>
> is simple enough.. to me.