Subject: Re: Need of advice for XDP sockets on top of the interfaces behind a Linux bonding device

From: Toke Høiland-Jørgensen <[email-masked]>
Date: Tue, 30 Jan 2024 15:32:06 +0100

Pavel Vazharov <[email-masked]> writes:

>> On Sat, Jan 27, 2024 at 7:08 AM Pavel Vazharov <[email-masked]> wrote:
>>>
>>> On Sat, Jan 27, 2024 at 6:39 AM Jakub Kicinski <[email-masked]> wrote:
>>> >
>>> > On Sat, 27 Jan 2024 05:58:55 +0200 Pavel Vazharov wrote:
>>> > > > Well, it will be up to your application to ensure that it is not. The
>>> > > > XDP program will run before the stack sees the LACP management traffic,
>>> > > > so you will have to take some measure to ensure that any such management
>>> > > > traffic gets routed to the stack instead of to the DPDK application. My
>>> > > > immediate guess would be that this is the cause of those warnings?
>>> > >
>>> > > Thank you for the response.
>>> > > I already checked the XDP program.
>>> > > It redirects particular pools of IPv4 (TCP or UDP) traffic to the application.
>>> > > Everything else is passed to the Linux kernel.
>>> > > However, I'll check it again. Just to be sure.
>>> >
>>> > What device driver are you using, if you don't mind sharing?
>>> > The pass thru code path may be much less well tested in AF_XDP
>>> > drivers.
>>> These are the kernel version and the drivers for the 3 ports in the
>>> above bonding.
>>> ~# uname -a
>>> Linux 6.3.2 #1 SMP Wed May 17 08:17:50 UTC 2023 x86_64 GNU/Linux
>>> ~# lspci -v | grep -A 16 -e 1b:00.0 -e 3b:00.0 -e 5e:00.0
>>> 1b:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
>>> SFI/SFP+ Network Connection (rev 01)
>>>        ...
>>>         Kernel driver in use: ixgbe
>>> --
>>> 3b:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
>>> SFI/SFP+ Network Connection (rev 01)
>>>         ...
>>>         Kernel driver in use: ixgbe
>>> --
>>> 5e:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit
>>> SFI/SFP+ Network Connection (rev 01)
>>>         ...
>>>         Kernel driver in use: ixgbe
>>>
>>> I think they should be well supported, right?
>>> So far, it seems that the present usage scenario should work and the
>>> problem is somewhere in my code.
>>> I'll double check it again and try to simplify everything in order to
>>> pinpoint the problem.
> I've managed to pinpoint that forcing the copying of the packets
> between the kernel and the user space
> (XDP_COPY) fixes the issue with the malformed LACPDUs and the not
> working bonding.

(+Magnus)

Right, okay, that seems to suggest a bug in the internal kernel copying
that happens on XDP_PASS in zero-copy mode. Which would be a driver bug;
any chance you could test with a different driver and see if the same
issue appears there?

-Toke