Subject: Re: BUG(?): igc link up and XDP program init fails

From: Vinicius Costa Gomes <[email-masked]>
Date: Fri, 25 Aug 2023 15:56:01 -0700

Hi Ferenc,

Ferenc Fejes <[email-masked]> writes:

> Dear igc Maintainers!
>
> I noticed that ip link set dev up fails with igc (Intel i225) driver
> when XDP program is attached to it. More precisely, only when we have
> incoming traffic and the incoming packet rate is too fast (like 100
> packets per-sec).
>
> I don't have a very smart reproducer, so 4 i225 cards are needed to
> trigger it. My setup (enp3s0 and enp4s0 directly connected with a
> cable, similarly enp6s0 and enp7s0).
>
> veth0 ----> veth1 --redir---> enp3s0 ~~~~~~~ enp4s0
> 			  |
>                           +-> enp6s0 ~~~~~~~ enp7s0
>
> ip link add dev type veth
> ip nei change 1.2.3.4 lladdr aa:aa:aa:aa:aa:aa dev veth0
> xdp-bench redirect-multi veth1 enp3s0 enp6s0	#in terminal 1
> xdpdump -i enp4s0				#in terminal 2
> ping -I veth0 1.2.3.4 -i 0.5 #slow packet rate  #in terminal 3
>

I was just able to reproduce this issue, with a different setup: 

|             System A                 |   System B   | 
veth0 ----> veth1 --redir---> "enp3s0" ~~~~~~~ "enp4s0"

And running xdp-bench like this:

$ xdp-bench redirect-multi veth1 enp3s0

Also I am running a different traffic generator.

> Now in a separate terminal do a "ip link set dev enp4s0 down" and "ip
> link set dev enp4s0 up". After a while, xdpdump will see the incoming
> packets.
>

It seems that anything that triggers a reset of the adapter would
trigger the bug: I am able to trigger the bug when I run 'xdp-bench'
last (after "ping"/traffic generator), no need for 'link down/link up'.

> Now in terminal 3, change the ping to a faster rate:
> ping -I veth0 1.2.3.4 -i 0.01
>
> And do the ip link down/up again. In my setup, I no longer see incoming
> packets. With bpftrace I see the driver keep trying to initialize
> itself in an endless loop.
>
> Now stop the ping, wait about 4-5 seconds, and start the ping again.
> This is enough time for the driver to initialize properly, and packets
> are visible in xdpdump again.
>
> If anyone has an idea what is wrong with my setup I would be happy to
> hear it, and I can help with testing fixes if this is indeed a bug.
> I have replicated the setup with veths and it looks fine.
>

I don't think there's anything wrong with your setup.

I am considering this a bug, I don't have any patches from the top of my
head for you to try, but taking a look.

Anyway, thanks for the report.

Cheers,
-- 
Vinicius