Subject: Re: [PATCH net-next] selftests: forwarding: Add missing config entries

From: Jakub Kicinski <[email-masked]>
Date: Mon, 29 Jan 2024 18:31:42 -0800

On Mon, 29 Jan 2024 11:45:07 +0100 Petr Machata wrote:
> Jakub Kicinski <[email-masked]> writes:
> > Thanks a lot for fixing this stuff! The patch went into the
> > net-next-2024-01-26--18-00 branch we got: pass 94 / skip 2 / fail 15
> >
> > https://netdev.bots.linux.dev/contest.html?branch=net-next-2024-01-26--18-00&executor=vmksft-forwarding&pw-y=0
> >
> > Clicking thru a handful of the failures it looks like it's about a 50/50
> > split between timeouts and perf mismatch.   
> 
> Looking at some recent runs. A number of failures are probably due to
> the system failing to oversubscribe the interface with the tested
> qdiscs. That's sch_ets, sch_tbf_ets, sch_tbf_prio, sch_tbf_root,
> tc_police.
> 
> Not sure what to do about it. Maybe separate out heavy traffic tests,
> and add a make run_lotraf_tests?

Either that or the other way - express the expectation that the
environment is slow to the test. It came up in the "net-next is OPEN"
thread and also off-list. Perhaps we should discuss tomorrow on the
netdev call?

https://lore.kernel.org/all/[email-masked]/

To make sure I've done my homework I kicked off few more instances of
the tester metal-* and metal-*-dbg. They are running in AWS (AWS Linux,
again) but on bare metal. Former are identical to the previous ones,
just with KVM / HW virt support now, so those should really be all
green. The metal-*-dbg ones have kernel/configs/debug.config and
kernel/configs/x86_debug.config configed in, so "worst case slow".

> tc_actions started getting a passible deadlocking warning between Jan 27
> 00:37 and Jan 28 18:27:
> 
>     https://netdev-2.bots.linux.dev/vmksft-forwarding/results/438201/108-tc-actions-sh/
>     https://netdev-2.bots.linux.dev/vmksft-forwarding/results/438566/109-tc-actions-sh/
> 
> So either something landed that broke it, or the host kernel now has
> more debugging enabled, so it now gives a citation.

Hm. configs are identical.

$ git diff net-next-2024-01-26--18-00..net-next-2024-01-27--00-04 --stat 
 Documentation/devicetree/bindings/net/snps,dwmac.yaml            |  11 +++---
 Documentation/devicetree/bindings/net/starfive,jh7110-dwmac.yaml |  72 +++++++++++++++++++++++++++------------
 drivers/net/ethernet/amd/pds_core/adminq.c                       |  74 +++++++++++++++++++++++++++-------------
 drivers/net/ethernet/amd/pds_core/core.c                         | 130 +++++++++++++++++++++++++++++++++++++++++-----------------------------
 drivers/net/ethernet/amd/pds_core/core.h                         |   3 +-
 drivers/net/ethernet/amd/pds_core/debugfs.c                      |  12 ++++---
 drivers/net/ethernet/amd/pds_core/dev.c                          |  30 ++++++++++++-----
 drivers/net/ethernet/amd/pds_core/devlink.c                      |   3 +-
 drivers/net/ethernet/amd/pds_core/fw.c                           |   3 ++
 drivers/net/ethernet/amd/pds_core/main.c                         |  26 +++++++++++---
 drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.c      |  16 ++++-----
 drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_keys.c         |   9 ++---
 drivers/net/ethernet/mellanox/mlxsw/spectrum.c                   | 160 +++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------
 drivers/net/ethernet/mellanox/mlxsw/spectrum.h                   |  15 ++-------
 drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c               |  11 +++---
 drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_tcam.c          |  17 +++++-----
 drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c            |  15 +++++----
 drivers/net/ethernet/mellanox/mlxsw/spectrum_switchdev.c         |   8 ++---
 drivers/net/ethernet/stmicro/stmmac/Kconfig                      |   6 ++--
 drivers/net/ethernet/stmicro/stmmac/dwmac-starfive.c             |  32 +++++++++++++++---
 net/core/dev.c                                                   |  27 +++++++++------
 net/core/dev.h                                                   |   1 +
 net/ipv4/ip_output.c                                             |   9 +----
 tools/testing/selftests/net/config                               |  14 ++++----
 24 files changed, 430 insertions(+), 274 deletions(-)

No idea.

> ip6gre_inner_v6_multipath is just noisy? It failed the last run, but
> passed several before.

FWIW I made this: https://netdev.bots.linux.dev/flakes.html

> router_multicast and router get a complaint about a missing control
> socket. I think at first approximation they need:
> 
>     # mkdir -p /usr/local/var/run
> 
> But even then I'm getting a fail. This and the others seem to all be in
> IPv6 multicast.