Subject: Re: UDP regression with packets rates < 10k per sec

From: Eric Dumazet <[email-masked]>
Date: Wed, 09 Sep 2009 00:52:41 +0200

Christoph Lameter a écrit :
> Looks like we have a regression since 2.6.22 due to latency increases in
> the network stack? The following is the result of measuring latencies for
> UDP multicast traffic at packet rates of 10pps 100pps 1kpps 10kpps and
> 100k pps. Two system running "mcast -n1 -r<rate>" (mcast tool from
> http://gentwo.org/ll).
> 
> Measurements in microseconds for one hop using bnx2 on Dell R610 (64 bit
> 2.6.31-rc9) and Dell 1950 (32 bit 2.6.22.19 3.3Ghz). Dell R610 RX usecs
> tuned to 0. 32 bit tuned to 1 (NIC is flaky at 0).
> 
> Kernel			10pps	100pps	1kpps	10kpps	100kpps
> ---------------------------------------------------------------
> 2.6.22 (32bit)		30	29.5	29	30	41
> 2.6.31-rc9(64 bit)	64	63	46	30	40
> 
> The only minor improvement is at a rate of 100kpps. All rates
> lower than 10k regress significantly.
> 
> Could there be something wrong with the bnx2 interrupt routing? They all
> end up on cpu0 here. There are 8 of them in a system with 16 "processors".
> How do those need to be configured? There are some sparse comments in
> Documentation/networking/multiqueue.txt but the text does not say anything
> about the irq routing.
> 
> 

Hi Christoph

In order to reproduce this here, could you tell me if you use 

Producer linux-2.6.22 -> Receiver 2.6.22
Producer linux-2.6.31 -> Receiver 2.6.31

Or a mix of
Producer linux-2.6.31 -> Receiver 2.6.22
Producer linux-2.6.22 -> Receiver 2.6.31-rc9

Not clear what is your exact setup

Thanks