Subject: Re: [PATCH net-next] rps: overflow prevention for saturated cpus

From: Rick Jones <[email-masked]>
Date: Thu, 06 Dec 2012 14:25:17 -0800

On 12/06/2012 12:36 PM, Willem de Bruijn wrote:
> RPS and RFS balance load across cpus with flow affinity. This can
> cause local bottlenecks, where a small number or single large flow
> (DoS) can saturate one CPU while others are idle.
>
> This patch maintains flow affinity in normal conditions, but
> trades it for throughput when a cpu becomes saturated. Then, packets
> destined to that cpu (only) are redirected to the lightest loaded cpu
> in the rxqueue's rps_map. This breaks flow affinity under high load
> for some flows, in favor of processing packets up to the capacity
> of the complete rps_map cpuset in all circumstances.

I thought (one of) the ideas behind RFS at least was to give the CPU 
scheduler control over where network processing took place instead of it 
being dictated solely by the addressing.  I would have expected the CPU 
scheduler to migrate some work off the saturated CPU.  Or will this only 
affect RPS and not RFS?

Allowing individual flows to straddle the CPUs - won't that be somewhat 
like what happens in bonding with mode-rr in the outbound case - packet 
reordering evil?  What kind of workload is this targeting that calls for 
such intra-flow parallelism?

With respect to the examples given, what happens when it is TCP traffic 
rather than UDP?

happy benchmarking,

rick jones