Subject: Re: [PATCH 00/10 net-next] Introduce per interface ipv4 statistics

From: Stephen Hemminger <[email-masked]>
Date: Fri, 16 Dec 2011 09:19:23 -0800

On Fri, 16 Dec 2011 10:55:18 -0600 (CST)
Christoph Lameter <[email-masked]> wrote:

> On Fri, 16 Dec 2011, Eric Dumazet wrote:
> 
> > Or we could use a hierarchical split : Say 16 (or 32 or 64) cpus share
> > same counters (must be atomic if NR_CPUS > 16/32/64)
> 
> Then you'd need to have locking or full atomic operations for the
> counters. Reduction of network processing to a set of processors also will
> have other beneficial effects in addition to cache hotness. It would
> removing OS jitter etc etc.
> 
> > percpu_alloc() -> percpugroup_alloc()
> 
> Sounds like a per cpuset/cgroup allocation?

The problem is not per cpu usage for the traffic counters at the ipv4 level. The problem is
the multiplicative growth with 10,000 interfaces and 1024 cpus! Also, IPV6
was ridiculous with keeping for per-cpu counters for things that don't matter
the ICMP counters (thanks to Eric for addressing that one. Only a few
values in the ipstats mib are really in the hot path, others seem to be handled
that way only because the code is cleaner doing it that way.