From: ebiederm@xmission•com (Eric W. Biederman)
To: Tejun Heo <tj@kernel•org>
Cc: Ingo Molnar <mingo@elte•hu>,
Rusty Russell <rusty@rustcorp•com.au>,
Herbert Xu <herbert@gondor•apana.org.au>,
akpm@linux-foundation•org, hpa@zytor•com, brgerst@gmail•com,
cl@linux-foundation•org, travis@sgi•com,
linux-kernel@vger•kernel.org, steiner@sgi•com, hugh@veritas•com,
"David S. Miller" <davem@davemloft•net>,
netdev@vger•kernel.org,
Mathieu Desnoyers <mathieu.desnoyers@polymtl•ca>
Subject: Re: [PATCH] percpu: add optimized generic percpu accessors
Date: Wed, 21 Jan 2009 03:21:23 -0800 [thread overview]
Message-ID: <m1fxjcucp8.fsf@fess.ebiederm.org> (raw)
In-Reply-To: <4976B82E.1080002@kernel.org> (Tejun Heo's message of "Wed\, 21 Jan 2009 14\:52\:46 +0900")
Tejun Heo <tj@kernel•org> writes:
> Ingo Molnar wrote:
>> The larger point still remains: the kernel dominantly uses static percpu
>> variables by a margin of 10 to 1, so we cannot just brush away the static
>> percpu variables and must concentrate on optimizing that side with
>> priority. It's nice if the dynamic percpu-alloc side improves as well, of
>> course.
>
> Well, the infrequent usage of dynamic percpu allocation is in some
> part due to the poor implementation, so it's sort of chicken and egg
> problem. I got into this percpu thing because I wanted a percpu
> reference count which can be dynamically allocated and it sucked.
Counters are our other special case, and counters are interesting
because they are individually very small. I just looked and the vast
majority of the alloc_percpu users are counters.
I just did a rough count in include/linux/snmp.h and I came
up with 171*2 counters. At 8 bytes per counter that is roughly
2.7K. Or about two thirds of a 4K page.
What makes this is a challenge is that those counters are per network
namespace, and there are no static limits on the number of network
namespaces.
If we push the system and allocate 1024 network namespaces we wind up
needing 2.7MB per cpu, just for the SNMP counters.
Which nicely illustrates the principle that typically each individual
per cpu allocation is small, but with dynamic allocation we have the
challenge that number of allocations becomes unbounded and in some cases
could be large, while the typical per cpu size is likely to be very small.
I wonder if for the specific case of counters it might make sense to
simply optimize the per cpu allocator for machine word size allocations
and allocate each counter individually freeing us from the burden of
worrying about fragmentation.
The pain with the current alloc_percpu implementation when working
with counters is that it has to allocate an array with one entry
for each cpu to point at the per cpu data. Which isn't especially efficient.
Eric
next prev parent reply other threads:[~2009-01-21 11:21 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20090115183942.GA6325@elte.hu>
[not found] ` <20090116001200.GA9137@gondor.apana.org.au>
[not found] ` <20090116001544.GA11073@elte.hu>
2009-01-16 0:18 ` [PATCH] percpu: add optimized generic percpu accessors Herbert Xu
[not found] ` <200901170827.33729.rusty@rustcorp.com.au>
2009-01-16 22:08 ` Ingo Molnar
[not found] ` <200901201328.24605.rusty@rustcorp.com.au>
2009-01-20 6:25 ` Tejun Heo
2009-01-20 10:36 ` Ingo Molnar
[not found] ` <200901271213.18605.rusty@rustcorp.com.au>
2009-01-27 2:24 ` Tejun Heo
2009-01-27 13:13 ` Ingo Molnar
2009-01-27 23:07 ` Tejun Heo
2009-01-28 3:36 ` Tejun Heo
2009-01-28 8:12 ` Tejun Heo
2009-01-27 20:08 ` Christoph Lameter
2009-01-27 21:47 ` David Miller
2009-01-27 22:47 ` Rick Jones
2009-01-28 0:17 ` Luck, Tony
2009-01-28 16:48 ` Christoph Lameter
2009-01-28 17:15 ` Luck, Tony
2009-01-28 16:45 ` Christoph Lameter
2009-01-28 20:47 ` David Miller
2009-01-28 10:38 ` Rusty Russell
2009-01-28 10:56 ` Tejun Heo
2009-01-29 2:06 ` Rusty Russell
2009-01-31 6:11 ` Tejun Heo
2009-01-28 16:50 ` Christoph Lameter
2009-01-28 18:07 ` Mathieu Desnoyers
2009-01-29 18:33 ` Christoph Lameter
2009-01-29 18:48 ` H. Peter Anvin
2009-01-20 10:40 ` Ingo Molnar
2009-01-21 5:52 ` Tejun Heo
2009-01-21 10:05 ` Ingo Molnar
2009-01-21 11:21 ` Eric W. Biederman [this message]
2009-01-21 12:45 ` Stephen Hemminger
2009-01-21 14:13 ` Eric W. Biederman
2009-01-21 20:34 ` David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=m1fxjcucp8.fsf@fess.ebiederm.org \
--to=ebiederm@xmission$(echo .)com \
--cc=akpm@linux-foundation$(echo .)org \
--cc=brgerst@gmail$(echo .)com \
--cc=cl@linux-foundation$(echo .)org \
--cc=davem@davemloft$(echo .)net \
--cc=herbert@gondor$(echo .)apana.org.au \
--cc=hpa@zytor$(echo .)com \
--cc=hugh@veritas$(echo .)com \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=mathieu.desnoyers@polymtl$(echo .)ca \
--cc=mingo@elte$(echo .)hu \
--cc=netdev@vger$(echo .)kernel.org \
--cc=rusty@rustcorp$(echo .)com.au \
--cc=steiner@sgi$(echo .)com \
--cc=tj@kernel$(echo .)org \
--cc=travis@sgi$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox