From: ebiederm@xmission•com (Eric W. Biederman)
To: Benjamin LaHaise <bcrl@lhnet•ca>
Cc: Eric Dumazet <eric.dumazet@gmail•com>,
Octavian Purdila <opurdila@ixiacom•com>,
netdev@vger•kernel.org, Cosmin Ratiu <cratiu@ixiacom•com>
Subject: Re: [PATCH] net: allow netdev_wait_allrefs() to run faster
Date: Thu, 29 Oct 2009 18:45:32 -0700 [thread overview]
Message-ID: <m1vdhxu01v.fsf@fess.ebiederm.org> (raw)
In-Reply-To: <20091029233848.GV3141@kvack.org> (Benjamin LaHaise's message of "Thu\, 29 Oct 2009 19\:38\:48 -0400")
Benjamin LaHaise <bcrl@lhnet•ca> writes:
> On Thu, Oct 29, 2009 at 04:07:18PM -0700, Eric W. Biederman wrote:
>> Could you keep me in the loop with that. I have some pending cleanups for
>> all of those pieces of code and may be able to help/advice/review.
>
> Here are the sysfs scaling improvements. I have to break them up, as there
> are 3 separate changes in this patch: 1. use an rbtree for name lookup in
> sysfs, 2. keep track of the number of directories for the purpose of
> generating the link count, as otherwise too much cpu time is spent in
> sysfs_count_nlink when new entries are added, and 3. when adding a new
> sysfs_dirent, walk the list backwards when linking it in, as higher
> numbered inodes tend to be at the end of the list, not the beginning.
The reason for the existence of sysfs_dirent is as things grow larger
we want to keep the amount of RAM consumed down. So we don't pin
everything in the dcache. So we try and keep the amount of memory
consumed down.
So I would like to see how much we can par down.
For dealing with seeks in the middle of readdir I expect the best way
to do that is to be inspired by htrees in extNfs and return a hash of
the filename as our position, and keep the filename list sorted by
that hash. Since we are optimizing for size we don't need to store
that hash. Then we can turn that list into a some flavor of sorted
binary tree.
I'm surprised sysfs_count_nlink shows up, as it is not directly on the
add or remove path. I think the answer there is to change s_flags
into a set of bitfields and make link_count one of them, perhaps
16bits long. If we ever overflow our bitfield we can just set link
count to 0, and userspace (aka find) will know it can't optimized
based on link count.
I was expecting someone to run into problems with the linear directory
of sysfs someday.
Eric
next prev parent reply other threads:[~2009-10-30 1:45 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-10-17 22:18 [PATCH/RFC] make unregister_netdev() delete more than 4 interfaces per second Benjamin LaHaise
2009-10-18 4:26 ` Eric Dumazet
2009-10-18 16:13 ` Benjamin LaHaise
2009-10-18 17:51 ` Eric Dumazet
2009-10-18 18:21 ` Benjamin LaHaise
2009-10-18 19:36 ` Eric Dumazet
2009-10-21 12:39 ` Octavian Purdila
2009-10-21 15:40 ` [PATCH] net: allow netdev_wait_allrefs() to run faster Eric Dumazet
2009-10-21 16:09 ` Eric Dumazet
2009-10-21 16:51 ` Benjamin LaHaise
2009-10-21 19:54 ` Eric Dumazet
2009-10-29 23:07 ` Eric W. Biederman
2009-10-29 23:38 ` Benjamin LaHaise
2009-10-30 1:45 ` Eric W. Biederman [this message]
2009-10-30 14:35 ` Benjamin LaHaise
2009-10-30 14:43 ` Eric Dumazet
2009-10-30 23:25 ` Eric W. Biederman
2009-10-30 23:53 ` Benjamin LaHaise
2009-10-31 0:37 ` Eric W. Biederman
2010-08-09 17:23 ` Ben Greear
2010-08-09 17:34 ` Benjamin LaHaise
2010-08-09 17:44 ` Ben Greear
2010-08-09 17:48 ` Benjamin LaHaise
2010-08-09 18:03 ` Ben Greear
2010-08-09 19:59 ` Eric W. Biederman
2010-08-09 21:03 ` Benjamin LaHaise
2010-08-09 21:17 ` Eric W. Biederman
2009-10-21 16:55 ` Octavian Purdila
2009-10-23 21:13 ` Paul E. McKenney
2009-10-24 4:35 ` Eric Dumazet
2009-10-24 5:49 ` Paul E. McKenney
2009-10-24 8:49 ` Eric Dumazet
2009-10-24 13:52 ` Paul E. McKenney
2009-10-24 14:24 ` Eric Dumazet
2009-10-24 14:46 ` Paul E. McKenney
2009-10-24 23:49 ` Octavian Purdila
2009-10-25 4:47 ` Paul E. McKenney
2009-10-25 8:35 ` Eric Dumazet
2009-10-25 15:19 ` Octavian Purdila
2009-10-25 19:28 ` Eric Dumazet
2009-10-24 20:22 ` Stephen Hemminger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=m1vdhxu01v.fsf@fess.ebiederm.org \
--to=ebiederm@xmission$(echo .)com \
--cc=bcrl@lhnet$(echo .)ca \
--cc=cratiu@ixiacom$(echo .)com \
--cc=eric.dumazet@gmail$(echo .)com \
--cc=netdev@vger$(echo .)kernel.org \
--cc=opurdila@ixiacom$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox