From: Theodore Tso <tytso@mit•edu>
To: Suparna Bhattacharya <suparna@in•ibm.com>
Cc: Andrew Morton <akpm@osdl•org>, Willy Tarreau <w@1wt•eu>,
Linus Torvalds <torvalds@osdl•org>,
"H. Peter Anvin" <hpa@zytor•com>,
git@vger•kernel.org, nigel@nigel•suspend2.net,
"J.H." <warthog9@kernel•org>,
Randy Dunlap <randy.dunlap@oracle•com>,
Pavel Machek <pavel@ucw•cz>,
kernel list <linux-kernel@vger•kernel.org>,
webmaster@kernel•org,
"linux-ext4@vger•kernel.org" <linux-ext4@vger•kernel.org>
Subject: Re: How git affects kernel.org performance
Date: Mon, 8 Jan 2007 07:58:19 -0500 [thread overview]
Message-ID: <20070108125819.GA32756@thunk.org> (raw)
In-Reply-To: <20070108030555.GA7289@in.ibm.com>
On Mon, Jan 08, 2007 at 08:35:55AM +0530, Suparna Bhattacharya wrote:
> > Yeah, slowly-growing directories will get splattered all over the disk.
> >
> > Possible short-term fixes would be to just allocate up to (say) eight
> > blocks when we grow a directory by one block. Or teach the
> > directory-growth code to use ext3 reservations.
> >
> > Longer-term people are talking about things like on-disk rerservations.
> > But I expect directories are being forgotten about in all of that.
>
> By on-disk reservations, do you mean persistent file preallocation ? (that
> is explicit preallocation of blocks to a given file) If so, you are
> right, we haven't really given any thought to the possibility of directories
> needing that feature.
The fastest and probably most important thing to add is some readahead
smarts to directories --- both to the htree and non-htree cases. If
you're using some kind of b-tree structure, such as XFS does for
directories, preallocation doesn't help you much. Delayed allocation
can save you if your delayed allocator knows how to structure disk
blocks so that a btree-traversal is efficient, but I'm guessing the
biggest reason why we are losing is because we don't have sufficient
readahead. This also has the advantage that it will help without
needing to doing a backup/restore to improve layout.
Allocating some number of empty blocks when we grow the directory
would be a quick hack that I'd probably do as a 2nd priority. It
won't help pre-existing directories, but combined with readahead
logic, should help us out greatly in the non-btree case.
- Ted
next prev parent reply other threads:[~2007-01-08 13:15 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20061214223718.GA3816@elf.ucw.cz>
[not found] ` <20061216094421.416a271e.randy.dunlap@oracle.com>
[not found] ` <20061216095702.3e6f1d1f.akpm@osdl.org>
[not found] ` <458434B0.4090506@oracle.com>
[not found] ` <1166297434.26330.34.camel@localhost.localdomain>
[not found] ` <1166304080.13548.8.camel@nigel.suspend2.net>
[not found] ` <459152B1.9040106@zytor.com>
[not found] ` <1168140954.2153.1.camel@nigel.suspend2.net>
2007-01-07 4:22 ` [KORG] Re: kernel.org lies about latest -mm kernel Jeff Garzik
2007-01-07 4:29 ` Linus Torvalds
2007-01-07 20:11 ` Greg KH
2007-01-07 21:30 ` H. Peter Anvin
2007-01-07 21:54 ` Junio C Hamano
2007-01-07 22:21 ` Jeff Garzik
2007-01-07 22:53 ` Linus Torvalds
2007-01-07 23:32 ` Martin Langhoff
[not found] ` <45A08269.4050504@zytor.com>
2007-01-07 5:24 ` How git affects kernel.org performance H. Peter Anvin
2007-01-07 5:39 ` Linus Torvalds
2007-01-07 8:55 ` Willy Tarreau
2007-01-07 8:58 ` H. Peter Anvin
2007-01-07 9:03 ` Willy Tarreau
2007-01-07 10:28 ` Christoph Hellwig
2007-01-07 10:52 ` Willy Tarreau
2007-01-07 18:17 ` Linus Torvalds
2007-01-07 19:13 ` Linus Torvalds
[not found] ` <9e4733910701071126r7931042eldfb73060792f4f41@mail.gmail.com>
2007-01-07 19:35 ` Linus Torvalds
2007-01-07 10:50 ` Jan Engelhardt
2007-01-07 18:49 ` Randy Dunlap
2007-01-07 19:07 ` Jan Engelhardt
2007-01-07 19:28 ` Randy Dunlap
2007-01-07 19:37 ` Linus Torvalds
2007-01-07 9:15 ` Andrew Morton
2007-01-07 9:38 ` Rene Herman
2007-01-08 3:05 ` Suparna Bhattacharya
2007-01-08 12:58 ` Theodore Tso [this message]
2007-01-08 13:41 ` Johannes Stezenbach
2007-01-08 13:56 ` Theodore Tso
2007-01-08 13:59 ` Pavel Machek
2007-01-08 14:17 ` Theodore Tso
2007-01-08 13:43 ` Jeff Garzik
2007-01-09 1:09 ` Paul Jackson
2007-01-09 2:18 ` Jeremy Higdon
[not found] ` <20070109075945.GA8799@mail.ustc.edu.cn>
2007-01-09 7:59 ` Fengguang Wu
2007-01-09 7:59 ` Fengguang Wu
2007-01-09 7:59 ` Fengguang Wu
2007-01-09 16:23 ` Linus Torvalds
[not found] ` <20070110015739.GA26978@mail.ustc.edu.cn>
2007-01-10 1:57 ` Fengguang Wu
2007-01-10 1:57 ` Fengguang Wu
2007-01-10 1:57 ` Fengguang Wu
2007-01-10 3:20 ` Nigel Cunningham
[not found] ` <20070110140730.GA986@mail.ustc.edu.cn>
2007-01-10 14:07 ` Fengguang Wu
2007-01-10 14:07 ` Fengguang Wu
2007-01-10 14:07 ` Fengguang Wu
2007-01-12 10:54 ` Nigel Cunningham
2007-01-07 14:57 ` Robert Fitzsimons
2007-01-07 19:12 ` J.H.
2007-01-08 1:51 ` Jakub Narebski
2007-01-07 15:06 ` Krzysztof Halasa
2007-01-07 20:31 ` Shawn O. Pearce
2007-01-08 14:46 ` Nicolas Pitre
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20070108125819.GA32756@thunk.org \
--to=tytso@mit$(echo .)edu \
--cc=akpm@osdl$(echo .)org \
--cc=git@vger$(echo .)kernel.org \
--cc=hpa@zytor$(echo .)com \
--cc=linux-ext4@vger$(echo .)kernel.org \
--cc=linux-kernel@vger$(echo .)kernel.org \
--cc=nigel@nigel$(echo .)suspend2.net \
--cc=pavel@ucw$(echo .)cz \
--cc=randy.dunlap@oracle$(echo .)com \
--cc=suparna@in$(echo .)ibm.com \
--cc=torvalds@osdl$(echo .)org \
--cc=w@1wt$(echo .)eu \
--cc=warthog9@kernel$(echo .)org \
--cc=webmaster@kernel$(echo .)org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox