From: Junio C Hamano <gitster@pobox•com>
To: Duy Nguyen <pclouds@gmail•com>
Cc: Philippe Vaucher <philippe.vaucher@gmail•com>,
Jonathan Nieder <jrnieder@gmail•com>, David Kastrup <dak@gnu•org>,
Christian Jaeger <chrjae@gmail•com>,
Git Mailing List <git@vger•kernel.org>
Subject: Re: git gc --aggressive led to about 40 times slower "git log --raw"
Date: Fri, 21 Feb 2014 09:47:36 -0800 [thread overview]
Message-ID: <xmqqzjlk4atj.fsf@gitster.dls.corp.google.com> (raw)
In-Reply-To: <CACsJy8AeZWPz=39ySQr9MrSUiLyJDQbs02sumS9VjbbGWzP9pw@mail.gmail.com> (Duy Nguyen's message of "Fri, 21 Feb 2014 06:35:06 +0700")
Duy Nguyen <pclouds@gmail•com> writes:
> For old projects, commits older than 1-2 years is probably less often
> accessed and could use some aggressive packing.
I used to repack older part of history manually with a deeper depth,
mark the result with the .keep bit, and then repack the whole thing
again to have the remainder in a shallower depth. Something like:
git rev-list --objects v1.5.3 |
git pack-objects --depth=128 --delta-base-offset pack
would give me the first pack (in real life, I would use a larger
window size like 4096), and then after placing the resulting .pack
and .idx files along with a .keep file in .git/objects/pack/,
running "git repack -a -d" to pack the rest.
> This still hits git-blame badly. We could even make sure all
> objects "on the blame surface" have short delta chain. But that
> may be pushing pack-objects too much.
Yes, you can do a similar trick by blaming all the paths that ever
existed in the project, parse its --porcelain output to learn all
the commits and paths involved, to find the objects that need
quicker access. Pack such objects in a pack with a shallow depth,
tentatively mark that pack with .keep, repack the remainder with a
deep depth, remove .keep from the first pack and mark the new pack
with .keep to prevent it from getting repacked, or something like
that.
next prev parent reply other threads:[~2014-02-21 17:47 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-02-18 7:25 git gc --aggressive led to about 40 times slower "git log --raw" Christian Jaeger
2014-02-18 8:55 ` David Kastrup
2014-02-18 9:45 ` Duy Nguyen
2014-02-18 10:25 ` David Kastrup
2014-02-18 15:59 ` Jonathan Nieder
2014-02-18 20:59 ` Junio C Hamano
2014-02-18 22:46 ` Duy Nguyen
2014-02-19 0:10 ` Junio C Hamano
2014-02-19 0:33 ` Duy Nguyen
2014-02-19 8:38 ` Philippe Vaucher
2014-02-19 9:01 ` David Kastrup
2014-02-19 10:24 ` Duy Nguyen
2014-02-19 10:14 ` Duy Nguyen
2014-02-20 4:09 ` Christian Jaeger
2014-02-20 16:48 ` David Kastrup
2014-02-20 17:06 ` David Kastrup
2014-02-20 18:07 ` David Kastrup
2014-02-19 18:59 ` Junio C Hamano
2014-02-20 23:35 ` Duy Nguyen
2014-02-21 0:32 ` Christian Jaeger
2014-02-21 17:36 ` Junio C Hamano
2014-02-21 5:09 ` Duy Nguyen
2014-02-21 17:47 ` Junio C Hamano [this message]
2014-02-24 9:27 ` Philippe Vaucher
2014-02-22 0:36 ` Duy Nguyen
2014-02-22 6:20 ` David Kastrup
2014-02-22 8:53 ` David Kastrup
2014-02-22 9:14 ` Duy Nguyen
2014-02-22 13:00 ` Duy Nguyen
2014-02-22 9:57 ` Andreas Schwab
2014-02-18 16:43 ` Christian Jaeger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqzjlk4atj.fsf@gitster.dls.corp.google.com \
--to=gitster@pobox$(echo .)com \
--cc=chrjae@gmail$(echo .)com \
--cc=dak@gnu$(echo .)org \
--cc=git@vger$(echo .)kernel.org \
--cc=jrnieder@gmail$(echo .)com \
--cc=pclouds@gmail$(echo .)com \
--cc=philippe.vaucher@gmail$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox