From: David Kastrup <dak@gnu•org>
To: Duy Nguyen <pclouds@gmail•com>
Cc: Philippe Vaucher <philippe.vaucher@gmail•com>,
Junio C Hamano <gitster@pobox•com>,
Jonathan Nieder <jrnieder@gmail•com>,
Christian Jaeger <chrjae@gmail•com>,
Git Mailing List <git@vger•kernel.org>
Subject: Re: git gc --aggressive led to about 40 times slower "git log --raw"
Date: Thu, 20 Feb 2014 18:06:44 +0100 [thread overview]
Message-ID: <87y515r9wb.fsf@fencepost.gnu.org> (raw)
In-Reply-To: <8738jdspbe.fsf@fencepost.gnu.org> (David Kastrup's message of "Thu, 20 Feb 2014 17:48:21 +0100")
David Kastrup <dak@gnu•org> writes:
> Duy Nguyen <pclouds@gmail•com> writes:
>
>> I can think of two improvements we could make, either increase cache
>> size dynamically (within limits) or make it configurable. If we have N
>> entries in worktree (both trees and blobs) and depth M, then we might
>> need to cache N*M objects for it to be effective. Christian, if you
>> want to experiment this, update MAX_DELTA_CACHE in sha1_file.c and
>> rebuild.
>
> Well, my optimized "git-blame" code is considerably hit by an
> aggressively packed Emacs repository so I took a look at it with the
> MAX_DELTA_CACHE value set to the default 256, and then 512, 1024, 2048.
[...]
> Trying with 16384:
> dak@lola:/usr/local/tmp/emacs$ time ../git/git blame src/xdisp.c >/dev/null
>
> real 2m8.000s
> user 0m54.968s
> sys 1m12.624s
>
> And memory consumption did not exceed about 200m all the while, so is
> far lower than what would have been available.
Of course, this has to do with delta_base_cache_limit defaulting to 16m.
> Something's _really_ fishy about that cache behavior. Note that the
> _system_ time goes up considerably, not just user time. Since the
> packs are zlib-packed, it's reasonable that more I/O time is also
> associated with more user time and it is well possible that the user
> time increase is entirely explainable by the larger amount of
> compressed data to access.
>
> But this stinks.
And an obvious contender for the stinking is that the "LRU" scheme used
here is _strictly_ freeing memory based on which cache entry has been
_created_ the longest time ago, not which cache entry has been
_accessed_ the longest time ago. Which means a pure round-robin
strategy for freeing memory rather than LRU.
Let's see what happens when changing this.
--
David Kastrup
next prev parent reply other threads:[~2014-02-20 17:06 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-02-18 7:25 git gc --aggressive led to about 40 times slower "git log --raw" Christian Jaeger
2014-02-18 8:55 ` David Kastrup
2014-02-18 9:45 ` Duy Nguyen
2014-02-18 10:25 ` David Kastrup
2014-02-18 15:59 ` Jonathan Nieder
2014-02-18 20:59 ` Junio C Hamano
2014-02-18 22:46 ` Duy Nguyen
2014-02-19 0:10 ` Junio C Hamano
2014-02-19 0:33 ` Duy Nguyen
2014-02-19 8:38 ` Philippe Vaucher
2014-02-19 9:01 ` David Kastrup
2014-02-19 10:24 ` Duy Nguyen
2014-02-19 10:14 ` Duy Nguyen
2014-02-20 4:09 ` Christian Jaeger
2014-02-20 16:48 ` David Kastrup
2014-02-20 17:06 ` David Kastrup [this message]
2014-02-20 18:07 ` David Kastrup
2014-02-19 18:59 ` Junio C Hamano
2014-02-20 23:35 ` Duy Nguyen
2014-02-21 0:32 ` Christian Jaeger
2014-02-21 17:36 ` Junio C Hamano
2014-02-21 5:09 ` Duy Nguyen
2014-02-21 17:47 ` Junio C Hamano
2014-02-24 9:27 ` Philippe Vaucher
2014-02-22 0:36 ` Duy Nguyen
2014-02-22 6:20 ` David Kastrup
2014-02-22 8:53 ` David Kastrup
2014-02-22 9:14 ` Duy Nguyen
2014-02-22 13:00 ` Duy Nguyen
2014-02-22 9:57 ` Andreas Schwab
2014-02-18 16:43 ` Christian Jaeger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87y515r9wb.fsf@fencepost.gnu.org \
--to=dak@gnu$(echo .)org \
--cc=chrjae@gmail$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=gitster@pobox$(echo .)com \
--cc=jrnieder@gmail$(echo .)com \
--cc=pclouds@gmail$(echo .)com \
--cc=philippe.vaucher@gmail$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox