Subject: Re: Git is not scalable with too many refs/*

From: Thomas Rast <[email-masked]>
Date: Fri, 9 Sep 2011 03:05:15 +0200

Martin Fick wrote:
> An update, I bisected it down to this commit:
> 
>   88a21979c5717e3f37b9691e90b6dbf2b94c751a
> 
>    fetch/pull: recurse into submodules when necessary
> 
> Since this can be disabled with the --no-recurse-submodules switch, I tried
> that and indeed, even with the latest 1.7.7rc it becomes fast (~8mins)
> again. The strange part about this is that the repository does not have any
> submodules. Anyway, I hope that this can be useful to others since it is a
> workaround which speed things up enormously. Let me know if you have any
> other tests that you want me to perform,

Jens should know about this, so let's Cc him.

I took a quick look and I'm guessing that there's at least one
quadratic behaviour: in check_for_new_submodule_commits(), I see

+       const char *argv[] = {NULL, NULL, "--not", "--all", NULL};
+       int argc = ARRAY_SIZE(argv) - 1;
+
+       init_revisions(&rev, NULL);

which means that the --all needs to walk all commits reachable from
all refs and flag them as uninteresting.  But that function is called
for every ref update, so IIUC the time spent is on the order of
#ref updates*#commits.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch