Subject: Re: Git is not scalable with too many refs/*

From: Martin Fick <[email-masked]>
Date: Mon, 26 Sep 2011 09:42:08 -0600

On Monday, September 26, 2011 09:32:14 am Michael Haggerty 
wrote:
> On 09/25/2011 10:43 PM, Martin Fick wrote:
> > A coworker of mine pointed out to me that a simple
> > 
> >   git checkout
> > 
> > can also take rather long periods of time > 3 mins when
> > run on a repo with ~100K refs.
> > 
> > While this is not massive like the other problem I
> > reported, it still seems like it is more than one
> > would expect.  So, I tried an older version of git,
> > and to my surprise/delight, it was much faster (.2s). 
> > So, I bisected this issue also, and it seems that the
> > "offending" commit is
> 
> > 680955702990c1d4bfb3c6feed6ae9c6cb5c3c07:
> I'm still working on changes to store references
> hierarchically in the cache and read them lazily.  I
> hope that it will help some scaling problems with large
> number of refs.
> 
> Unfortunately I keep getting tangled up in side issues,
> so it is taking a lot longer than expected.  But there's
> still hope.
> 
> Michael

Thanks Michael, I look forward to those changes.  In the 
meantime however, I will try to take advantage of the 
current inefficiencies of large ref counts to attempt to 
find places where there are obvious problems in the code 
paths.  I suspect that there are several commands in git 
which inadvertently scan all the refs when they probably 
shouldn't.  Since this is likely very slow now, it should be 
easy to find those, if it were faster, this might get 
overlooked.  I feel like git checkout is one of those cases, 
it does not seem like git checkout should be affected by the 
number of refs in a repo?

-Martin

-- 
Employee of Qualcomm Innovation Center, Inc. which is a 
member of Code Aurora Forum