Subject: Re: git slow unless piped to cat

From: SZEDER Gábor <[email-masked]>
Date: Tue, 27 Aug 2019 11:44:07 +0200

On Tue, Aug 27, 2019 at 11:12:06AM +0200, Philipp Zabel wrote:
> On Tue, 2019-08-27 at 10:56 +0200, Uwe Kleine-König wrote:
> > On Tue, Aug 27, 2019 at 10:41:11AM +0200, SZEDER Gábor wrote:
> > > On Tue, Aug 27, 2019 at 10:15:59AM +0200, Uwe Kleine-König wrote:
> > > > I have a problem here with git being slow in some situations.
> > > > Using git 2.23.0 (from Debian) the effect is:
> > > > 
> > > > [email-masked]:/ptx/src/git/linux.git$ sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"; time git show v5.2
> > > > tag v5.2
> > > > ...
> > > > 
> > > > real	0m12.727s
> > > > user	0m0.300s
> > > > sys	0m0.371s
> > > > 
> > > > But to get the actual data isn't the problem:
> > > > 
> > > > [email-masked]:/ptx/src/git/linux.git$ sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"; time git show v5.2 | cat
> > > > tag v5.2
> > > > ...
> > > > 
> > > > real	0m0.764s
> > > > user	0m0.014s
> > > > sys	0m0.020s
> > > > 
> > > 
> > > How does 'git --no-pager show v5.2' perform?  If it's as fast as the
> > > case piping the output to cat, then look into what pager and pager
> > > options you use.
> > 
> > 	[email-masked]:/ptx/src/git/linux.git$ sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"; time git --no-pager show v5.2
> > 	tag v5.2
> > 	...
> > 
> > 	real	0m13.225s
> > 	user	0m0.355s
> > 	sys	0m0.336s
> > 
> > So this doesn't seem to be the problem. Also the local configuration
> > can be ruled out:
> > 
> > 	[email-masked]:/ptx/src/git/linux.git$ sudo sh -c "echo 3 > /proc/sys/vm/drop_caches"; time env GIT_CONFIG_NOSYSTEM=1 HOME=/nonexistant XDG_CONFIG_HOME=/nonexistant git --no-pager show --no-color v5.2
> > 	tag v5.2
> > 	...
> > 
> > 	real	0m13.587s
> > 	user	0m0.335s
> > 	sys	0m0.336s
> > 
> > Thanks
> > Uwe
> 
> Have you checked strace output? I see a directory walk through .git/refs
> and .git/packed-refs if the output is not redirected.

That's a good point, 'git show/log' show decorations (refs pointing to
any commits shown) when the output is a terminal:

  $ git show v5.2
  [...]
  commit 0ecfebd2b52404ae0c54a878c872bb93363ada36 (HEAD -> master, tag: v5.2)
  [...]
  $ git show v5.2 |cat
  [...]
  commit 0ecfebd2b52404ae0c54a878c872bb93363ada36
  [...]

And indeed, if you have a lot of refs and cold cache, then that might
account for a couple of seconds difference.  So, how does 'git show
--no-decorate v5.2' perform, and if it performs well, then how many
refs do you have ('git for-each-ref |wc -l')?