Subject: Why the Makefile is so eager to re-build & re-link

From: Ævar Arnfjörð Bjarmason <[email-masked]>
Date: Thu, 24 Jun 2021 15:16:48 +0200

This is probably all stuff that's been on list-before / known by
some/all people in the CC list, but in case not: I looked a bit into why
we'e so frequently re-linking and re compiling things these days,
slowing down e.g. "git rebase --exec='make ...'".

These are all fixable issues, I haven't worked on them, just some notes
in case anyone has better ideas:

 * version.c: The biggest offender, every time you move to a new commit
   we have a FORCE on GIT-VERSION-FILE, if it changes (which it will, if
   you're in a git checkout), we in turn re-compile version.o, link
   libgit to contain it, and then need to re-link everything that uses
   libgit.

   Some of this can be micro-optimized by moving that out of libgit,
   e.g. we re-link git-shell, which doesn't ever report the version, we
   do the same for all the perl scripts, just because obscure codepaths
   of them that could shell out to "git version" want to report it, and
   we embed it in the generated code.

   But I think the best approach here is to piggy-back on
   SKIP_DASHED_BUILT_INS, if you have that enabled then only "git" (and
   git-upload-pack etc.) need to be aware of the version, which they can
   then set in the environment for the others.

   This also applies to e.g. the git-http* stuff, which has a user agent
   derived from the version.

 * {command,config}-list.h (and in-flight, my hook-list.h): Every time
   you touch a Documentation/git-*.txt we need to re-generate these, and
   since their mtime changes we re-compile and re-link all the way up to
   libgit and our other tools.

   I think the best solution here is to make the generate-*.sh
   shellscripts faster (just one takes ~300ms of nested shellscripting,
   just to grep out the first few lines of every git-*.txt, in e.g. Perl
   or a smarter awk script this would be <5ms).

   Then we make those FORCE, but most of the time the config or command
   summary (or list of hooks) doesn't change, so we don't need to
   replace the file.

Perhaps even better would be to piggy-back on the RUNTIME_PREFIX
support, and simply drop in generated plain-text files, so in your build
checkout the list of hooks, commands etc. would be parsed instead of
compiled in. Then we wouldn't need to re-build or re-link anything for
the version or this other data.

We could/should still have some facility to compile those in for
"install", see also [1] for some concerns / ideas for other similar
things that could use that.

1. https://lore.kernel.org/git/[email-masked]/