Subject: Re: Stat cache in .git/index hinders syncing of repositories

From: Christoph Groth <[email-masked]>
Date: Sat, 18 Jan 2020 20:06:21 +0100

Junio C Hamano wrote:

> Christoph Groth <[email-masked]> writes:
>
> > However, there is one problem with syncing git repositories, that
> > has been noticed by multiple people [1]: The file .git/index
> > contains not only the “git index”, but also a cache of stat-data of
> > the files in the working directory.  Some file synchronizers are
> > able to sync mtimes, but syncing ctimes would be bizarre (if it is
> > even possible).
>
> The stat-data in the index file is meant to be a mere optimization,
> and after copying .git/index and the working tree files to a new box,
> running "git update-index --refresh" would make them in sync, no?

Let’s assume that one somehow manages to teach the synchronizer to run
"git update-index --refresh" immediately after copying the index file
and to consider the resulting modified .git/index file as identical to
the version on the other side.  (That would be already really difficult,
because it’s against the design of synchronizers to consider differing
files to be identical.)

Even then, .git/index will change again whenever the mtime of any
tracked file changes.  That can easily happen through "touch" or
modifying a file and then reverting the change.  There’s no way for the
synchronizer to tell the difference between relevant and irrelevant
changes to .git/index.  (Short of writing a specialized tool for
comparing git index files and embedding it.)

Would it be feasible to move this optimization data to a separate file
without breaking backwards and forwards compatibility?  I guess it
would: the data format of .git/index could remain identical, but newer
versions of git could ignore the stat data in the index file and use
a separate file for that.  If an older version of git is used with the
repository, it would simply notice that the stat data in .git/index is
not up-to-date.

But if the above is not feasible for some reason, would it be possible
to provide a switch for disabling stat caching optimization?

I believe that synchronizing files between machines is something that
will become ever more important with new tools like syncthing.  So it
would be really cool if git were to support such usage.  It *almost*
does!