Subject: Re: [PATCH v4 01/20] sparse-index: design doc and format update

From: SZEDER Gábor <[email-masked]>
Date: Fri, 26 Mar 2021 21:29:40 +0100

On Tue, Mar 23, 2021 at 01:44:09PM +0000, Derrick Stolee via GitGitGadget wrote:
> Currently, the index format is only updated in the presence of
> extensions.sparseIndex instead of increasing a file format version
> number. This is temporary, and index v5 is part of the plan for future
> work in this area.

> diff --git a/Documentation/technical/sparse-index.txt b/Documentation/technical/sparse-index.txt
> new file mode 100644
> index 000000000000..62f6dc225a44
> --- /dev/null
> +++ b/Documentation/technical/sparse-index.txt

> +To start, we use a new repository extension, `extensions.sparseIndex`, to
> +allow inserting sparse-directory entries into indexes with file format
> +versions 2, 3, and 4. This prevents Git versions that do not understand
> +the sparse-index from operating on one, but it also prevents other
> +operations that do not use the index at all.

Why is this not a non-optional index extension?  That would allow
older Git versions and other implementations not understanding sparse
directory entries to still perform any operation that doesn't involve
the index.  More importantly, that would prevent older Git versions
and other implementations not understanding repository extensions from
potentially wreaking havoc when the index contains sparse directory
entries.  Notably JGit's current version (5.11.0.202103091610-r) does
still completely ignore repository extensions, and e.g. happily
attempts any operations on a SHA256 repository, so it would do the
same in the presence of 'extensions.sparseIndex' as well.  JGit does
respect non-optional index extensions, see e.g. 87a6bb701a
(t5310-pack-bitmaps: make JGit tests work with GIT_TEST_SPLIT_INDEX,
2018-05-10).

This really should be a non-optional index extension.

> A new format, index v5, will
> +be introduced that includes sparse-directory entries by default. It might
> +also introduce other features that have been considered for improving the
> +index, as well.