Subject: Re: What's cooking in git.git (Sep 2023, #05; Fri, 15)

From: Taylor Blau <[email-masked]>
Date: Tue, 19 Sep 2023 14:32:00 -0400

On Fri, Sep 15, 2023 at 06:41:15PM -0700, Junio C Hamano wrote:
> * tb/repack-existing-packs-cleanup (2023-09-13) 8 commits
>   (merged to 'next' on 2023-09-14 at bb8065e89c)
>  + builtin/repack.c: extract common cruft pack loop
>  + builtin/repack.c: avoid directly inspecting "util"
>  + builtin/repack.c: store existing cruft packs separately
>  + builtin/repack.c: extract `has_existing_non_kept_packs()`
>  + builtin/repack.c: extract redundant pack cleanup for existing packs
>  + builtin/repack.c: extract redundant pack cleanup for --geometric
>  + builtin/repack.c: extract marking packs for deletion
>  + builtin/repack.c: extract structure to store existing packs
>
>  The code to keep track of existing packs in the repository while
>  repacking has been refactored.
>
>  Will merge to 'master'.
>  source: <[email-masked]>

Nice, I'm happy to see that this is moving along. Thanks, everybody, for
participating in the review :-). It's very nice to have a second set of
eyes when touching the repack machinery.

> * cc/repack-sift-filtered-objects-to-separate-pack (2023-09-11) 9 commits
>  . gc: add `gc.repackFilterTo` config option
>  . repack: implement `--filter-to` for storing filtered out objects
>  . gc: add `gc.repackFilter` config option
>  . repack: add `--filter=<filter-spec>` option
>  . pack-bitmap-write: rebuild using new bitmap when remapping
>  . repack: refactor finding pack prefix
>  . repack: refactor finishing pack-objects command
>  . t/helper: add 'find-pack' test-tool
>  . pack-objects: allow `--filter` without `--stdout`
>
>  "git repack" machinery learns to pay attention to the "--filter="
>  option.
>
>  May need to wait until tb/repack-existing-packs-cleanup stablizes.
>  source: <[email-masked]>

I sent Christian a draft of what I think the conflict resolution should
look like when rebasing on top of tb/repack-existing-packs-cleanup [1].

I've looked over all but one of the previous rounds, and have seen the
most recent round and am happy with the result. So I think that this
could go in relatively quickly following merging the above.

I have a couple of other repacking-related topics that are in my queue,
but I've been holding off on sending them until Christian's series has
stabilized. They are:

  - tb/cruft-max-size ([email-masked]:ttaylorr/git.git)
  - tb/cruft-preferred-inference ([email-masked]:ttaylorr/git.git)

The former is on the list at [2], and needs a bit of work before
queueing again. The latter isn't on the list, but fixes a performance
bug where it is possible in rare circumstances to select the cruft pack
as preferred when generating a MIDX bitmap during repacking.

The second isn't crucial to get in any time soon, but I would love to
get it off of my queue before we start cutting pre-releases. At this
rate, I doubt it will be a problem.

[1]: https://lore.kernel.org/git/[email-masked]/
[2]: https://lore.kernel.org/git/[email-masked]/

> * tb/path-filter-fix (2023-08-30) 15 commits
>  - bloom: introduce `deinit_bloom_filters()`
>  - commit-graph: reuse existing Bloom filters where possible
>  - object.h: fix mis-aligned flag bits table
>  - commit-graph: drop unnecessary `graph_read_bloom_data_context`
>  - commit-graph.c: unconditionally load Bloom filters
>  - t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`
>  - bloom: prepare to discard incompatible Bloom filters
>  - bloom: annotate filters with hash version
>  - commit-graph: new filter ver. that fixes murmur3
>  - repo-settings: introduce commitgraph.changedPathsVersion
>  - t4216: test changed path filters with high bit paths
>  - t/helper/test-read-graph: implement `bloom-filters` mode
>  - bloom.h: make `load_bloom_filter_from_graph()` public
>  - t/helper/test-read-graph.c: extract `dump_graph_info()`
>  - gitformat-commit-graph: describe version 2 of BDAT
>
>  The Bloom filter used for path limited history traversal was broken
>  on systems whose "char" is unsigned; update the implementation and
>  bump the format version to 2.
>
>  Needs more work.
>  cf. <[email-masked]>
>  source: <[email-masked]>

I think that Jonathan's most recent round of this is ready to get merged
up, cf. [3]. The outstanding issue you note in
<[email-masked]> can be addressed separately, I
believe. To that end, I have a RFC-level patch proposed here [4].

[3]: https://lore.kernel.org/git/[email-masked]/
[4]: https://lore.kernel.org/git/[email-masked]/

Thanks,
Taylor