Subject: [PATCH v3 00/24] Multi-pack-index (MIDX)

From: Derrick Stolee <[email-masked]>
Date: Thu, 5 Jul 2018 20:52:57 -0400

Thanks for the feedback on v2. I cleaned up the patch to respond to all
that feedback. I'll include the version diff in a follow-up email.

You can see the CI builds for Linux, Mac, and Windows linked from the
GitHub pull request [1].

Biggest changes in this version:

* Deleted the extra static method.

* Use $(...) instead of `...` in test script

* Fixed spaces in tabbing (hopefully I caught them all)

* Cleaned up some inter-patch diff issues

* `...` quotes in git-multi-pack-index.txt

* Replace FREE_AND_NULL() with free() when obvious that a variable is
  going out of scope

* Due to how Windows handles open file handles when replacing a
  lockfile, be sure to close the midx before writing a new one.

I do still want to revisit the "order by fanout" vs "merge sort"
discussion that we had on v1, but this series is very big already. I'd
like to come back to that as a follow-up.

Thanks,
-Stolee

[1] https://github.com/gitgitgadget/git/pull/5

Derrick Stolee (24):
  multi-pack-index: add design document
  multi-pack-index: add format details
  multi-pack-index: add builtin
  multi-pack-index: add 'write' verb
  midx: write header information to lockfile
  multi-pack-index: load into memory
  multi-pack-index: expand test data
  packfile: generalize pack directory list
  multi-pack-index: read packfile list
  multi-pack-index: write pack names in chunk
  midx: read pack names into array
  midx: sort and deduplicate objects from packfiles
  midx: write object ids in a chunk
  midx: write object id fanout chunk
  midx: write object offsets
  config: create core.multiPackIndex setting
  midx: prepare midxed_git struct
  midx: read objects from multi-pack-index
  midx: use midx in abbreviation calculations
  midx: use existing midx when writing new one
  midx: use midx in approximate_object_count
  midx: prevent duplicate packfile loads
  packfile: skip loading index if in multi-pack-index
  midx: clear midx on repack

 .gitignore                                   |   3 +-
 Documentation/config.txt                     |   4 +
 Documentation/git-multi-pack-index.txt       |  56 ++
 Documentation/technical/multi-pack-index.txt | 109 +++
 Documentation/technical/pack-format.txt      |  77 ++
 Makefile                                     |   3 +
 builtin.h                                    |   1 +
 builtin/multi-pack-index.c                   |  46 +
 builtin/repack.c                             |   8 +
 cache.h                                      |   1 +
 command-list.txt                             |   1 +
 config.c                                     |   5 +
 environment.c                                |   1 +
 git.c                                        |   1 +
 midx.c                                       | 896 +++++++++++++++++++
 midx.h                                       |  20 +
 object-store.h                               |  33 +
 packfile.c                                   | 169 +++-
 packfile.h                                   |   9 +
 sha1-name.c                                  |  70 ++
 t/helper/test-read-midx.c                    |  54 ++
 t/helper/test-tool.c                         |   1 +
 t/helper/test-tool.h                         |   1 +
 t/t5319-multi-pack-index.sh                  | 191 ++++
 24 files changed, 1717 insertions(+), 43 deletions(-)
 create mode 100644 Documentation/git-multi-pack-index.txt
 create mode 100644 Documentation/technical/multi-pack-index.txt
 create mode 100644 builtin/multi-pack-index.c
 create mode 100644 midx.c
 create mode 100644 midx.h
 create mode 100644 t/helper/test-read-midx.c
 create mode 100755 t/t5319-multi-pack-index.sh

base-commit: 53f9a3e157dbbc901a02ac2c73346d375e24978c
-- 
2.18.0.118.gd4f65b8d14