From: "Eric W. Biederman" <ebiederm@xmission•com>
To: Junio C Hamano <gitster@pobox•com>
Cc: "brian m. carlson" <sandals@crustytoothpaste•net>, git@vger•kernel.org
Subject: [PATCH v3 02/32] doc hash-function-transition: Augment compatObjectFormat with readCompatMap
Date: Mon, 11 Sep 2023 18:46:19 -0500 [thread overview]
Message-ID: <87ledcb7ec.fsf_-_@email.froward.int.ebiederm.org> (raw)
In-Reply-To: <87sf7kd5xg.fsf_-_@email.froward.int.ebiederm.org> (Eric W. Biederman's message of "Mon, 11 Sep 2023 11:35:07 -0500")
Deeply and fundamentally the plan is to only operate one one hash
function for the core of git, to use only one hash function for what
is stored in the repository.
To avoid requring a flag day to transition hash functions for naming
objects, and to support being able to access objects using legacy object
names a mapping functionality will be provided.
We want to provide user facing configuration that is robust enough
that it can accomodate multiple different scenarios on how git
evolves and how people use their repositories.
There are two different ways it is envisioned to use mapped object
ids. The first is to require every object in the repository to have a
mapping, so that pushes and pulls from repositories using a different
hash algorithm can work. The second is to have an incomplete mapping
of object ids so that old references to objects in emails, commit
messages, bug trackers and are usable in a read-only manner
with tools like "git show".
The first way fundamentally needs every object in the repository to
have a mapping, which requires the repository to be marked incompatible
for writes fron older versions of git. Thus the compatObjectFormat option
is placed in [extensions].
The ext2 family of filesystems has 3 ways of describing new features
compatible, read-only-compatible, and incompatible. The current git
configurtation has compat (any feature mentioned anywhere in the
configuration outside of [extensions] section), and incompatible (any
configuration inside of the [extensions] section. It would be nice to
have a read-only compatible section for the mandatory mapping
function. Would it be worth adding it now so that we have it for
future extensions?
Having a mapping that is just used in a read-only mode for looking up
old objects with old object ids will be needed post-transition. Such
a mode does not require computing the old hash function or even
support automatically writing any new mappings. So it is completely
safe to enable in a backwards compatible mode. Fort that let's
use core.readCompatMap to make it clear the mappings only read.
I have documented that both of the options readCompatMap and
compatObjectFormat can be specified multiple times if that is needed to
support the desired configuration of git.
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission•com>
---
My v2 version was just silly. Changing the name of the option in
the [extensions] section made practical sense. It was just me being
contrary for no good reason. I still think we should have an additional
option for reading old hashes and to document that we expect multiple of
these.
So here is my proposal for extending the documentation along those
lines.
Additionally just accepting the existing option name means I am not
bottlenecked for writing tests convert_object_file which is the
important part right now.
My apologies for all of the noise.
.../technical/hash-function-transition.txt | 37 +++++++++++++++++++
1 file changed, 37 insertions(+)
diff --git a/Documentation/technical/hash-function-transition.txt b/Documentation/technical/hash-function-transition.txt
index 4b937480848a..26dfc3138b3b 100644
--- a/Documentation/technical/hash-function-transition.txt
+++ b/Documentation/technical/hash-function-transition.txt
@@ -171,6 +171,43 @@ repository, instead producing an error message.
objectformat
compatobjectformat
+Configurate for a future hash function transition would be:
+
+ [core]
+ repositoryFormatVersion = 1
+ [extensions]
+ objectFormat = futureHash
+ compatObjectFormat = sha256
+ compatObjectFormat = sha1
+
+Or possibly:
+
+ [core]
+ repositoryFormatVersion = 1
+ readCompatMap = sha1
+ [extensions]
+ objectFormat = futureHash
+ compatObjectFormat = sha256
+
+Or post transition to futureHash:
+
+ [core]
+ repositoryFormatVersion = 1
+ readCompatMap = sha1
+ readComaptMap = sha256
+ [extensions]
+ objectFormat = futureHash
+
+The difference between compatObjectFormat and readCompatMap would be that
+compatObjectFormat would ask git to read existing maps, but would not ask
+git to write or create them. Which is enough to support looking up
+old oids post transition, when they are only needed to support
+references in commit logs, bug trackers, emails and the like.
+
+Meanwhile with compatObjectFormat set every object in the entire
+repository would be required to have a bi-directional mapping from the
+the mapped object format to the repositories storage hash function.
+
See the "Transition plan" section below for more details on these
repository extensions.
--
2.41.0
next prev parent reply other threads:[~2023-09-12 1:42 UTC|newest]
Thread overview: 59+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-08 23:05 [RFC][PATCH 0/32] SHA256 and SHA1 interoperability Eric W. Biederman
2023-09-08 23:10 ` [PATCH 01/32] doc hash-file-transition: A map file for mapping between sha1 and sha256 Eric W. Biederman
2023-09-10 14:24 ` brian m. carlson
2023-09-10 18:07 ` Eric W. Biederman
2023-09-12 0:14 ` brian m. carlson
2023-09-12 13:36 ` Eric W. Biederman
2023-09-08 23:10 ` [PATCH 02/32] doc hash-function-transition: Replace compatObjectFormat with compatMap Eric W. Biederman
2023-09-10 14:34 ` brian m. carlson
2023-09-10 18:00 ` Eric W. Biederman
2023-09-11 6:11 ` Junio C Hamano
2023-09-11 16:35 ` [PATCH v2 02/32] doc hash-function-transition: Replace compatObjectFormat with mapObjectFormat Eric W. Biederman
2023-09-11 23:46 ` Eric W. Biederman [this message]
2023-09-12 7:57 ` [PATCH v3 02/32] doc hash-function-transition: Augment compatObjectFormat with readCompatMap Oswald Buddenhagen
2023-09-12 12:11 ` Eric W. Biederman
2023-09-13 8:10 ` Oswald Buddenhagen
2023-09-08 23:10 ` [PATCH 03/32] object-file-convert: Stubs for converting from one object format to another Eric W. Biederman
2023-09-08 23:10 ` [PATCH 04/32] object-name: Initial support for ^{sha1} and ^{sha256} Eric W. Biederman
2023-09-08 23:10 ` [PATCH 05/32] repository: add a compatibility hash algorithm Eric W. Biederman
2023-09-08 23:10 ` [PATCH 06/32] repository: Implement core.compatMap Eric W. Biederman
2023-09-08 23:10 ` [PATCH 07/32] loose: add a mapping between SHA-1 and SHA-256 for loose objects Eric W. Biederman
2023-09-08 23:10 ` [PATCH 08/32] loose: Compatibilty short name support Eric W. Biederman
2023-09-08 23:10 ` [PATCH 09/32] object-file: Update the loose object map when writing loose objects Eric W. Biederman
2023-09-08 23:10 ` [PATCH 10/32] bulk-checkin: Only accept blobs Eric W. Biederman
2023-09-08 23:10 ` [PATCH 11/32] pack: Communicate the compat_oid through struct pack_idx_entry Eric W. Biederman
2023-09-08 23:10 ` [PATCH 12/32] bulk-checkin: hash object with compatibility algorithm Eric W. Biederman
2023-09-11 6:17 ` Junio C Hamano
2023-09-08 23:10 ` [PATCH 13/32] object-file: Add a compat_oid_in parameter to write_object_file_flags Eric W. Biederman
2023-09-08 23:10 ` [PATCH 14/32] commit: write commits for both hashes Eric W. Biederman
2023-09-11 6:25 ` Junio C Hamano
2023-09-08 23:10 ` [PATCH 15/32] cache: add a function to read an OID of a specific algorithm Eric W. Biederman
2023-09-08 23:10 ` [PATCH 16/32] object: Factor out parse_mode out of fast-import and tree-walk into in object.h Eric W. Biederman
2023-09-08 23:10 ` [PATCH 17/32] object-file-convert: add a function to convert trees between algorithms Eric W. Biederman
2023-09-08 23:10 ` [PATCH 18/32] object-file-convert: convert commit objects when writing Eric W. Biederman
2023-09-08 23:10 ` [PATCH 19/32] object-file-convert: convert tag commits " Eric W. Biederman
2023-09-08 23:10 ` [PATCH 20/32] builtin/cat-file: Let the oid determine the output algorithm Eric W. Biederman
2023-09-08 23:10 ` [PATCH 21/32] tree-walk: init_tree_desc take an oid to get the hash algorithm Eric W. Biederman
2023-09-08 23:10 ` [PATCH 22/32] object-file: Handle compat objects in check_object_signature Eric W. Biederman
2023-09-08 23:10 ` [PATCH 23/32] builtin/ls-tree: Let the oid determine the output algorithm Eric W. Biederman
2023-09-08 23:10 ` [PATCH 24/32] builtin/pack-objects: Communicate the compatibility hash through struct pack_idx_entry Eric W. Biederman
2023-09-08 23:10 ` [PATCH 25/32] pack-compat-map: Add support for .compat files of a packfile Eric W. Biederman
2023-09-11 6:30 ` Junio C Hamano
2023-10-05 18:14 ` Taylor Blau
2023-09-08 23:10 ` [PATCH 26/32] object-file-convert: Implement convert_object_file_{begin,step,end} Eric W. Biederman
2023-09-11 6:28 ` Junio C Hamano
2023-09-08 23:10 ` [PATCH 27/32] builtin/fast-import: compute compatibility hashs for imported objects Eric W. Biederman
2023-09-08 23:10 ` [PATCH 28/32] builtin/index-pack: Add a simple oid index Eric W. Biederman
2023-09-08 23:10 ` [PATCH 29/32] builtin/index-pack: Compute the compatibility hash Eric W. Biederman
2023-09-08 23:10 ` [PATCH 30/32] builtin/index-pack: Make the stack in compute_compat_oid explicit Eric W. Biederman
2023-09-08 23:10 ` [PATCH 31/32] unpack-objects: Update to compute and write the compatibility hashes Eric W. Biederman
2023-09-08 23:10 ` [PATCH 32/32] object-file-convert: Implement repo_submodule_oid_to_algop Eric W. Biederman
2023-09-09 12:58 ` [RFC][PATCH 0/32] SHA256 and SHA1 interoperability Eric W. Biederman
2023-09-10 15:38 ` brian m. carlson
2023-09-10 18:20 ` Eric W. Biederman
2023-09-11 6:37 ` Junio C Hamano
2023-09-11 16:13 ` Eric W. Biederman
2023-09-11 22:05 ` brian m. carlson
2023-09-12 21:19 ` Eric W. Biederman
2023-09-12 16:20 ` Junio C Hamano
2023-09-14 19:57 ` Eric W. Biederman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87ledcb7ec.fsf_-_@email.froward.int.ebiederm.org \
--to=ebiederm@xmission$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=gitster@pobox$(echo .)com \
--cc=sandals@crustytoothpaste$(echo .)net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox