Subject: Re: Is the sha256 object format experimental or not?

From: Jonathan Nieder <[email-masked]>
Date: Mon, 17 May 2021 22:32:51 -0700

Hi,

[email-masked] wrote:

> I think we should make one last breaking change for digests and not go
> with the existing SHA-256 implementation but instead switch to
> self-describing digests and digital signatures and rely on external
> tools that Git talks to using a standard protocol. We can maintain full
> backward compatibility and even support full round tripping using some
> of the similar techniques that Brian came up with.

Forgive my ignorance: can you describe what compatibility break you
mean?  Do you mean _removing_ support for gpgsig-sha256?  If so,
why --- couldn't you get the same benefit by introducing the new
functionality you're describing without getting rid of historical
functionality at the same time?

A nice thing about signatures is that they don't change the semantics
of the object.  So some future version of Git can remove support for
verifying them, if they turn out

By the way, to be clear, the hash-function-transition doc in
Documentation/technical/ is not by Brian alone.  It is the result of
collaboration by various people on list (see its git history for
details).

[...]
> object 04b871796dc0420f8e7561a895b52484b701d51a
> obj 0ED_zgYrQg584bCrqKPoUvxaQ5aMis0GtnW_NrZFTTxUlHLUOyp77LanoZEGV6ajhYGLGTaTfCIQhryovyeNFJuG
> type commit
> tag signedtag
> tagger C O Mitter <[email-masked]> 1465981006 +0000
> signtype openpgp
> sign LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0KVmVyc2lvbjogR251UEcgdjEKCmlRRWN
> CQUFCQWdBR0JRSlhZUmhPQUFvSkVHRUpMb1czSW5HSmtsa0lBSWNuaEw3UndFYi8rUWVYOWVua1
> hoeG4KcnhmZHFydldkMUs4MHNsMlRPdDhCZy9OWXdyVUJ3L1JXSitzZy9oaEhwNFd0dkUxSERHS
> GxrRXozeTExTGt1aAo4dFN4UzNxS1R4WFVHb3p5UEd1RTkwc0pmRXhoWmxXNGtuSVExd3QveVdx
> TSszM0U5cE40aHpQcUx3eXJkb2RzCnE4RldFcVBQVWJTSlhvTWJSUHcwNFM1anJMdFpTc1VXYlJ
> Zam1KQ0h6bGhTZkZXVzRlRmQzN3VxdUlhTFVCUzAKcmtDM0pyeDc0MjBqa0lwZ0ZjVEkyczYwdW
> hTUUx6Z2NDd2RBMnVrU1lJUm5qZy96RGtqOCszaC9HYVJPSjcyeApsWnlJNkhXaXhLSmtXdzhsR
> TlhQU9EOVRtVFc5c0ZKd2NWQXptQXVGWDJrVXJlRFVLTVpkdUdjb1JZR3BEN0U9Cj1qcFhhCi0t
> LS0tRU5EIFBHUCBTSUdOQVRVUkUtLS0tLQo
[...]
> I think a good move to make right now would be to add a general function
> for stripping out any number of named fields from objects and also
> stripping out in-body signatures found in tags. That way we can add
> support in today's Git for stripping out fields/data for things like
> creating/verifying the object digest and/or digital signature.

Can you say a little more about the user-facing model here?  How does
a user know whether the signature verification result they're looking
at describes the part of the object they care about or has stripped it
out?

[...]
> Let me try to lay out the case for making a breaking change to sha256
> right now that will future-proof repos going forward.
>
> It has been known for a few decades now that cryptography has a
> shelf-life.

Yes, this is a key assumption of the hash function transition.  It is
meant to be repeatable, so that we are not stuck on a particular
cryptographic hash.

[...]
>                                       The end result is that in high
> security software, SHA-256 is being replaced with SHA-3 and Blake2
> digests.

Do you mean that practice is drifting away from the conclusion of
https://www.imperialviolet.org/2017/05/31/skipsha3.html?  Where can I
read more?

It took a while to decide on sha256 as the hash for Git to use to
replace sha1.  The process involved useful feedback from Keccak team
and others, and I feel pretty comfortable with how thoroughly it was
discussed, though of course I wouldn't be surprised if the state of
cryptanalysis has changed in some way since then.

The front runners were from the SHA2, SHA3, and Blake2 families.  The
main factor that led to deciding on SHA2 is the wide availability of
efficient and trustworthy implementations, in hardware and software.
See https://lore.kernel.org/git/alpine.DEB.2.21.1.1706151122180.4200@virtualbox/#t
and https://lore.kernel.org/git/[email-masked]/#t
for some of the discussion that led there.

[...]
> 4. switch to "late binding", "self describing" cryptographic constructs.

As Junio mentioned, Git does not impose a requirement on the signature
algorithm used in a signature block, including the digest involved.
However, signing history typically involves signing object names, and
object names use a cryptographic hash for other reasons.  If we want
Git to stop using a content addressable object store, that would be a
more fundamental changes to its design.

Thanks and hope that helps,
Jonathan