Subject: Re: With big repos and slower connections, git clone can be hard to work with

From: Toon claes <[email-masked]>
Date: Thu, 13 Jun 2024 12:10:19 +0200

ellie <[email-masked]> writes:

> Sorry for again another total newcomer/outsider question:

Don't apologize for asking these questions, you're more than welcome.

> Is a bundle or pack file something any regular git HTTPS instance
> would naturally provide when setup the usual ways?

Yes and no. Bundle and packfile format can used in many places.
Packfiles are used to transfer a bunch of objects, or store them locally
in Git's object database. A bundle is a packfile, but with a leading
header describing refs. You can read about that at
https://git-scm.com/docs/gitformat-bundle.

> Like, if resume relied on that, would this work when following the
> standard smart HTTP setup procedure
> https://git-scm.com/book/en/v2/Git-on-the-Server-Smart-HTTP (sorry if
> I got the wrong link) and then git cloning from that? That would
> result in the best availability of such a resume feature, if it ever
> came to be.

As mentioned elsewhere in the thread, on clone (and fetch) the client
negotiates with the server which objects to download. Because the state
of the remote repository can change between clones, so will the result
of this negotiation. This means the content of the packfile sent over
might differ, which is disruptive for caching these files.

That's why the proposal of bundle URI or packfile URI is suggested. In
case of bundle URI, it will tell the client to download a pre-made
bundle before starting the negotiation. This bundle can be stored on a
CDN or whatever static HTTP(s) server. But it requires the server to
create it, store it, and tell the client about it. This is not something
that's builtin into Git itself at the moment.

This is not really related to the Smart HTTP protocol, because it can be
used over SSH as well. But when such file is stored on a regular HTTP
server, we can rely on resumable downloads. Only after that bundle is
downloaded, the client will start the negotiation with the server to get
missing objects and refs (which should be a small subset when the bundle
is recent).

--
Toon