From: Junio C Hamano <gitster@pobox•com>
To: Richard Maw <richard.maw@codethink•co.uk>
Cc: git@vger•kernel.org
Subject: Re: Some issues when trying to set up a shallow git mirror server
Date: Fri, 08 Jan 2016 13:37:02 -0800 [thread overview]
Message-ID: <xmqqd1tb21oh.fsf@gitster.mtv.corp.google.com> (raw)
In-Reply-To: <xmqq4mep5kyg.fsf@gitster.mtv.corp.google.com> (Junio C. Hamano's message of "Thu, 07 Jan 2016 10:00:07 -0800")
Junio C Hamano <gitster@pobox•com> writes:
> Richard Maw <richard.maw@codethink•co.uk> writes:
>
>> This is inconvenient for us,
>> as we were explicitly using refspecs which didn't force the fetch,
>> since we were using the "non fast-forward update" errors
>> to detect whether upstream force pushed important refs
>> which could be a sign of tampering.
>>
>> While the client doesn't have enough information
>> the server has those commits.
>> Would it make sense for the server to be able to tell the client
>> "trust me, that commit is a descendant of the previous one"?
>
> It does not in our security model, as you do not blindly trust the
> other side, whether you are a "client" or a "server".
Thinking about it more, I think my answer was flawed.
A client may ask the server to give a history, and before it accepts
the result, it must do its own consistency check. A client may send
its history, and before the server accepts it, the server must do
its own consistency check. This is the general principle around our
consistency model (it is not limited to "security", but it is more
about "correctness" in general).
While that consistency principle must hold everywhere in our system,
it does not mean a client cannot ask a server to do something whose
result it has to trust, at least to some degree, because there is
fundamentally no way to independenty verify the result.
I think what you were hinting falls into that category. The client
cannot verify that the new tip is a descendant of the old one
without having the full history between these two points, but
transfering that history would defeat the whole point of using a
shallowed history.
So in that sense I do think that such a protocol extension does make
sense. But it would involve some extra pieces of information that
need to be sent between the client and the server, not just "I'll
trust you".
In the current "git fetch" protocol, the sender gives the receiver a
list of refs, and for each of these refs, the object name it points
at. Then the receiver asks the sender to give it the objects that
appear in the history leading to a set of objects (aka "want"), and
tells the sender what objects it completely has already (aka
"have"). The idea is that the sender can exclude objects that are
reachable from "have"s when it enumerates the objects that need to
be sent. When the receiver is shallow, it also tells the sender
that its current history is truncated at such and such commits,
lacking things before them. After that exchange, the sender just
gives the receiver a packfile that contains the objects requested.
The receiver verifies that the packfile makes sense, e.g. it has all
the "want" objects it asked for, and the objects that they refer to
(recursively) are available, before updating its refs with (some of)
the "want" objects.
In the simplest case, for example, where you might have a single
refs/heads/master that currently points at O and the other side has
N at its refs/heads/master, the sender would say "I have N at
refs/heads/master" and the receiver would say "I want N, and I have
O". A shallow receiver may also say "I do not have history behind
O" and "I only want a history 1 commit deep". And the sender would
send objects that is necessary to satisfy the request, e.g. commit N
itself and the trees and blobs in commit N that are not common with
commit O.
Notice that nowhere in this exchange the receiver tells the sender
what it intends to do with the "want" objects?
Because the receiver does not say "I want N and I intend to replace
O I currently have refs/heads/master", there is no way for the
sender to say "trust me, N is a descendant of O". It simply does
not know if the receiver _cares_ how N and O are related with each
other.
So if you want to do this, a new protocol extension needs to allow
your updated sender (upload-pack) and receiver (fetch-pack) to work
more like this:
* The sender would advertise "I support that extension", while
giving the usual "here are my refs and its current values".
* The receiver would say "I want to use that extension", and to
each of its "want" (which usually consists of "want" followed by
an object name and nothing else), it optionally adds names of the
objects it wants to verify ancestry relationship with.
E.g. if you have O at the tip of the master branch and P at the
tip of the maint branch, the sender has N at both of these two
branches, and if you are updating your master and maint with
their master and maint, you would say something like "want N O P"
to tell the sender that you want history leading to N, and you
want to see if N is a descendant of O and if N is a descendant of
P.
* The receiver and the sender then does the usual "have"/"ack"
exchange, which does not have to change any behaviour with this
extension.
* Finally, when the sender sends out the resulting packfile, it
also has to tell the receiver which of the object pairs the
receiver asked it to check the ancestry relationship violate the
fast-forward rule. In the earlier example of fast-forwarding O
and P with N, where the receiver asked "want N O P", the receiver
asked to check object pairs <N, O> and <N, P>. If P fast-forwards
to N but O does not, then the sender would tell the receiver the
fact that "O does not fast forward to N".
With such an extension, your updated receiver can receive the
necessary objects to update your history to "N", but notice that it
would result in non-ff update to update master (that used to be O)
with the new commit N.
next prev parent reply other threads:[~2016-01-08 21:37 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-07 16:54 Some issues when trying to set up a shallow git mirror server Richard Maw
2016-01-07 18:00 ` Junio C Hamano
2016-01-08 10:19 ` Richard Maw
2016-01-08 10:44 ` Duy Nguyen
2016-01-08 10:52 ` Richard Maw
2016-01-08 21:37 ` Junio C Hamano [this message]
2016-01-11 15:51 ` Richard Maw
2016-01-12 18:29 ` Junio C Hamano
2016-01-13 11:37 ` Richard Maw
2016-01-13 17:14 ` Junio C Hamano
2016-01-13 17:43 ` Richard Maw
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqd1tb21oh.fsf@gitster.mtv.corp.google.com \
--to=gitster@pobox$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=richard.maw@codethink$(echo .)co.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox