Subject: Re: [PATCH 4/4] fetch-pack: print and use dangling .gitmodules

From: Ævar Arnfjörð Bjarmason <[email-masked]>
Date: Thu, 18 Feb 2021 13:07:30 +0100

On Wed, Feb 17 2021, Jonathan Tan wrote:

>> Sorry for being unclear here. I don't think (honestly I don't remember,
>> it's been almost a month) that I meant to you should use the skipList.
>> 
>> Looking at that code again we use object_on_skiplist() to do an early
>> punt in report(), but also fsck_blob(), presumably you never want the
>> latter, and that early punting wouldn't be needed if your report()
>> function intercepted the modules blob id for stashing it away / later
>> reporting / whatever.
>> 
>> So yeah, I'm 99% sure now that's not what I meant :)
>> 
>> What I meant with:
>> 
>>     Or if we want to keep the "print <list> | process"[...]
>> 
>> Is that we have an existing ad-hoc IPC model for these commands in
>> passing along the skipList, which is made more complex because sometimes
>> the initial process reads the file, sometimes it passes it along as-is
>> to the child.
>> 
>> And then there's this patch that passes OIDs too, but through a
>> different mechanism.
>> 
>> I was suggesting that perhaps it made more sense to refactor both so
>> they could use the same mechanism, because we're potentially passing two
>> lists of OIDs between the two. Just one goes via line-at-a-time in the
>> output, the other via a config option on the command-line.
>
> Thanks for your explanation. I still think that they are quite different
> - skiplist is a user-written file containing a list of OIDs that will
> likely never change, whereas my list of dangling .gitmodules is a list
> of OIDs dynamically generated (and thus, always different) whenever a
> fetch is done. So I think it's quite reasonable to pass skiplist as a
> file name, and my list should be passed line-by-line.

Sure, but I'm not talking about passing it as a tempfile.

Yes, I suggested that in the third-to-last paragraph of [1] but then
went on to say that we could also move to some IPC mechanism where you
spew in the list of dangling .gitmodules, and we also spew in the
skipList and anything else we want to pass in.

I'm not saying this needs to be part of this series. But let me
rephrase:

We now have some combination of
{receive-pack,upload-pack,send-pack,fetch-pack,unpack-objects} that need
to communicate locally or pass data back & forth, passing data either
via a CLI option to read a file, packnames/refs on --stdin, or (now) a
single list of OIDs on stdout.

Let's say we don't just need to pass the .gitmodules OIDs, but also
e.g. .mailmap OIDs or whatever (due to some future vulnerability).

Would this IPC mechanism deal with that, or would we need to introduce a
breaking change (Re: my recently send mail about concurrent updates of
libexec programs)? Can we use soemething like pkt-line to talk back &
forth in an extensible way?

Not needed now, just food for thought...

1. https://lore.kernel.org/git/[email-masked]/