From: Junio C Hamano <gitster@pobox•com>
To: Stefan Beller <sbeller@google•com>
Cc: git@vger•kernel.org
Subject: Re: [PATCH 05/12] git submodule update: Use its own list implementation.
Date: Fri, 16 Oct 2015 14:02:18 -0700 [thread overview]
Message-ID: <xmqqy4f2cysl.fsf@gitster.mtv.corp.google.com> (raw)
In-Reply-To: <1444960333-16003-6-git-send-email-sbeller@google.com> (Stefan Beller's message of "Thu, 15 Oct 2015 18:52:06 -0700")
Stefan Beller <sbeller@google•com> writes:
> Discussions turned out that we cannot parallelize the whole loop below
> `git submodule--helper list` in `git submodule update`, because some
> changes should be done only one at a time, such as messing up a submodule
> and leave it up to the user to cleanup the conflicted rebase or merge.
>
> The submodules which are need to be cloned however do not expect to create
> problems which require attention by the user one at a time, so we want to
> parallelize that first.
>
> To do so we will start with a literal copy of `git submodule--helper list`
> and port over features gradually.
I am not sure what you mean by this.
Surely, the current implementation of "update" does the fetching and
updating as a single unit of task and iterate over these tasks, and
we would rather want to instead have one iteration of submodules to
do the fetching part (without doing other things that can fail and
have to get attention of the end user), followed by another
iteration that does the "other things", in order to get closer to
the end goal of doing the fetch in parallel and then doing the
remainder one-module-at-a-time sequencially.
I would imagine that the logical first step towards the end goal, if
I understood you correctly, would be to split that single large loop
that does a fetch and other things for a single module in each
iteration into two, one that iterates and fetches all, followed by a
new one that does the checkout/merge.
What I do not understand is why that requires a different kind of
enumerator (unless this is a kind of premature optimization, knowing
that the set of modules iterated by these two loops are slightly
different or something).
> int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
> diff --git a/git-submodule.sh b/git-submodule.sh
> index bb8b2c7..d2d80e2 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -656,7 +656,7 @@ cmd_update()
> fi
>
> cloned_modules=
> - git submodule--helper list --prefix "$wt_prefix" "$@" | {
> + git submodule--helper list-or-clone --prefix "$wt_prefix" "$@" | {
> err=
> while read mode sha1 stage sm_path
> do
next prev parent reply other threads:[~2015-10-16 21:02 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-16 1:52 [RFC PATCHv1 00/12] git submodule update in C with parallel cloning Stefan Beller
2015-10-16 1:52 ` [PATCH 01/12] git submodule update: Announce skipping submodules on stderr Stefan Beller
2015-10-16 20:37 ` Junio C Hamano
2015-10-16 20:47 ` Stefan Beller
2015-10-16 1:52 ` [PATCH 02/12] git submodule update: Announce uninitialized modules " Stefan Beller
2015-10-16 20:54 ` Junio C Hamano
2015-10-16 1:52 ` [PATCH 03/12] git submodule update: Move branch calculation to where it's needed Stefan Beller
2015-10-16 20:54 ` Junio C Hamano
2015-10-16 1:52 ` [PATCH 04/12] git submodule update: Announce outcome of submodule operation to stderr Stefan Beller
2015-10-16 1:52 ` [PATCH 05/12] git submodule update: Use its own list implementation Stefan Beller
2015-10-16 21:02 ` Junio C Hamano [this message]
2015-10-16 21:08 ` Stefan Beller
2015-10-16 1:52 ` [PATCH 06/12] git submodule update: Handle unmerged submodules in C Stefan Beller
2015-10-20 21:11 ` Junio C Hamano
2015-10-20 21:21 ` Stefan Beller
2015-10-16 1:52 ` [PATCH 07/12] submodule config: keep update strategy around Stefan Beller
2015-10-16 1:52 ` [PATCH 08/12] git submodule update: check for "none" in C Stefan Beller
2015-10-16 1:52 ` [PATCH 09/12] git submodule update: Check url " Stefan Beller
2015-10-16 1:52 ` [PATCH 10/12] git submodule update: Clone projects from within C Stefan Beller
2015-10-16 1:52 ` [PATCH 11/12] submodule--helper: Do not emit submodules to process directly Stefan Beller
2015-10-16 1:52 ` [PATCH 12/12] WIP/broken Clone all outstanding submodules in parallel Stefan Beller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqy4f2cysl.fsf@gitster.mtv.corp.google.com \
--to=gitster@pobox$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=sbeller@google$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox