From: Heiko Voigt <hvoigt@hvoigt•net>
To: Jens Lehmann <Jens.Lehmann@web•de>
Cc: Junio C Hamano <gitster@pobox•com>,
git@vger•kernel.org, Fredrik Gustafsson <iveqy@iveqy•com>
Subject: Re: [PATCH v5 3/3] push: teach --recurse-submodules the on-demand option
Date: Mon, 26 Mar 2012 21:33:34 +0200 [thread overview]
Message-ID: <20120326193334.GC41087@book.hvoigt.net> (raw)
In-Reply-To: <4F3C3172.8030505@web.de>
On Wed, Feb 15, 2012 at 11:28:02PM +0100, Jens Lehmann wrote:
> Am 14.02.2012 04:34, schrieb Junio C Hamano:
> > Heiko Voigt <hvoigt@hvoigt•net> writes:
> >
> >> diff --git a/submodule.c b/submodule.c
> >> index 3c714c2..ff0cfd8 100644
> >> --- a/submodule.c
> >> +++ b/submodule.c
> >> @@ -411,6 +411,54 @@ int check_submodule_needs_pushing(unsigned char new_sha1[20],
> >> return needs_pushing->nr;
> >> }
> >>
> >> +static int push_submodule(const char *path)
> >> +{
> >> + if (add_submodule_odb(path))
> >> + return 1;
> >> +
> >> + if (for_each_remote_ref_submodule(path, has_remote, NULL) > 0) {
> >> + struct child_process cp;
> >> + const char *argv[] = {"push", NULL};
> >> +
> >> + memset(&cp, 0, sizeof(cp));
> >> + cp.argv = argv;
> >> + cp.env = local_repo_env;
> >> + cp.git_cmd = 1;
> >> + cp.no_stdin = 1;
> >> + cp.dir = path;
> >> + if (run_command(&cp))
> >> + return 0;
> >> + close(cp.out);
> >> + }
> >> +
> >> + return 1;
> >> +}
> >
> > Hmm, this makes me wonder if we fire subprocesses and have them run in
> > parallel (to a reasonably limited parallelism), it might make the overall
> > user experience more pleasant, and if we did the same on the fetching
> > side, it would be even nicer.
>
> Yeah, I had the same idea and did some experiments when working on
> fetch some time ago.
>
> > We would need to keep track of children and after firing a handful of them
> > we would need to start waiting for some to finish and collect their exit
> > status before firing more, and at the end we would need to wait for the
> > remaining ones and find how each one of them did before returning from
> > push_unpushed_submodules(). If we were to do so, what are the missing
> > support we would need from the run_command() subsystem?
>
> We would not only have to collect the exit status but also the output
> lines. You don't want to see the output of multiple fetches or pushes
> mixed together, so it makes sense to just defer that until the command
> exited and then print everything at once. The interesting part I couldn't
> come up with an easy solution for is to preserve the output order between
> the stdout and stdin lines, as they contain different parts of the
> progress which would look strange when shuffled around.
>
> And I saw that sometimes parallel fetches took way longer than doing them
> sequentially (in my case because of strange DNS behavior of my DSL router),
> so we would definitely want a config option for that (maybe setting the
> maximum number of simultaneous threads to be used).
>
> But don't get me wrong, I'm all for having that feature! :-)
Me too, but I would suggest that we first implement the basic recursive
push functionality and then start to do optimizations like this. I think
we will definitely be interested in parallel pushing once we start using it.
Cheers Heiko
next prev parent reply other threads:[~2012-03-26 19:33 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-02-13 9:25 [PATCH v5 0/3] push: submodule support Heiko Voigt
2012-02-13 9:27 ` [PATCH v5 1/3] Teach revision walking machinery to walk multiple times sequencially Heiko Voigt
2012-02-14 1:33 ` Junio C Hamano
2012-03-26 19:32 ` Heiko Voigt
2012-03-26 21:28 ` Junio C Hamano
2012-02-13 9:29 ` [PATCH v5 2/3] Refactor submodule push check to use string list instead of integer Heiko Voigt
2012-02-14 3:28 ` Junio C Hamano
2012-03-26 19:33 ` Heiko Voigt
2012-03-26 19:55 ` Heiko Voigt
2012-03-26 21:29 ` Junio C Hamano
2012-02-13 9:30 ` [PATCH v5 3/3] push: teach --recurse-submodules the on-demand option Heiko Voigt
2012-02-14 3:34 ` Junio C Hamano
2012-02-15 22:28 ` Jens Lehmann
2012-03-26 19:33 ` Heiko Voigt [this message]
2012-05-13 14:47 ` [RFC/PATCH] read from 2 filedescriptors simultaneously into one strbuf Heiko Voigt
2012-03-26 21:22 ` [PATCH v5 3/3] push: teach --recurse-submodules the on-demand option Zbigniew Jędrzejewski-Szmek
2012-03-28 15:30 ` Heiko Voigt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120326193334.GC41087@book.hvoigt.net \
--to=hvoigt@hvoigt$(echo .)net \
--cc=Jens.Lehmann@web$(echo .)de \
--cc=git@vger$(echo .)kernel.org \
--cc=gitster@pobox$(echo .)com \
--cc=iveqy@iveqy$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox