From: Junio C Hamano <gitster@pobox•com>
To: Johannes Sixt <j6t@kdbg•org>
Cc: Stefan Beller <sbeller@google•com>,
"git\@vger.kernel.org" <git@vger•kernel.org>,
Ramsay Jones <ramsay@ramsayjones•plus.com>,
Jacob Keller <jacob.keller@gmail•com>, Jeff King <peff@peff•net>,
Jonathan Nieder <jrnieder@gmail•com>,
Johannes Schindelin <johannes.schindelin@gmail•com>,
Jens Lehmann <Jens.Lehmann@web•de>,
Eric Sunshine <ericsunshine@gmail•com>
Subject: Re: [PATCHv3 02/11] run-command: report failure for degraded output just once
Date: Wed, 04 Nov 2015 13:01:53 -0800 [thread overview]
Message-ID: <xmqq4mh1a37i.fsf@gitster.mtv.corp.google.com> (raw)
In-Reply-To: <563A6C3D.2050805@kdbg.org> (Johannes Sixt's message of "Wed, 4 Nov 2015 21:36:13 +0100")
Johannes Sixt <j6t@kdbg•org> writes:
> I think that a scenario where A and B are communicating is rather
> far-fetched. We are talking about parallelizing independent tasks. I
> would not worry.
I wouldn't worry too much if this were merely a hack that is only
applicable to submodules, but I do not think it is a healthy
attitude to dismiss potential problem as far-fetched without
thinking things through, when you are designing what goes into
the run-command API.
I'd grant you that a complete deadlock is unlikely to be a problem
on its own. Somewhere somebody will eventually time out and unblock
the deadlock anyway.
But the symptom does not have to be as severe as a total deadlock to
be problematic. If we block B (and other tasks) by not reading from
them quickly because we are blocked on reading from A, which may
take forever (in timescale of B and other tasks) to feed us enough
to satisfy strbuf_read_once(), we are wasting resource by spawning B
(and other tasks) early when we are not prepared to service them
well, on both our end and on the other side of the connection.
By the way, A and B do not have to be directly communicating to
deadlock. The underlying system, either the remote end or the local
end or even a relaying system in between (think: network) can
throttle to cause the same symptom without A and B knowing (which
was the example I gave).
next prev parent reply other threads:[~2015-11-04 21:02 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-04 0:37 [PATCHv3 00/11] Expose the submodule parallelism to the user Stefan Beller
2015-11-04 0:37 ` [PATCHv3 01/11] run_processes_parallel: delimit intermixed task output Stefan Beller
2015-11-04 0:37 ` [PATCHv3 02/11] run-command: report failure for degraded output just once Stefan Beller
2015-11-04 18:14 ` Junio C Hamano
2015-11-04 20:14 ` Stefan Beller
2015-11-04 20:36 ` Johannes Sixt
2015-11-04 21:01 ` Junio C Hamano [this message]
2015-11-04 22:56 ` Jeff King
2015-11-05 2:05 ` Junio C Hamano
2015-11-05 6:51 ` Jeff King
2015-11-05 7:32 ` Junio C Hamano
2015-11-05 17:37 ` Stefan Beller
2015-11-04 20:42 ` Junio C Hamano
2015-11-04 21:04 ` Stefan Beller
2015-11-04 21:19 ` Junio C Hamano
2015-11-04 21:41 ` Stefan Beller
2015-11-04 0:37 ` [PATCHv3 03/11] run-command: omit setting file descriptors to non blocking in Windows Stefan Beller
2015-11-04 0:37 ` [PATCHv3 04/11] submodule-config: keep update strategy around Stefan Beller
2015-11-04 0:37 ` [PATCHv3 05/11] submodule-config: drop check against NULL Stefan Beller
2015-11-04 0:37 ` [PATCHv3 06/11] submodule-config: remove name_and_item_from_var Stefan Beller
2015-11-04 0:37 ` [PATCHv3 07/11] submodule-config: introduce parse_generic_submodule_config Stefan Beller
2015-11-04 0:37 ` [PATCHv3 08/11] fetching submodules: respect `submodule.jobs` config option Stefan Beller
2015-11-10 22:21 ` Jens Lehmann
2015-11-10 22:29 ` Stefan Beller
2015-11-11 19:55 ` Jens Lehmann
2015-11-11 23:34 ` Stefan Beller
2015-11-13 20:47 ` Jens Lehmann
2015-11-13 21:29 ` Stefan Beller
2015-11-04 0:37 ` [PATCHv3 09/11] git submodule update: have a dedicated helper for cloning Stefan Beller
2015-11-04 0:37 ` [PATCHv3 10/11] submodule update: expose parallelism to the user Stefan Beller
2015-11-04 0:37 ` [PATCHv3 11/11] clone: allow an explicit argument for parallel submodule clones Stefan Beller
2015-11-04 17:54 ` [PATCHv3 00/11] Expose the submodule parallelism to the user Junio C Hamano
2015-11-04 18:08 ` Stefan Beller
2015-11-04 18:17 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqq4mh1a37i.fsf@gitster.mtv.corp.google.com \
--to=gitster@pobox$(echo .)com \
--cc=Jens.Lehmann@web$(echo .)de \
--cc=ericsunshine@gmail$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=j6t@kdbg$(echo .)org \
--cc=jacob.keller@gmail$(echo .)com \
--cc=johannes.schindelin@gmail$(echo .)com \
--cc=jrnieder@gmail$(echo .)com \
--cc=peff@peff$(echo .)net \
--cc=ramsay@ramsayjones$(echo .)plus.com \
--cc=sbeller@google$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox