From: Junio C Hamano <gitster@pobox•com>
To: Stefan Beller <sbeller@google•com>
Cc: git@vger•kernel.org, ramsay@ramsayjones•plus.com,
jacob.keller@gmail•com, peff@peff•net, jrnieder@gmail•com,
johannes.schindelin@gmail•com, Jens.Lehmann@web•de,
ericsunshine@gmail•com, j6t@kdbg•org
Subject: Re: [PATCHv3 02/11] run-command: report failure for degraded output just once
Date: Wed, 04 Nov 2015 10:14:43 -0800 [thread overview]
Message-ID: <xmqqd1vpbpik.fsf@gitster.mtv.corp.google.com> (raw)
In-Reply-To: <1446597434-1740-3-git-send-email-sbeller@google.com> (Stefan Beller's message of "Tue, 3 Nov 2015 16:37:05 -0800")
Stefan Beller <sbeller@google•com> writes:
> The warning message is cluttering the output itself,
> so just report it once.
>
> Signed-off-by: Stefan Beller <sbeller@google•com>
> ---
> run-command.c | 20 ++++++++++++++------
> 1 file changed, 14 insertions(+), 6 deletions(-)
>
> diff --git a/run-command.c b/run-command.c
> index 7c00c21..3ae563f 100644
> --- a/run-command.c
> +++ b/run-command.c
> @@ -1012,13 +1012,21 @@ static void pp_cleanup(struct parallel_processes *pp)
>
> static void set_nonblocking(int fd)
> {
> + static int reported_degrade = 0;
> int flags = fcntl(fd, F_GETFL);
> - if (flags < 0)
> - warning("Could not get file status flags, "
> - "output will be degraded");
> - else if (fcntl(fd, F_SETFL, flags | O_NONBLOCK))
> - warning("Could not set file status flags, "
> - "output will be degraded");
> + if (flags < 0) {
> + if (!reported_degrade) {
> + warning("Could not get file status flags, "
> + "output will be degraded");
> + reported_degrade = 1;
> + }
> + } else if (fcntl(fd, F_SETFL, flags | O_NONBLOCK)) {
> + if (!reported_degrade) {
> + warning("Could not set file status flags, "
> + "output will be degraded");
> + reported_degrade = 1;
> + }
> + }
> }
Imagine that we are running two things A and B at the same time. We
ask poll(2) and it says both A and B have some data ready to be
read, and we try to read from A. strbuf_read_once() would try to
read up to 8K, relying on the fact that you earlier set the IO to be
nonblock. It will get stuck reading from A without allowing output
from B to drain. B's write may get stuck because we are not reading
from it, and would cause B to stop making progress.
What if the other sides of the connection from A and B are talking
with each other, and B's non-progress caused the processing for A on
the other side of the connection to block, causing it not to produce
more output to allow us to make progress reading from A (so that
eventually we can give B a chance to drain its output)? Imagine A
and B are pushes to the same remote, B may be pushing a change to a
submodule while A may be pushing a matching change to its
superproject, and the server may be trying to make sure that the
submodule update completes and updates the ref before making the
superproject's tree that binds that updated submodule's commit
availble, for example? Can we make any progress from that point?
I am not convinced that the failure to set nonblock IO is merely
"output will be degraded". It feels more like a fatal error if we
are driving more than one task at the same time.
next prev parent reply other threads:[~2015-11-04 18:14 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-04 0:37 [PATCHv3 00/11] Expose the submodule parallelism to the user Stefan Beller
2015-11-04 0:37 ` [PATCHv3 01/11] run_processes_parallel: delimit intermixed task output Stefan Beller
2015-11-04 0:37 ` [PATCHv3 02/11] run-command: report failure for degraded output just once Stefan Beller
2015-11-04 18:14 ` Junio C Hamano [this message]
2015-11-04 20:14 ` Stefan Beller
2015-11-04 20:36 ` Johannes Sixt
2015-11-04 21:01 ` Junio C Hamano
2015-11-04 22:56 ` Jeff King
2015-11-05 2:05 ` Junio C Hamano
2015-11-05 6:51 ` Jeff King
2015-11-05 7:32 ` Junio C Hamano
2015-11-05 17:37 ` Stefan Beller
2015-11-04 20:42 ` Junio C Hamano
2015-11-04 21:04 ` Stefan Beller
2015-11-04 21:19 ` Junio C Hamano
2015-11-04 21:41 ` Stefan Beller
2015-11-04 0:37 ` [PATCHv3 03/11] run-command: omit setting file descriptors to non blocking in Windows Stefan Beller
2015-11-04 0:37 ` [PATCHv3 04/11] submodule-config: keep update strategy around Stefan Beller
2015-11-04 0:37 ` [PATCHv3 05/11] submodule-config: drop check against NULL Stefan Beller
2015-11-04 0:37 ` [PATCHv3 06/11] submodule-config: remove name_and_item_from_var Stefan Beller
2015-11-04 0:37 ` [PATCHv3 07/11] submodule-config: introduce parse_generic_submodule_config Stefan Beller
2015-11-04 0:37 ` [PATCHv3 08/11] fetching submodules: respect `submodule.jobs` config option Stefan Beller
2015-11-10 22:21 ` Jens Lehmann
2015-11-10 22:29 ` Stefan Beller
2015-11-11 19:55 ` Jens Lehmann
2015-11-11 23:34 ` Stefan Beller
2015-11-13 20:47 ` Jens Lehmann
2015-11-13 21:29 ` Stefan Beller
2015-11-04 0:37 ` [PATCHv3 09/11] git submodule update: have a dedicated helper for cloning Stefan Beller
2015-11-04 0:37 ` [PATCHv3 10/11] submodule update: expose parallelism to the user Stefan Beller
2015-11-04 0:37 ` [PATCHv3 11/11] clone: allow an explicit argument for parallel submodule clones Stefan Beller
2015-11-04 17:54 ` [PATCHv3 00/11] Expose the submodule parallelism to the user Junio C Hamano
2015-11-04 18:08 ` Stefan Beller
2015-11-04 18:17 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqd1vpbpik.fsf@gitster.mtv.corp.google.com \
--to=gitster@pobox$(echo .)com \
--cc=Jens.Lehmann@web$(echo .)de \
--cc=ericsunshine@gmail$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=j6t@kdbg$(echo .)org \
--cc=jacob.keller@gmail$(echo .)com \
--cc=johannes.schindelin@gmail$(echo .)com \
--cc=jrnieder@gmail$(echo .)com \
--cc=peff@peff$(echo .)net \
--cc=ramsay@ramsayjones$(echo .)plus.com \
--cc=sbeller@google$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox