Subject: Re: [PATCH v2 1/2] t5411: use different out file to prevent overwriting

From: SZEDER Gábor <[email-masked]>
Date: Wed, 20 Jan 2021 13:49:06 +0100

On Tue, Jan 19, 2021 at 05:24:58AM -0500, Jiang Xin wrote:
> From: Jiang Xin <[email-masked]>
> 
> SZEDER reported that t5411 failed in Travis CI's s390x environment a
> couple of times, and could be reproduced with '--stress' test on this
> specific environment.  The test failure messages might look like this:
> 
>     + test_cmp expect actual
>     --- expect      2021-01-17 21:55:23.430750004 +0000
>     +++ actual      2021-01-17 21:55:23.430750004 +0000
>     @@ -1 +1 @@
>     -<COMMIT-A> refs/heads/main
>     +<COMMIT-A> refs/heads/maifatal: the remote end hung up unexpectedly
>     error: last command exited with $?=1
>     not ok 86 - proc-receive: not support push options (builtin protocol)
> 
> The file 'actual' is filtered from the file 'out' which contains result
> of 'git show-ref' command.  Due to the error messages from other process
> is written into the file 'out' accidentally, t5411 failed.  SZEDER finds
> the root cause of this issue:
> 
>  - 'git push' is executed with its standard output and error redirected
>    to the file 'out'.
> 
>  - 'git push' executes 'git receive-pack' internally, which inherits
>    the open file descriptors, so its output and error goes into that
>    same 'out' file.
> 
>  - 'git push' ends without waiting for the close of 'git-receive-pack'
>    for some cases, and the file 'out' is reused for test of
>    'git show-ref' afterwards.
> 
>  - A mixture of the output of 'git show-ref' abd 'git receive-pack'
>    leads to this issue.
> 
> The first intuitive reaction to resolve this issue is to remove the
> file 'out' after use, so that the newly created file 'out' will have a
> different file descriptor and will not be overwritten by the
> 'git receive-pack' process.  But Johannes pointed out that removing an
> open file is not possible on Windows.  So we use different temporary
> file names to store the output of 'git push' to solve this issue.
> 
> Reported-by: SZEDER Gábor <[email-masked]>
> Helped-by: Johannes Sixt <[email-masked]>
> Signed-off-by: Jiang Xin <[email-masked]>
> ---
>  t/t5411/test-0000-standard-git-push.sh        |  8 +--
>  .../test-0001-standard-git-push--porcelain.sh |  8 +--
>  t/t5411/test-0002-pre-receive-declined.sh     |  4 +-
>  ...st-0003-pre-receive-declined--porcelain.sh |  4 +-
>  t/t5411/test-0011-no-hook-error.sh            |  8 +--
>  t/t5411/test-0012-no-hook-error--porcelain.sh |  8 +--
>  t/t5411/test-0013-bad-protocol.sh             | 50 +++++++++----------
>  t/t5411/test-0014-bad-protocol--porcelain.sh  | 50 +++++++++----------
>  t/t5411/test-0020-report-ng.sh                |  8 +--
>  t/t5411/test-0021-report-ng--porcelain.sh     |  8 +--
>  t/t5411/test-0022-report-unexpect-ref.sh      |  4 +-
>  ...est-0023-report-unexpect-ref--porcelain.sh |  4 +-
>  t/t5411/test-0024-report-unknown-ref.sh       |  4 +-
>  ...test-0025-report-unknown-ref--porcelain.sh |  4 +-
>  t/t5411/test-0026-push-options.sh             |  4 +-
>  t/t5411/test-0027-push-options--porcelain.sh  |  4 +-
>  t/t5411/test-0032-report-with-options.sh      |  4 +-
>  ...est-0033-report-with-options--porcelain.sh |  4 +-
>  t/t5411/test-0038-report-mixed-refs.sh        |  4 +-
>  .../test-0039-report-mixed-refs--porcelain.sh |  4 +-
>  20 files changed, 98 insertions(+), 98 deletions(-)

Wow, this is a lot of churn.

Would it be possible/desirable to die gracefully when the receiving
end doesn't support push options?

> diff --git a/t/t5411/test-0000-standard-git-push.sh b/t/t5411/test-0000-standard-git-push.sh
> index 47b058af7e..b074417d4b 100644
> --- a/t/t5411/test-0000-standard-git-push.sh
> +++ b/t/t5411/test-0000-standard-git-push.sh
> @@ -35,11 +35,11 @@ test_expect_success "git-push --atomic ($PROTOCOL)" '
>  	test_must_fail git -C workbench push --atomic origin \
>  		main \
>  		$B:refs/heads/next \
> -		>out 2>&1 &&
> +		>out-0000-1 2>&1 &&
>  	filter_out_user_friendly_and_stable_output \
>  		-e "/^To / { p; }" \
>  		-e "/^ ! / { p; }" \
> -		<out >actual &&
> +		<out-0000-1 >actual &&
>  	cat >expect <<-EOF &&
>  	To <URL/of/upstream.git>
>  	 ! [rejected] main -> main (non-fast-forward)
> @@ -65,8 +65,8 @@ test_expect_success "non-fast-forward git-push ($PROTOCOL)" '
>  		push origin \
>  		main \
>  		$B:refs/heads/next \
> -		>out 2>&1 &&
> -	make_user_friendly_and_stable_output <out >actual &&
> +		>out-0000-2 2>&1 &&
> +	make_user_friendly_and_stable_output <out-0000-2 >actual &&

There will be a lot of sequential numbers this way, which might lead
to additional churn in the future, if we ever were to add more tests
somewhere in the middle.  However, our test framework does already
have a counter for test cases, so we could perhaps use that, i.e.
'out-$test_count', to make sure that each test case has its own output
file.