Subject: Re: [PATCH] t1092: use GIT_PROGRESS_DELAY for consistent results

From: Junio C Hamano <[email-masked]>
Date: Tue, 25 May 2021 11:49:30 +0900

Taylor Blau <[email-masked]> writes:

> What would perhaps make more sense is to silence the progress meters
> from the commands themselves. AFAICT the only command called by
> run_on_sparse() which generates a progress meter is 'git checkout',
> 'git merge', and 'git submodule', all of which support '--no-progress'.
>
> Might it be worth passing that option instead of setting
> GIT_PROGRESS_DELAY to a large value?

Yup.  The progress meters are obvious source of variation, but there
is no guarantee that is the only source of variation.  The test
strategy to run the same command twice and judge only by observing
their stdout and stderr is, eh, suboptimal.  If we devise a sequence
of commands that are tested immediately followed by a sequence of
commands to check the outcome of these commands, and the output from
the latter for two runs are the same, then that is a more sensible
approach, as the latter "visualize the outcome of the commands that
are just tested" can be controlled more tightly to ignore meaningless
variations (like progress meters or base-delta paring in a resulting
packfile of a parallel packing) and focus on the aspects of the
outcome that matter.