Subject: Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"

From: Junio C Hamano <[email-masked]>
Date: Tue, 13 Apr 2021 14:55:05 -0700

Matheus Tavares Bernardino <[email-masked]> writes:

> On Tue, Apr 13, 2021 at 4:01 PM Ævar Arnfjörð Bjarmason
> <[email-masked]> wrote:
>>
>> > Nit: before these two patches, "$actual" was only removed when the
>> > test succeeded. So, in case of failure, the failed output files would
>> > still be there for debugging. It might be interesting to keep this
>> > behavior and only remove "$actual" at the end of the test.
>>
>> Either I'm missing something or you are, that's how test_when_finished
>> works.
>>
>> It's skipped under e.g. "--immediate --debug". See b586744a864 (test:
>> skip clean-up when running under --immediate mode, 2011-06-27)
>
> I was mostly thinking about the `artifacts` zip we get from our CI
> when a test fails. I find the final trash dir quite useful for some
> post-mortem analysis, especially to debug WIP tests that only fail
> occasionally or test failures on OSes I don't have quick access to.

Yeah, without "--immediate" we do run the clean-up tasks registered
with test_when_finished, so it may be a problem.