Subject: Re: [PATCH v2] t/Makefile: remove 'test-results' on 'make clean'

From: Ævar Arnfjörð Bjarmason <[email-masked]>
Date: Mon, 26 Sep 2022 11:08:55 +0200

On Wed, Sep 21 2022, Junio C Hamano wrote:

> SZEDER Gábor <[email-masked]> writes:
>
>> The 't/test-results' directory and its contents are by-products of the
>> test process, so 'make clean' should remove them, but, alas, this has
>> been broken since fee65b194d (t/Makefile: don't remove test-results in
>> "clean-except-prove-cache", 2022-07-28).
>
> What I find more disturbing is this is part of "leak check" topic,
> and I see no reason why we need to futz with "make clean" rule in
> the makefile while extending SANITIZE=leak support.  We really
> should yell louder when we see topics that tries to do too much
> "while at it" and reject them to minimize the risk of introducing
> this kind of breakage.

FWIW this wasn't a "while at it", it was critical to making another part
of that topic work at all.

It adds a "check" mode, which I use to check that the
TEST_PASSES_SANITIZE_LEAK=true markings are correct.

Intrinsic to how that works (and [1] has more details) is that we "pass"
all tests, when some of them really failed under SANITIZE=leak. I.e. a
failure of a test not opted in to TEST_PASSES_SANITIZE_LEAK=true is
considered OK.

Without fee65b194d that combined with GIT_TEST_SANITIZE_LEAK_LOG=true
added in the same topic would make post-test leak analysis I do useless,
because before fee65b194d the Makefile made the assumption that
"test-results" directories were only need in case the tests failed.

1. e92684e1a26 (test-lib: add a GIT_TEST_PASSING_SANITIZE_LEAK=check
   mode, 2022-07-28)