Subject: Re: [PATCH v2 1/3] test-lib: allow selecting tests by substring/regex with --run

From: Junio C Hamano <[email-masked]>
Date: Thu, 15 Oct 2020 13:08:22 -0700

Jeff King <[email-masked]> writes:

> On Thu, Oct 15, 2020 at 09:20:13AM -0700, Junio C Hamano wrote:
>
>> Jeff King <[email-masked]> writes:
>> 
>> > On Tue, Oct 13, 2020 at 07:19:44PM +0000, Elijah Newren via GitGitGadget wrote:
>> >
>> >> Many of our test scripts have several "setup" tests.  It's a lot easier
>> >> to say
>> >> 
>> >>    ./t0050-filesystem.sh --run=setup,9
>> >
>> > I like this direction very well.
>> >
>> > There was a small discussion recently that we might be better off
>> > dropping test script numbers entirely,...
>> 
>> I think I missed that one.  A pointer, if you have one handy?
>
> The sub-thread between me and Jonathan starting here:
>
>   https://lore.kernel.org/git/[email-masked]/
>
> but specifically:
>
>   https://lore.kernel.org/git/[email-masked]/

Ah, I see.  I actually do use "git ls-files t/ | grep ..." to look
for tests that are relevant to the issue I have at hand quite often,
so unlike what Jonathan said in the thread, having a good name does
matter to me.

As far as I can tell, the numbers in the test names serve only two
purposes.

One is as a unique key to avoid colliding in the test result
aggrevation database, and the other is as a unique key to use in
GIT_SKIP_TESTS (which in turn is used by the Meta/Make wrapper I
use, found on the todo branch, like 'Meta/Make --test=0050,1400
test').

I would be heavily inconvenienced if we decide to remove numbers
becuase it would rob the latter use case from me, but I'd survive if
we just are going to lift the requirement that numbers must be
unique.  I may end up running irrelevant 0050 and 1400 when the
tests I really want to run are the other 0050 and 1400 with
"--test=0050,1400", but when I am trying to run only 2 among 900+
scripts, running 2 extra ones I didn't have to run only because
their prefix collide is still much better and tolerable.

Thanks.