Subject: Re: [PATCH v2 16/19] parallel-checkout: add tests for basic operations

From: Jonathan Nieder <[email-masked]>
Date: Mon, 19 Oct 2020 21:16:25 -0700

Matheus Tavares Bernardino wrote:
> On Mon, Oct 19, 2020 at 10:36 PM Jonathan Nieder <[email-masked]> wrote:

>> Can we use an artificial repo instead of git.git?  Using git.git as
>> test data seems like a recipe for hard-to-reproduce test failures.
>
> I think we could maybe drop these tests. There are already some
> similar tests below these, which use an artificial repository. The
> goal of using git.git in this section was to test parallel-checkout
> with a real-world repo, and hopefully catch errors that we might not
> see with small artificial ones.  But you have a very valid concern, as
> well. Hmm, I'm not sure what is the best solution to this case. What
> do you think?

I see.  I suppose my preference would be to have a real-world example
in t/perf/ (see t/perf/README for how it allows an arbitrary repo to
be passed in) instead of in the regression tests.  In the regression
testsuite I'd focus more on particular behaviors I want to test (e.g.,
a file being replaced by a directory, that kind of thing).

Behaviors exercised by git.git are in some sense the *least* important
thing to test here, since developers in the Git project know to
advocate for those and exercise them day-to-day.  Where the testsuite
shines is in being able to advocate for use cases that are exercised
by other populations --- a testsuite failure can be a reminder to not
forget about the features other people need that are not part of our
own daily lives.

Thanks,
Jonathan