Subject: [RFC/PATCH] Makefile: add test-all target

From: Junio C Hamano <[email-masked]>
Date: Wed, 08 Dec 2021 12:04:50 -0800

We ship contrib/ stuff within our primary source tree but except for
the completion scripts that are tested from our primary test suite,
their test suites are not run in the CI.

Teach the main Makefile a "test-extra" target, which goes into each
package in contrib/ whose Makefile has its own "test" target and
runs "make test" there.  Add a "test-all" target to make it easy to
drive both the primary tests and these contrib tests from CI and use
it.

Signed-off-by: Junio C Hamano <[email-masked]>
---
Junio C Hamano <[email-masked]> writes:

> That is an interesting way to demonstrate how orthogonal the issues
> are, which in turn means that it is not such a big deal to add back
> the coverage to the part that goes to contrib/scalar/.  As the actual
> implementation, it is a bit too icky, though.

So, how about doing it this way?  This is based on 'master' and does
not cover contrib/scalar, but if we want to go this route, it should
be trivial to do it on top of a merge of ab/ci-updates and js/scalar
into 'master'.  Good idea?  Terrible idea?  Not good enough?

 Makefile                  | 12 +++++++++++-
 ci/run-build-and-tests.sh | 10 +++++-----
 2 files changed, 16 insertions(+), 6 deletions(-)

diff --git i/Makefile w/Makefile
index d56c0e4aad..ca14558e3c 100644
--- i/Makefile
+++ w/Makefile
@@ -2878,10 +2878,20 @@ export TEST_NO_MALLOC_CHECK
 test: all
 	$(MAKE) -C t/ all
 
+# Additional tests from places in contrib/ that are prepared to take
+# "make -C $there test", but expects that the primary build is done
+# already.
+test-extra: all
+	$(MAKE) -C contrib/diff-highlight test
+	$(MAKE) -C contrib/mw-to-git test
+	$(MAKE) -C contrib/subtree test
+
+test-all:: test test-extra
+
 perf: all
 	$(MAKE) -C t/perf/ all
 
-.PHONY: test perf
+.PHONY: test test-extra test-all perf
 
 .PRECIOUS: $(TEST_OBJS)
 
diff --git i/ci/run-build-and-tests.sh w/ci/run-build-and-tests.sh
index cc62616d80..9da0f26665 100755
--- i/ci/run-build-and-tests.sh
+++ w/ci/run-build-and-tests.sh
@@ -19,7 +19,7 @@ make
 case "$jobname" in
 linux-gcc)
 	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
-	make test
+	make test-all
 	export GIT_TEST_SPLIT_INDEX=yes
 	export GIT_TEST_MERGE_ALGORITHM=recursive
 	export GIT_TEST_FULL_IN_PACK_ARRAY=true
@@ -33,20 +33,20 @@ linux-gcc)
 	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master
 	export GIT_TEST_WRITE_REV_INDEX=1
 	export GIT_TEST_CHECKOUT_WORKERS=2
-	make test
+	make test-all
 	;;
 linux-clang)
 	export GIT_TEST_DEFAULT_HASH=sha1
-	make test
+	make test-all
 	export GIT_TEST_DEFAULT_HASH=sha256
-	make test
+	make test-all
 	;;
 linux-gcc-4.8|pedantic)
 	# Don't run the tests; we only care about whether Git can be
 	# built with GCC 4.8 or with pedantic
 	;;
 *)
-	make test
+	make test-all
 	;;
 esac