* [PATCH 0/4] t: fix broken TAP output
@ 2026-06-02 8:54 Patrick Steinhardt
2026-06-02 8:54 ` [PATCH 1/4] t7527: " Patrick Steinhardt
` (5 more replies)
0 siblings, 6 replies; 22+ messages in thread
From: Patrick Steinhardt @ 2026-06-02 8:54 UTC (permalink / raw)
To: git
Hi,
this small patch series fixes another instance of broken TAP output that
has landed via 4d11b9c218 (Merge branch 'pt/fsmonitor-linux', 2026-05-31).
As this has happened multiple times by now I decided to have a look at
whether we can fix this class of issues a bit more holistically. So this
series also contains a change that makes prove bail out when it sees
invalid TAP output, which uncovers a small set of preexisting issues in
our test suite.
Thanks!
Patrick
---
Patrick Steinhardt (4):
t7527: fix broken TAP output
t/test-lib: silence EBUSY errors on Windows during test cleanup
t/lib-git-p4: silence output when killing p4d and its watchdog
t: let prove fail when parsing invalid TAP output
t/lib-git-p4.sh | 3 ++-
t/t7527-builtin-fsmonitor.sh | 7 ++++---
t/test-lib.sh | 10 ++++++++--
3 files changed, 14 insertions(+), 6 deletions(-)
---
base-commit: 1666c1265231b0bc5f613fbbf3f0a9896cdef76e
change-id: 20260601-pks-t7527-fix-tap-output-105da1d73df0
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH 1/4] t7527: fix broken TAP output 2026-06-02 8:54 [PATCH 0/4] t: fix broken TAP output Patrick Steinhardt @ 2026-06-02 8:54 ` Patrick Steinhardt 2026-06-02 8:54 ` [PATCH 2/4] t/test-lib: silence EBUSY errors on Windows during test cleanup Patrick Steinhardt ` (4 subsequent siblings) 5 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-02 8:54 UTC (permalink / raw) To: git Before running the tests in t7527 we first verify whether the fsmonitor even works, which seems to depend on the actual filesystem that is in use. The verification executes outside of any prerequisite or test body, so its stdout/stderr is not being redirected. The consequence of this is that any command that prints to stdout/stderr may break the TAP specification by printing invalid lines. And in fact we already do that, as git-init(1) prints the path to the created Git repository by default. Fix this issue by moving the logic into a lazy prerequisite. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/t7527-builtin-fsmonitor.sh | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index b63c162f9b..d881e27466 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -25,7 +25,8 @@ maybe_timeout () { "$@" fi } -verify_fsmonitor_works () { + +test_lazy_prereq FSMONITOR_WORKS ' git init test_fsmonitor_smoke || return 1 GIT_TRACE_FSMONITOR="$PWD/smoke.trace" && @@ -50,9 +51,9 @@ verify_fsmonitor_works () { ret=$? rm -rf test_fsmonitor_smoke smoke.trace return $ret -} +' -if ! verify_fsmonitor_works +if ! test_have_prereq FSMONITOR_WORKS then skip_all="filesystem does not deliver fsmonitor events (container/overlayfs?)" test_done -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 2/4] t/test-lib: silence EBUSY errors on Windows during test cleanup 2026-06-02 8:54 [PATCH 0/4] t: fix broken TAP output Patrick Steinhardt 2026-06-02 8:54 ` [PATCH 1/4] t7527: " Patrick Steinhardt @ 2026-06-02 8:54 ` Patrick Steinhardt 2026-06-02 8:54 ` [PATCH 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog Patrick Steinhardt ` (3 subsequent siblings) 5 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-02 8:54 UTC (permalink / raw) To: git When tests have finished we clean up the trash directory via `rm -rf`. On Windows this can fail with EBUSY in cases where a process still holds some of the files open, for example when we have spawned a daemonized process that wasn't properly terminated. We thus retry several times, but every failure will result in error messages being printed, and that in turn breaks the TAP output format. One such case where this is causing issues is in t921x, which contains tests related to Scalar. Some tests spawn the fsmonitor daemon, and we never properly terminate it. The obvious fix would be to ensure that we never leak any processes, but that gets ugly fast. Instead, let's work around the issue by silencing error messages printed by the `rm -rf` calls. We already know to print an error when the retry loop fails, so we don't loose much. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/test-lib.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 4a7357b547..d1d24c4124 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1299,10 +1299,10 @@ test_done () { error "Tests passed but trash directory already removed before test cleanup; aborting" cd "$TRASH_DIRECTORY/.." && - rm -fr "$TRASH_DIRECTORY" || { + rm -fr "$TRASH_DIRECTORY" 2>/dev/null || { # try again in a bit sleep 5; - rm -fr "$TRASH_DIRECTORY" + rm -fr "$TRASH_DIRECTORY" 2>/dev/null } || error "Tests passed but test cleanup failed; aborting" fi -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog 2026-06-02 8:54 [PATCH 0/4] t: fix broken TAP output Patrick Steinhardt 2026-06-02 8:54 ` [PATCH 1/4] t7527: " Patrick Steinhardt 2026-06-02 8:54 ` [PATCH 2/4] t/test-lib: silence EBUSY errors on Windows during test cleanup Patrick Steinhardt @ 2026-06-02 8:54 ` Patrick Steinhardt 2026-06-02 9:32 ` Junio C Hamano 2026-06-02 8:54 ` [PATCH 4/4] t: let prove fail when parsing invalid TAP output Patrick Steinhardt ` (2 subsequent siblings) 5 siblings, 1 reply; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-02 8:54 UTC (permalink / raw) To: git When stopping the p4d watchdog process via "kill -9", the shell may print a job-control notification like: ./test-lib.sh: line 1269: 57960 Killed: 9 while true; do if test $nr_tries_left -eq 0; then kill -9 $p4d_pid; exit 1; fi; sleep 1; nr_tries_left=$(($nr_tries_left - 1)); done 2> /dev/null 4>&2 (wd: ~) This message is printed asynchronously by the shell when it reaps the process. While harmless right now, this will cause breakage once we enable strict parsing of the TAP protocol in a subsequent commit. Fix this by using `wait` so that we can synchronously reap the watchdog process and swallow the diagnostic. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/lib-git-p4.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh index d22e9c684a..0afa5111ab 100644 --- a/t/lib-git-p4.sh +++ b/t/lib-git-p4.sh @@ -65,6 +65,7 @@ pidfile="$TRASH_DIRECTORY/p4d.pid" stop_p4d_and_watchdog () { kill -9 $p4d_pid $watchdog_pid + wait $p4d $watchdog_pid 2>/dev/null } # git p4 submit generates a temp file, which will @@ -175,7 +176,7 @@ retry_until_success () { stop_and_cleanup_p4d () { kill -9 $p4d_pid $watchdog_pid - wait $p4d_pid + wait $p4d_pid $watchdog_pid 2>/dev/null rm -rf "$db" "$cli" "$pidfile" } -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog 2026-06-02 8:54 ` [PATCH 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog Patrick Steinhardt @ 2026-06-02 9:32 ` Junio C Hamano 2026-06-02 10:20 ` Patrick Steinhardt 0 siblings, 1 reply; 22+ messages in thread From: Junio C Hamano @ 2026-06-02 9:32 UTC (permalink / raw) To: Patrick Steinhardt; +Cc: git Patrick Steinhardt <ps@pks•im> writes: > stop_p4d_and_watchdog () { > kill -9 $p4d_pid $watchdog_pid > + wait $p4d $watchdog_pid 2>/dev/null > } Shoudln't we be waiting on $p4d_pid (not $p4d)... > @@ -175,7 +176,7 @@ retry_until_success () { > > stop_and_cleanup_p4d () { > kill -9 $p4d_pid $watchdog_pid > - wait $p4d_pid > + wait $p4d_pid $watchdog_pid 2>/dev/null > rm -rf "$db" "$cli" "$pidfile" > } ... like we do here? ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog 2026-06-02 9:32 ` Junio C Hamano @ 2026-06-02 10:20 ` Patrick Steinhardt 2026-06-02 13:16 ` Junio C Hamano 0 siblings, 1 reply; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-02 10:20 UTC (permalink / raw) To: Junio C Hamano; +Cc: git On Tue, Jun 02, 2026 at 06:32:55PM +0900, Junio C Hamano wrote: > Patrick Steinhardt <ps@pks•im> writes: > > > stop_p4d_and_watchdog () { > > kill -9 $p4d_pid $watchdog_pid > > + wait $p4d $watchdog_pid 2>/dev/null > > } > > Shoudln't we be waiting on $p4d_pid (not $p4d)... > > > @@ -175,7 +176,7 @@ retry_until_success () { > > > > stop_and_cleanup_p4d () { > > kill -9 $p4d_pid $watchdog_pid > > - wait $p4d_pid > > + wait $p4d_pid $watchdog_pid 2>/dev/null > > rm -rf "$db" "$cli" "$pidfile" > > } > > ... like we do here? Oh, good catch. The statement basically doesn't do anything, which isn't much of a problem because we really only care about silencing the error message when the watchdog is being terminated. Will fix. Patrick ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog 2026-06-02 10:20 ` Patrick Steinhardt @ 2026-06-02 13:16 ` Junio C Hamano 0 siblings, 0 replies; 22+ messages in thread From: Junio C Hamano @ 2026-06-02 13:16 UTC (permalink / raw) To: Patrick Steinhardt; +Cc: git Patrick Steinhardt <ps@pks•im> writes: > On Tue, Jun 02, 2026 at 06:32:55PM +0900, Junio C Hamano wrote: >> Patrick Steinhardt <ps@pks•im> writes: >> >> > stop_p4d_and_watchdog () { >> > kill -9 $p4d_pid $watchdog_pid >> > + wait $p4d $watchdog_pid 2>/dev/null >> > } >> >> Shoudln't we be waiting on $p4d_pid (not $p4d)... >> >> > @@ -175,7 +176,7 @@ retry_until_success () { >> > >> > stop_and_cleanup_p4d () { >> > kill -9 $p4d_pid $watchdog_pid >> > - wait $p4d_pid >> > + wait $p4d_pid $watchdog_pid 2>/dev/null >> > rm -rf "$db" "$cli" "$pidfile" >> > } >> >> ... like we do here? > > Oh, good catch. The statement basically doesn't do anything, which isn't > much of a problem because we really only care about silencing the error > message when the watchdog is being terminated. Will fix. Thanks. Another thing I noticed is that they look suspiciously similar. ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH 4/4] t: let prove fail when parsing invalid TAP output 2026-06-02 8:54 [PATCH 0/4] t: fix broken TAP output Patrick Steinhardt ` (2 preceding siblings ...) 2026-06-02 8:54 ` [PATCH 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog Patrick Steinhardt @ 2026-06-02 8:54 ` Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 0/4] t: fix broken " Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 0/8] t: fix broken " Patrick Steinhardt 5 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-02 8:54 UTC (permalink / raw) To: git To make the result of our tests accessible we use the TAP protocol. This protocol is parsed by either prove or by Meson. Unfortunately, these two tools differ when it comes to their strictness when parsing the protocol: - Prove by default happily accepts lines not specified by the protocol. - Meson will also accept such lines, but prints a big and ugly warning message. We have fixed our test suite in the past to not print invalid TAP lines anymore via b1dc2e796e (Merge branch 'ps/meson-tap-parse', 2025-06-17). But as none of our tools perform a strict check it's still possible for broken tests to sneak back in, like for example in 362f69547f (Merge branch 'ps/t1006-tap-fix', 2025-07-16). This doesn't hurt at all when using prove, but it's quite annoying when using Meson due to the generated warnings. Unfortunately, there doesn't seem to be a portable way to make all tools complain about violations of the TAP format. The TAP 14 specification has added pragmas to the protocol that would allow us to say `pragma +strict`, and the effect of that would be to treat invalid TAP lines as a test failure. But the release of TAP 14 is still rather recent, and Test-Harness for example only gained support for it in version 3.48, which was released in 2023. In fact though, this pragma was already introduced as an inofficial extension of the TAP protocol with Test-Harness 3.10, released in 2008. So while not all tools understand the pragma, at least prove does for a long time. Unconditionally enable the pragma when using prove so that we'll detect tests that emit broken TAP output right away. This would have detected the issues fixed in preceding commits: $ prove t7527-builtin-fsmonitor.sh t7527-builtin-fsmonitor.sh .. All 69 subtests passed (less 6 skipped subtests: 63 okay) Test Summary Report ------------------- t7527-builtin-fsmonitor.sh (Wstat: 0 Tests: 69 Failed: 0) Parse errors: Unknown TAP token: "Initialized empty Git repository in /tmp/git/test_fsmonitor_smoke/.git/" Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/test-lib.sh | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/t/test-lib.sh b/t/test-lib.sh index d1d24c4124..ceefb99bff 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1532,6 +1532,12 @@ then BAIL_OUT 'You need to build test-tool; Run "make t/helper/test-tool" in the source (toplevel) directory' fi +if test -n "$HARNESS_ACTIVE" +then + say "TAP version 13" + say "pragma +strict" +fi + # Are we running this test at all? remove_trash= this_test=${0##*/} -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v2 0/4] t: fix broken TAP output 2026-06-02 8:54 [PATCH 0/4] t: fix broken TAP output Patrick Steinhardt ` (3 preceding siblings ...) 2026-06-02 8:54 ` [PATCH 4/4] t: let prove fail when parsing invalid TAP output Patrick Steinhardt @ 2026-06-03 5:39 ` Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 1/4] t7527: " Patrick Steinhardt ` (3 more replies) 2026-06-04 10:07 ` [PATCH v3 0/8] t: fix broken " Patrick Steinhardt 5 siblings, 4 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-03 5:39 UTC (permalink / raw) To: git; +Cc: Junio C Hamano Hi, this small patch series fixes another instance of broken TAP output that has landed via 4d11b9c218 (Merge branch 'pt/fsmonitor-linux', 2026-05-31). As this has happened multiple times by now I decided to have a look at whether we can fix this class of issues a bit more holistically. So this series also contains a change that makes prove bail out when it sees invalid TAP output, which uncovers a small set of preexisting issues in our test suite. Changes in v2: - Fix waiting for p4d, and deduplicate the logic that does this. - Link to v1: https://patch.msgid.link/20260602-pks-t7527-fix-tap-output-v1-0-db3da2a1b137@pks.im Thanks! Patrick --- Patrick Steinhardt (4): t7527: fix broken TAP output t/test-lib: silence EBUSY errors on Windows during test cleanup t/lib-git-p4: silence output when killing p4d and its watchdog t: let prove fail when parsing invalid TAP output t/lib-git-p4.sh | 4 ++-- t/t7527-builtin-fsmonitor.sh | 7 ++++--- t/test-lib.sh | 10 ++++++++-- 3 files changed, 14 insertions(+), 7 deletions(-) Range-diff versus v1: 1: 09977059d1 = 1: 18b4fc7b81 t7527: fix broken TAP output 2: 162d3d42d8 = 2: 8dd921534b t/test-lib: silence EBUSY errors on Windows during test cleanup 3: 4ecb8cb1ce < -: ---------- t/lib-git-p4: silence output when killing p4d and its watchdog -: ---------- > 3: 8b343176fe t/lib-git-p4: silence output when killing p4d and its watchdog 4: 95fb0d07ae = 4: e69aa0ab79 t: let prove fail when parsing invalid TAP output --- base-commit: 1666c1265231b0bc5f613fbbf3f0a9896cdef76e change-id: 20260601-pks-t7527-fix-tap-output-105da1d73df0 ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 1/4] t7527: fix broken TAP output 2026-06-03 5:39 ` [PATCH v2 0/4] t: fix broken " Patrick Steinhardt @ 2026-06-03 5:39 ` Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 2/4] t/test-lib: silence EBUSY errors on Windows during test cleanup Patrick Steinhardt ` (2 subsequent siblings) 3 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-03 5:39 UTC (permalink / raw) To: git; +Cc: Junio C Hamano Before running the tests in t7527 we first verify whether the fsmonitor even works, which seems to depend on the actual filesystem that is in use. The verification executes outside of any prerequisite or test body, so its stdout/stderr is not being redirected. The consequence of this is that any command that prints to stdout/stderr may break the TAP specification by printing invalid lines. And in fact we already do that, as git-init(1) prints the path to the created Git repository by default. Fix this issue by moving the logic into a lazy prerequisite. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/t7527-builtin-fsmonitor.sh | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index b63c162f9b..d881e27466 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -25,7 +25,8 @@ maybe_timeout () { "$@" fi } -verify_fsmonitor_works () { + +test_lazy_prereq FSMONITOR_WORKS ' git init test_fsmonitor_smoke || return 1 GIT_TRACE_FSMONITOR="$PWD/smoke.trace" && @@ -50,9 +51,9 @@ verify_fsmonitor_works () { ret=$? rm -rf test_fsmonitor_smoke smoke.trace return $ret -} +' -if ! verify_fsmonitor_works +if ! test_have_prereq FSMONITOR_WORKS then skip_all="filesystem does not deliver fsmonitor events (container/overlayfs?)" test_done -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v2 2/4] t/test-lib: silence EBUSY errors on Windows during test cleanup 2026-06-03 5:39 ` [PATCH v2 0/4] t: fix broken " Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 1/4] t7527: " Patrick Steinhardt @ 2026-06-03 5:39 ` Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 4/4] t: let prove fail when parsing invalid TAP output Patrick Steinhardt 3 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-03 5:39 UTC (permalink / raw) To: git; +Cc: Junio C Hamano When tests have finished we clean up the trash directory via `rm -rf`. On Windows this can fail with EBUSY in cases where a process still holds some of the files open, for example when we have spawned a daemonized process that wasn't properly terminated. We thus retry several times, but every failure will result in error messages being printed, and that in turn breaks the TAP output format. One such case where this is causing issues is in t921x, which contains tests related to Scalar. Some tests spawn the fsmonitor daemon, and we never properly terminate it. The obvious fix would be to ensure that we never leak any processes, but that gets ugly fast. Instead, let's work around the issue by silencing error messages printed by the `rm -rf` calls. We already know to print an error when the retry loop fails, so we don't loose much. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/test-lib.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 4a7357b547..d1d24c4124 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1299,10 +1299,10 @@ test_done () { error "Tests passed but trash directory already removed before test cleanup; aborting" cd "$TRASH_DIRECTORY/.." && - rm -fr "$TRASH_DIRECTORY" || { + rm -fr "$TRASH_DIRECTORY" 2>/dev/null || { # try again in a bit sleep 5; - rm -fr "$TRASH_DIRECTORY" + rm -fr "$TRASH_DIRECTORY" 2>/dev/null } || error "Tests passed but test cleanup failed; aborting" fi -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v2 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog 2026-06-03 5:39 ` [PATCH v2 0/4] t: fix broken " Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 1/4] t7527: " Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 2/4] t/test-lib: silence EBUSY errors on Windows during test cleanup Patrick Steinhardt @ 2026-06-03 5:39 ` Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 4/4] t: let prove fail when parsing invalid TAP output Patrick Steinhardt 3 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-03 5:39 UTC (permalink / raw) To: git; +Cc: Junio C Hamano When stopping the p4d watchdog process via "kill -9", the shell may print a job-control notification like: ./test-lib.sh: line 1269: 57960 Killed: 9 while true; do if test $nr_tries_left -eq 0; then kill -9 $p4d_pid; exit 1; fi; sleep 1; nr_tries_left=$(($nr_tries_left - 1)); done 2> /dev/null 4>&2 (wd: ~) This message is printed asynchronously by the shell when it reaps the process. While harmless right now, this will cause breakage once we enable strict parsing of the TAP protocol in a subsequent commit. Fix this by using `wait` so that we can synchronously reap the watchdog process and swallow the diagnostic. While at it, deduplicate the logic we have in `stop_p4d_and_watchdog ()` and `stop_and_cleanup_p4d ()`. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/lib-git-p4.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh index d22e9c684a..9108868187 100644 --- a/t/lib-git-p4.sh +++ b/t/lib-git-p4.sh @@ -65,6 +65,7 @@ pidfile="$TRASH_DIRECTORY/p4d.pid" stop_p4d_and_watchdog () { kill -9 $p4d_pid $watchdog_pid + wait $p4d_pid $watchdog_pid 2>/dev/null } # git p4 submit generates a temp file, which will @@ -174,8 +175,7 @@ retry_until_success () { } stop_and_cleanup_p4d () { - kill -9 $p4d_pid $watchdog_pid - wait $p4d_pid + stop_p4d_and_watchdog rm -rf "$db" "$cli" "$pidfile" } -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v2 4/4] t: let prove fail when parsing invalid TAP output 2026-06-03 5:39 ` [PATCH v2 0/4] t: fix broken " Patrick Steinhardt ` (2 preceding siblings ...) 2026-06-03 5:39 ` [PATCH v2 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog Patrick Steinhardt @ 2026-06-03 5:39 ` Patrick Steinhardt 3 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-03 5:39 UTC (permalink / raw) To: git; +Cc: Junio C Hamano To make the result of our tests accessible we use the TAP protocol. This protocol is parsed by either prove or by Meson. Unfortunately, these two tools differ when it comes to their strictness when parsing the protocol: - Prove by default happily accepts lines not specified by the protocol. - Meson will also accept such lines, but prints a big and ugly warning message. We have fixed our test suite in the past to not print invalid TAP lines anymore via b1dc2e796e (Merge branch 'ps/meson-tap-parse', 2025-06-17). But as none of our tools perform a strict check it's still possible for broken tests to sneak back in, like for example in 362f69547f (Merge branch 'ps/t1006-tap-fix', 2025-07-16). This doesn't hurt at all when using prove, but it's quite annoying when using Meson due to the generated warnings. Unfortunately, there doesn't seem to be a portable way to make all tools complain about violations of the TAP format. The TAP 14 specification has added pragmas to the protocol that would allow us to say `pragma +strict`, and the effect of that would be to treat invalid TAP lines as a test failure. But the release of TAP 14 is still rather recent, and Test-Harness for example only gained support for it in version 3.48, which was released in 2023. In fact though, this pragma was already introduced as an inofficial extension of the TAP protocol with Test-Harness 3.10, released in 2008. So while not all tools understand the pragma, at least prove does for a long time. Unconditionally enable the pragma when using prove so that we'll detect tests that emit broken TAP output right away. This would have detected the issues fixed in preceding commits: $ prove t7527-builtin-fsmonitor.sh t7527-builtin-fsmonitor.sh .. All 69 subtests passed (less 6 skipped subtests: 63 okay) Test Summary Report ------------------- t7527-builtin-fsmonitor.sh (Wstat: 0 Tests: 69 Failed: 0) Parse errors: Unknown TAP token: "Initialized empty Git repository in /tmp/git/test_fsmonitor_smoke/.git/" Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/test-lib.sh | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/t/test-lib.sh b/t/test-lib.sh index d1d24c4124..ceefb99bff 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1532,6 +1532,12 @@ then BAIL_OUT 'You need to build test-tool; Run "make t/helper/test-tool" in the source (toplevel) directory' fi +if test -n "$HARNESS_ACTIVE" +then + say "TAP version 13" + say "pragma +strict" +fi + # Are we running this test at all? remove_trash= this_test=${0##*/} -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 0/8] t: fix broken TAP output 2026-06-02 8:54 [PATCH 0/4] t: fix broken TAP output Patrick Steinhardt ` (4 preceding siblings ...) 2026-06-03 5:39 ` [PATCH v2 0/4] t: fix broken " Patrick Steinhardt @ 2026-06-04 10:07 ` Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 1/8] gitlab-ci: rearrange Linux jobs to match GitHub's order Patrick Steinhardt ` (7 more replies) 5 siblings, 8 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-04 10:07 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King Hi, this small patch series fixes another instance of broken TAP output that has landed via 4d11b9c218 (Merge branch 'pt/fsmonitor-linux', 2026-05-31). As this has happened multiple times by now I decided to have a look at whether we can fix this class of issues a bit more holistically. So this series also contains a change that makes prove bail out when it sees invalid TAP output, which uncovers a small set of preexisting issues in our test suite. Changes in v3: - Fix a test gap for AlmaLinux and Debian in GitLab CI, which uncovers an issue flagged by Peff. - Fix TAP breakage in t7810. - Link to v2: https://patch.msgid.link/20260603-pks-t7527-fix-tap-output-v2-0-cf3af5694e20@pks.im Changes in v2: - Fix waiting for p4d, and deduplicate the logic that does this. - Link to v1: https://patch.msgid.link/20260602-pks-t7527-fix-tap-output-v1-0-db3da2a1b137@pks.im Test runs can be found at [1] and [2]. Note that GitHub-side tests are failing on Windows, but that is a preexisting failure on "master". Thanks! Patrick [1]: https://gitlab.com/gitlab-org/git/-/merge_requests/585 [2]: https://github.com/git/git/pull/2320 --- Patrick Steinhardt (8): gitlab-ci: rearrange Linux jobs to match GitHub's order gitlab-ci: add missing Linux jobs ci: unify Linux images across GitLab and GitHub t7527: fix broken TAP output t7810: turn MB_REGEX check into a lazy prereq t/test-lib: silence EBUSY errors on Windows during test cleanup t/lib-git-p4: silence output when killing p4d and its watchdog t: let prove fail when parsing invalid TAP output .github/workflows/main.yml | 2 +- .gitlab-ci.yml | 23 +++++++++++++++-------- ci/lib.sh | 2 +- t/lib-git-p4.sh | 4 ++-- t/t7527-builtin-fsmonitor.sh | 7 ++++--- t/t7810-grep.sh | 5 +++-- t/test-lib.sh | 10 ++++++++-- 7 files changed, 34 insertions(+), 19 deletions(-) Range-diff versus v2: -: ---------- > 1: 5e817b102f gitlab-ci: rearrange Linux jobs to match GitHub's order -: ---------- > 2: 83646cc834 gitlab-ci: add missing Linux jobs -: ---------- > 3: cca1567fbf ci: unify Linux images across GitLab and GitHub 1: 52abbd5280 = 4: 430bc51818 t7527: fix broken TAP output -: ---------- > 5: 78ef22df8d t7810: turn MB_REGEX check into a lazy prereq 2: ea1f1eb466 = 6: 7bbaeff48c t/test-lib: silence EBUSY errors on Windows during test cleanup 3: e97a515470 = 7: abf2be09e6 t/lib-git-p4: silence output when killing p4d and its watchdog 4: 436d7d8cf3 = 8: 04367c34be t: let prove fail when parsing invalid TAP output --- base-commit: 1666c1265231b0bc5f613fbbf3f0a9896cdef76e change-id: 20260601-pks-t7527-fix-tap-output-105da1d73df0 ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v3 1/8] gitlab-ci: rearrange Linux jobs to match GitHub's order 2026-06-04 10:07 ` [PATCH v3 0/8] t: fix broken " Patrick Steinhardt @ 2026-06-04 10:07 ` Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 2/8] gitlab-ci: add missing Linux jobs Patrick Steinhardt ` (6 subsequent siblings) 7 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-04 10:07 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King Rearrange the order of Linux jobs that we have defined in GitLab CI so that it matches the order on GitHub's side. This makes it easier to compare whether the list of jobs actually matches on both sides. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- .gitlab-ci.yml | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index e0b9a0d82b..8cb41baa14 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -42,15 +42,15 @@ test:linux: - jobname: linux-reftable image: ubuntu:rolling CC: clang + - jobname: linux-TEST-vars + image: ubuntu:20.04 + CC: gcc + CC_PACKAGE: gcc-8 - jobname: linux-breaking-changes image: ubuntu:20.04 CC: gcc - jobname: fedora-breaking-changes-meson image: fedora:latest - - jobname: linux-TEST-vars - image: ubuntu:20.04 - CC: gcc - CC_PACKAGE: gcc-8 - jobname: linux-leaks image: ubuntu:rolling CC: gcc @@ -60,13 +60,14 @@ test:linux: - jobname: linux-asan-ubsan image: ubuntu:rolling CC: clang + - jobname: linux-meson + image: ubuntu:rolling + CC: gcc - jobname: linux-musl-meson image: alpine:latest + # Supported until 2025-04-02. - jobname: linux32 image: i386/ubuntu:20.04 - - jobname: linux-meson - image: ubuntu:rolling - CC: gcc artifacts: paths: - t/failed-test-artifacts -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 2/8] gitlab-ci: add missing Linux jobs 2026-06-04 10:07 ` [PATCH v3 0/8] t: fix broken " Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 1/8] gitlab-ci: rearrange Linux jobs to match GitHub's order Patrick Steinhardt @ 2026-06-04 10:07 ` Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 3/8] ci: unify Linux images across GitLab and GitHub Patrick Steinhardt ` (5 subsequent siblings) 7 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-04 10:07 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King The GitLab CI definitions are missing jobs for AlmaLinux and Debian, both of which exist in GitHub Workflows. Plug this gap. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- .gitlab-ci.yml | 6 ++++++ ci/lib.sh | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 8cb41baa14..a5bdec5159 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -68,6 +68,12 @@ test:linux: # Supported until 2025-04-02. - jobname: linux32 image: i386/ubuntu:20.04 + # A RHEL 8 compatible distro. Supported until 2029-05-31. + - jobname: almalinux-8 + image: almalinux:8 + # Supported until 2026-08-31. + - jobname: debian-11 + image: debian:11 artifacts: paths: - t/failed-test-artifacts diff --git a/ci/lib.sh b/ci/lib.sh index 6e3799cfc3..b939110a6e 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -254,7 +254,7 @@ then CI_OS_NAME=osx JOBS=$(nproc) ;; - *,alpine:*|*,fedora:*|*,ubuntu:*|*,i386/ubuntu:*) + *,almalinux:*|*,alpine:*|*,debian:*|*,fedora:*|*,ubuntu:*|*,i386/ubuntu:*) CI_OS_NAME=linux JOBS=$(nproc) ;; -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 3/8] ci: unify Linux images across GitLab and GitHub 2026-06-04 10:07 ` [PATCH v3 0/8] t: fix broken " Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 1/8] gitlab-ci: rearrange Linux jobs to match GitHub's order Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 2/8] gitlab-ci: add missing Linux jobs Patrick Steinhardt @ 2026-06-04 10:07 ` Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 4/8] t7527: fix broken TAP output Patrick Steinhardt ` (4 subsequent siblings) 7 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-04 10:07 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King The image for the "linux-breaking-changes" job has drifted apart across GitHub and GitLab. Adapt it to use "ubuntu:rolling" on both systems. With this change there's only one difference remaining: GitHub uses "ubuntu:focal" for the "linux32" job while GitLab uses "ubuntu:20.04". These are different names for the same image, so there is no actual difference here. Adjust GitHub to use the "20.04" tag -- this matches all the other jobs which use version numbers, and you don't have to learn Ubuntu's release names by heart. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- .github/workflows/main.yml | 2 +- .gitlab-ci.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 3da5326f0b..cf341d74db 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -407,7 +407,7 @@ jobs: image: alpine:latest # Supported until 2025-04-02. - jobname: linux32 - image: i386/ubuntu:focal + image: i386/ubuntu:20.04 # A RHEL 8 compatible distro. Supported until 2029-05-31. - jobname: almalinux-8 image: almalinux:8 diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index a5bdec5159..49f3689b6a 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -47,7 +47,7 @@ test:linux: CC: gcc CC_PACKAGE: gcc-8 - jobname: linux-breaking-changes - image: ubuntu:20.04 + image: ubuntu:rolling CC: gcc - jobname: fedora-breaking-changes-meson image: fedora:latest -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 4/8] t7527: fix broken TAP output 2026-06-04 10:07 ` [PATCH v3 0/8] t: fix broken " Patrick Steinhardt ` (2 preceding siblings ...) 2026-06-04 10:07 ` [PATCH v3 3/8] ci: unify Linux images across GitLab and GitHub Patrick Steinhardt @ 2026-06-04 10:07 ` Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 5/8] t7810: turn MB_REGEX check into a lazy prereq Patrick Steinhardt ` (3 subsequent siblings) 7 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-04 10:07 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King Before running the tests in t7527 we first verify whether the fsmonitor even works, which seems to depend on the actual filesystem that is in use. The verification executes outside of any prerequisite or test body, so its stdout/stderr is not being redirected. The consequence of this is that any command that prints to stdout/stderr may break the TAP specification by printing invalid lines. And in fact we already do that, as git-init(1) prints the path to the created Git repository by default. Fix this issue by moving the logic into a lazy prerequisite. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/t7527-builtin-fsmonitor.sh | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index b63c162f9b..d881e27466 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -25,7 +25,8 @@ maybe_timeout () { "$@" fi } -verify_fsmonitor_works () { + +test_lazy_prereq FSMONITOR_WORKS ' git init test_fsmonitor_smoke || return 1 GIT_TRACE_FSMONITOR="$PWD/smoke.trace" && @@ -50,9 +51,9 @@ verify_fsmonitor_works () { ret=$? rm -rf test_fsmonitor_smoke smoke.trace return $ret -} +' -if ! verify_fsmonitor_works +if ! test_have_prereq FSMONITOR_WORKS then skip_all="filesystem does not deliver fsmonitor events (container/overlayfs?)" test_done -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 5/8] t7810: turn MB_REGEX check into a lazy prereq 2026-06-04 10:07 ` [PATCH v3 0/8] t: fix broken " Patrick Steinhardt ` (3 preceding siblings ...) 2026-06-04 10:07 ` [PATCH v3 4/8] t7527: fix broken TAP output Patrick Steinhardt @ 2026-06-04 10:07 ` Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 6/8] t/test-lib: silence EBUSY errors on Windows during test cleanup Patrick Steinhardt ` (2 subsequent siblings) 7 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-04 10:07 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King In t7810 we verify whether the system has proper multibyte locale support by executing `test-tool regex` with a unicode character. When this check fails though we'll output an error that breaks the TAP format. Fix this issue by turning the logic into a lazy prerequisite. Reported-by: Jeff King <peff@peff•net> Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/t7810-grep.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/t/t7810-grep.sh b/t/t7810-grep.sh index 1b195bee59..d61c4a4d73 100755 --- a/t/t7810-grep.sh +++ b/t/t7810-grep.sh @@ -18,8 +18,9 @@ test_invalid_grep_expression() { ' } -LC_ALL=en_US.UTF-8 test-tool regex '^.$' '¿' && - test_set_prereq MB_REGEX +test_lazy_prereq MB_REGEX ' + LC_ALL=en_US.UTF-8 test-tool regex "^.$" "¿" +' cat >hello.c <<EOF #include <assert.h> -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 6/8] t/test-lib: silence EBUSY errors on Windows during test cleanup 2026-06-04 10:07 ` [PATCH v3 0/8] t: fix broken " Patrick Steinhardt ` (4 preceding siblings ...) 2026-06-04 10:07 ` [PATCH v3 5/8] t7810: turn MB_REGEX check into a lazy prereq Patrick Steinhardt @ 2026-06-04 10:07 ` Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 7/8] t/lib-git-p4: silence output when killing p4d and its watchdog Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 8/8] t: let prove fail when parsing invalid TAP output Patrick Steinhardt 7 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-04 10:07 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King When tests have finished we clean up the trash directory via `rm -rf`. On Windows this can fail with EBUSY in cases where a process still holds some of the files open, for example when we have spawned a daemonized process that wasn't properly terminated. We thus retry several times, but every failure will result in error messages being printed, and that in turn breaks the TAP output format. One such case where this is causing issues is in t921x, which contains tests related to Scalar. Some tests spawn the fsmonitor daemon, and we never properly terminate it. The obvious fix would be to ensure that we never leak any processes, but that gets ugly fast. Instead, let's work around the issue by silencing error messages printed by the `rm -rf` calls. We already know to print an error when the retry loop fails, so we don't loose much. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/test-lib.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 4a7357b547..d1d24c4124 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1299,10 +1299,10 @@ test_done () { error "Tests passed but trash directory already removed before test cleanup; aborting" cd "$TRASH_DIRECTORY/.." && - rm -fr "$TRASH_DIRECTORY" || { + rm -fr "$TRASH_DIRECTORY" 2>/dev/null || { # try again in a bit sleep 5; - rm -fr "$TRASH_DIRECTORY" + rm -fr "$TRASH_DIRECTORY" 2>/dev/null } || error "Tests passed but test cleanup failed; aborting" fi -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 7/8] t/lib-git-p4: silence output when killing p4d and its watchdog 2026-06-04 10:07 ` [PATCH v3 0/8] t: fix broken " Patrick Steinhardt ` (5 preceding siblings ...) 2026-06-04 10:07 ` [PATCH v3 6/8] t/test-lib: silence EBUSY errors on Windows during test cleanup Patrick Steinhardt @ 2026-06-04 10:07 ` Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 8/8] t: let prove fail when parsing invalid TAP output Patrick Steinhardt 7 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-04 10:07 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King When stopping the p4d watchdog process via "kill -9", the shell may print a job-control notification like: ./test-lib.sh: line 1269: 57960 Killed: 9 while true; do if test $nr_tries_left -eq 0; then kill -9 $p4d_pid; exit 1; fi; sleep 1; nr_tries_left=$(($nr_tries_left - 1)); done 2> /dev/null 4>&2 (wd: ~) This message is printed asynchronously by the shell when it reaps the process. While harmless right now, this will cause breakage once we enable strict parsing of the TAP protocol in a subsequent commit. Fix this by using `wait` so that we can synchronously reap the watchdog process and swallow the diagnostic. While at it, deduplicate the logic we have in `stop_p4d_and_watchdog ()` and `stop_and_cleanup_p4d ()`. Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/lib-git-p4.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh index d22e9c684a..9108868187 100644 --- a/t/lib-git-p4.sh +++ b/t/lib-git-p4.sh @@ -65,6 +65,7 @@ pidfile="$TRASH_DIRECTORY/p4d.pid" stop_p4d_and_watchdog () { kill -9 $p4d_pid $watchdog_pid + wait $p4d_pid $watchdog_pid 2>/dev/null } # git p4 submit generates a temp file, which will @@ -174,8 +175,7 @@ retry_until_success () { } stop_and_cleanup_p4d () { - kill -9 $p4d_pid $watchdog_pid - wait $p4d_pid + stop_p4d_and_watchdog rm -rf "$db" "$cli" "$pidfile" } -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 8/8] t: let prove fail when parsing invalid TAP output 2026-06-04 10:07 ` [PATCH v3 0/8] t: fix broken " Patrick Steinhardt ` (6 preceding siblings ...) 2026-06-04 10:07 ` [PATCH v3 7/8] t/lib-git-p4: silence output when killing p4d and its watchdog Patrick Steinhardt @ 2026-06-04 10:07 ` Patrick Steinhardt 7 siblings, 0 replies; 22+ messages in thread From: Patrick Steinhardt @ 2026-06-04 10:07 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King To make the result of our tests accessible we use the TAP protocol. This protocol is parsed by either prove or by Meson. Unfortunately, these two tools differ when it comes to their strictness when parsing the protocol: - Prove by default happily accepts lines not specified by the protocol. - Meson will also accept such lines, but prints a big and ugly warning message. We have fixed our test suite in the past to not print invalid TAP lines anymore via b1dc2e796e (Merge branch 'ps/meson-tap-parse', 2025-06-17). But as none of our tools perform a strict check it's still possible for broken tests to sneak back in, like for example in 362f69547f (Merge branch 'ps/t1006-tap-fix', 2025-07-16). This doesn't hurt at all when using prove, but it's quite annoying when using Meson due to the generated warnings. Unfortunately, there doesn't seem to be a portable way to make all tools complain about violations of the TAP format. The TAP 14 specification has added pragmas to the protocol that would allow us to say `pragma +strict`, and the effect of that would be to treat invalid TAP lines as a test failure. But the release of TAP 14 is still rather recent, and Test-Harness for example only gained support for it in version 3.48, which was released in 2023. In fact though, this pragma was already introduced as an inofficial extension of the TAP protocol with Test-Harness 3.10, released in 2008. So while not all tools understand the pragma, at least prove does for a long time. Unconditionally enable the pragma when using prove so that we'll detect tests that emit broken TAP output right away. This would have detected the issues fixed in preceding commits: $ prove t7527-builtin-fsmonitor.sh t7527-builtin-fsmonitor.sh .. All 69 subtests passed (less 6 skipped subtests: 63 okay) Test Summary Report ------------------- t7527-builtin-fsmonitor.sh (Wstat: 0 Tests: 69 Failed: 0) Parse errors: Unknown TAP token: "Initialized empty Git repository in /tmp/git/test_fsmonitor_smoke/.git/" Signed-off-by: Patrick Steinhardt <ps@pks•im> --- t/test-lib.sh | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/t/test-lib.sh b/t/test-lib.sh index d1d24c4124..ceefb99bff 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1532,6 +1532,12 @@ then BAIL_OUT 'You need to build test-tool; Run "make t/helper/test-tool" in the source (toplevel) directory' fi +if test -n "$HARNESS_ACTIVE" +then + say "TAP version 13" + say "pragma +strict" +fi + # Are we running this test at all? remove_trash= this_test=${0##*/} -- 2.54.0.1064.gd145956f57.dirty ^ permalink raw reply related [flat|nested] 22+ messages in thread
end of thread, other threads:[~2026-06-04 10:07 UTC | newest] Thread overview: 22+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-06-02 8:54 [PATCH 0/4] t: fix broken TAP output Patrick Steinhardt 2026-06-02 8:54 ` [PATCH 1/4] t7527: " Patrick Steinhardt 2026-06-02 8:54 ` [PATCH 2/4] t/test-lib: silence EBUSY errors on Windows during test cleanup Patrick Steinhardt 2026-06-02 8:54 ` [PATCH 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog Patrick Steinhardt 2026-06-02 9:32 ` Junio C Hamano 2026-06-02 10:20 ` Patrick Steinhardt 2026-06-02 13:16 ` Junio C Hamano 2026-06-02 8:54 ` [PATCH 4/4] t: let prove fail when parsing invalid TAP output Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 0/4] t: fix broken " Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 1/4] t7527: " Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 2/4] t/test-lib: silence EBUSY errors on Windows during test cleanup Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog Patrick Steinhardt 2026-06-03 5:39 ` [PATCH v2 4/4] t: let prove fail when parsing invalid TAP output Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 0/8] t: fix broken " Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 1/8] gitlab-ci: rearrange Linux jobs to match GitHub's order Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 2/8] gitlab-ci: add missing Linux jobs Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 3/8] ci: unify Linux images across GitLab and GitHub Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 4/8] t7527: fix broken TAP output Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 5/8] t7810: turn MB_REGEX check into a lazy prereq Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 6/8] t/test-lib: silence EBUSY errors on Windows during test cleanup Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 7/8] t/lib-git-p4: silence output when killing p4d and its watchdog Patrick Steinhardt 2026-06-04 10:07 ` [PATCH v3 8/8] t: let prove fail when parsing invalid TAP output Patrick Steinhardt
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox