From: Toon Claes <toon@iotcl•com>
To: Patrick Steinhardt <ps@pks•im>, git@vger•kernel.org
Subject: Re: [PATCH 4/5] meson: wire up benchmarks
Date: Thu, 10 Apr 2025 13:44:55 +0200 [thread overview]
Message-ID: <87r020dyvc.fsf@iotcl.com> (raw)
In-Reply-To: <20250331-pks-meson-benchmarks-v1-4-b2ace85616a3@pks.im>
Patrick Steinhardt <ps@pks•im> writes:
> Wire up benchmarks in Meson. The setup is mostly the same as how we wire
> up our tests. The only difference is that benchmarks get wired up via
> the `benchmark()` option instead of via `test()`, which gives them a bit
> of special treatment:
>
> - Benchmarks never run in parallel.
>
> - Benchmarks aren't run by default when tests are executed.
>
> - Meson does not inject the `MALLOC_PERTURB` environment variable.
>
> Using benchmarks is quite simple:
>
> ```
> $ meson setup build
> # Run all benchmarks.
> $ meson test -C build --benchmark
> # Run a specific benchmark.
> $ meson test -C build --benchmark p0000-*
> ```
I really appreciate the efforts, and I'd love to run the perf tests with
Meson, but at the moment I don't know how these changes are useful
because this doesn't print anything relevant:
$ meson test -C build --benchmark 'p0005-*'
ninja: Entering directory `/home/toon/devel/git/build'
[1/28] Generating GIT-VERSION-FILE with a custom command (wrapped by meson to set env)
1/1 p0005-status OK 7.39s
Ok: 1
Expected Fail: 0
Fail: 0
Unexpected Pass: 0
Skipped: 0
Timeout: 0
Full log written to /home/toon/devel/git/build/meson-logs/testlog.txt
> Other than that the usual command line arguments accepted when running
> tests are also accepted when running benchmarks.
>
> Note that the benchmarking target is somewhat limited because it will
> only run benchmarks for the current build. Other usecases, like running
> benchmarks against multiple different versions of Git, are not currently
> supported. Users should continue to use "t/perf/run" for those usecases.
> The script should get extended at one point in time to support Meson,
> but this is outside of the scope of this series.
Yeah, this is unfortunate, but totally understandable. I've been digging
in `t/perf/run` and `f/perf/aggregate.perl` and it doesn't look easy to
adapt to Meson.
But I was wondering, instead of trying to fully integrate Meson into
those scripts, could we modify the scripts so they work with binaries
built by Meson? I mean, if we could run
`cd t/perf && ./run ../../build1 ../../build2 p0005*` and it would
simply run the benchmarks in those directories (without trying to check
out code and build the sources). I think this would help a lot already.
--
Toon
next prev parent reply other threads:[~2025-04-10 11:45 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-03-31 6:16 [PATCH 0/5] meson: wire up support for benchmarks Patrick Steinhardt
2025-03-31 6:16 ` [PATCH 1/5] t/perf: fix benchmarks with alternate repo formats Patrick Steinhardt
2025-03-31 6:16 ` [PATCH 2/5] t/perf: use configured PERL_PATH Patrick Steinhardt
2025-04-10 11:43 ` Toon Claes
2025-04-14 6:50 ` Patrick Steinhardt
2025-04-14 19:20 ` Junio C Hamano
2025-04-15 10:01 ` Patrick Steinhardt
2025-03-31 6:16 ` [PATCH 3/5] t/perf: fix benchmarks with out-of-tree builds Patrick Steinhardt
2025-04-10 11:34 ` Toon Claes
2025-04-14 6:28 ` Toon Claes
2025-04-14 6:50 ` Patrick Steinhardt
2025-03-31 6:16 ` [PATCH 4/5] meson: wire up benchmarks Patrick Steinhardt
2025-04-10 11:44 ` Toon Claes [this message]
2025-04-14 6:50 ` Patrick Steinhardt
2025-04-14 9:07 ` Toon Claes
2025-03-31 6:16 ` [PATCH 5/5] meson: wire up benchmarking options Patrick Steinhardt
2025-04-14 6:51 ` [PATCH v2 0/5] meson: wire up support for benchmarks Patrick Steinhardt
2025-04-14 6:51 ` [PATCH v2 1/5] t/perf: fix benchmarks with alternate repo formats Patrick Steinhardt
2025-04-14 6:51 ` [PATCH v2 2/5] t/perf: use configured PERL_PATH Patrick Steinhardt
2025-04-14 6:51 ` [PATCH v2 3/5] t/perf: fix benchmarks with out-of-tree builds Patrick Steinhardt
2025-04-20 10:00 ` Christian Couder
2025-04-22 6:51 ` Patrick Steinhardt
2025-04-14 6:51 ` [PATCH v2 4/5] meson: wire up benchmarks Patrick Steinhardt
2025-04-20 10:00 ` Christian Couder
2025-04-14 6:51 ` [PATCH v2 5/5] meson: wire up benchmarking options Patrick Steinhardt
2025-04-15 14:36 ` [PATCH v2 0/5] meson: wire up support for benchmarks Junio C Hamano
2025-04-15 18:18 ` Junio C Hamano
2025-04-16 11:00 ` Patrick Steinhardt
2025-04-18 23:02 ` Junio C Hamano
2025-04-22 6:50 ` [PATCH v3 " Patrick Steinhardt
2025-04-22 6:50 ` [PATCH v3 1/5] t/perf: fix benchmarks with alternate repo formats Patrick Steinhardt
2025-04-22 6:50 ` [PATCH v3 2/5] t/perf: use configured PERL_PATH Patrick Steinhardt
2025-04-22 6:50 ` [PATCH v3 3/5] t/perf: fix benchmarks with out-of-tree builds Patrick Steinhardt
2025-04-22 6:50 ` [PATCH v3 4/5] meson: wire up benchmarks Patrick Steinhardt
2025-04-22 6:50 ` [PATCH v3 5/5] meson: wire up benchmarking options Patrick Steinhardt
2025-04-22 7:27 ` [PATCH v3 0/5] meson: wire up support for benchmarks Christian Couder
2025-04-22 7:53 ` Patrick Steinhardt
2025-04-23 14:44 ` Christian Couder
2025-04-24 4:31 ` Patrick Steinhardt
2025-04-24 6:28 ` Christian Couder
2025-04-24 11:13 ` Junio C Hamano
2025-04-24 13:49 ` Patrick Steinhardt
2025-04-23 14:12 ` Toon Claes
2025-04-25 7:28 ` [PATCH v4 " Patrick Steinhardt
2025-04-25 7:28 ` [PATCH v4 1/5] t/perf: fix benchmarks with alternate repo formats Patrick Steinhardt
2025-04-25 7:28 ` [PATCH v4 2/5] t/perf: use configured PERL_PATH Patrick Steinhardt
2025-04-25 7:28 ` [PATCH v4 3/5] t/perf: fix benchmarks with out-of-tree builds Patrick Steinhardt
2025-04-25 7:28 ` [PATCH v4 4/5] meson: wire up benchmarks Patrick Steinhardt
2025-04-25 7:57 ` Christian Couder
2025-04-25 8:01 ` Patrick Steinhardt
2025-04-25 8:09 ` Christian Couder
2025-04-25 8:27 ` Patrick Steinhardt
2025-04-25 7:28 ` [PATCH v4 5/5] meson: wire up benchmarking options Patrick Steinhardt
2025-04-25 8:06 ` [PATCH v4 0/5] meson: wire up support for benchmarks Christian Couder
2025-04-25 8:26 ` Patrick Steinhardt
2025-04-25 8:33 ` Christian Couder
2025-04-28 7:30 ` [PATCH v5 " Patrick Steinhardt
2025-04-28 7:30 ` [PATCH v5 1/5] t/perf: fix benchmarks with alternate repo formats Patrick Steinhardt
2025-04-28 7:30 ` [PATCH v5 2/5] t/perf: use configured PERL_PATH Patrick Steinhardt
2025-04-28 7:30 ` [PATCH v5 3/5] t/perf: fix benchmarks with out-of-tree builds Patrick Steinhardt
2025-04-28 7:30 ` [PATCH v5 4/5] meson: wire up benchmarks Patrick Steinhardt
2025-04-28 7:30 ` [PATCH v5 5/5] meson: wire up benchmarking options Patrick Steinhardt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87r020dyvc.fsf@iotcl.com \
--to=toon@iotcl$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=ps@pks$(echo .)im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox