From: Adrian Ratiu <adrian.ratiu@collabora•com>
To: git@vger•kernel.org, Jeff King <peff@peff•net>
Cc: Emily Shaffer <emilyshaffer@google•com>,
Junio C Hamano <gitster@pobox•com>,
Patrick Steinhardt <ps@pks•im>,
Josh Steadmon <steadmon@google•com>,
Kristoffer Haugsbakk <kristofferhaugsbakk@fastmail•com>,
Adrian Ratiu <adrian.ratiu@collabora•com>
Subject: [PATCH v7 09/12] hook: add jobs option
Date: Wed, 21 Jan 2026 23:54:33 +0200 [thread overview]
Message-ID: <20260121215436.1473800-10-adrian.ratiu@collabora.com> (raw)
In-Reply-To: <20260121215436.1473800-1-adrian.ratiu@collabora.com>
Allow the API callers to specify the number of jobs across which
hook execution can be parallelized. It defaults to 1 and no hook
currently changes it, so all hooks run sequentially as before.
This allows us to both pave the way for parallel hook execution
(that will be a follow-up patch series building upon this) and to
finish the API conversion of builtin/receive-pack.c, keeping the
output muxer thread design as Peff suggested.
When .jobs == 1 nothing changes, the simple "copy_to_sideband" mux
still outputs directly via sideband channel 2, keeping the current
(mostly) real-time output characteristics, avoids unecessary poll
delays or deadlock risks.
When .jobs > 1, a more complex muxer is needed, to buffer the hook
output and avoid interleaving. After working on this mux I quickly
realized I was re-implementing run-command with ungroup=0 so that
ideas was dropped in favor of run-command which outputs to stderr.
Then run-command's stderr just connects to the simple muxer input.
Maybe it's better to understand using ascii graphics:
[ Sequential (jobs = 1) ] [ Parallel (jobs > 1) ]
+--------------+ +--------+ +--------+
| Hook Process | | Hook 1 | | Hook 2 |
+--------------+ +--------+ +--------+
| | |
| stderr (inherited) | stderr pipe |
| | (captured) |
v v v
+-------------------------------------------------------------+
| Parent Process |
| |
| (direct write) [run-command (buffered)] |
| | | |
| | | writes |
| v v |
| +-------------------------------------------+ |
| | stderr (FD 2) | |
| +-------------------------------------------+ |
| | |
| | (dup2'd to pipe) |
| v |
| +-------------------------+ |
| | sideband "muxer" thread | |
| +-------------------------+ |
+-------------------------------------------------------------+
When use_sideband == 0, the sideband muxer thread is missing, so
the same architecture just outputs via the parent stderr stream.
See the following commits for the hook API conversions doing this,
using pre-existing sideband thread logic from `copy_to_sideband`.
Suggested-by: Jeff King <peff@peff•net>
Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora•com>
---
hook.c | 7 +++++--
hook.h | 9 +++++++++
2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/hook.c b/hook.c
index fde1f88ce8..aafed134af 100644
--- a/hook.c
+++ b/hook.c
@@ -152,8 +152,8 @@ int run_hooks_opt(struct repository *r, const char *hook_name,
.tr2_category = "hook",
.tr2_label = hook_name,
- .processes = 1,
- .ungroup = 1,
+ .processes = options->jobs,
+ .ungroup = options->jobs <= 1,
.get_next_task = pick_next_hook,
.start_failure = notify_start_failure,
@@ -169,6 +169,9 @@ int run_hooks_opt(struct repository *r, const char *hook_name,
if (options->path_to_stdin && options->feed_pipe)
BUG("options path_to_stdin and feed_pipe are mutually exclusive");
+ if (!options->jobs)
+ BUG("run_hooks_opt must be called with options.jobs >= 1");
+
if (options->invoked_hook)
*options->invoked_hook = 0;
diff --git a/hook.h b/hook.h
index 7cbeef0a1e..3db860bf3c 100644
--- a/hook.h
+++ b/hook.h
@@ -16,6 +16,14 @@ struct run_hooks_opt
/* Emit an error if the hook is missing */
unsigned int error_if_missing:1;
+ /**
+ * Number of processes to parallelize across.
+ *
+ * If > 1, output will be buffered and de-interleaved (ungroup=0).
+ * If == 1, output will be real-time (ungroup=1).
+ */
+ unsigned int jobs;
+
/**
* An optional initial working directory for the hook,
* translates to "struct child_process"'s "dir" member.
@@ -86,6 +94,7 @@ struct run_hooks_opt
.env = STRVEC_INIT, \
.args = STRVEC_INIT, \
.stdout_to_stderr = 1, \
+ .jobs = 1, \
}
struct hook_cb_data {
--
2.52.0.732.gb351b5166d.dirty
next prev parent reply other threads:[~2026-01-21 21:55 UTC|newest]
Thread overview: 187+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-25 12:53 [PATCH 00/10] Convert remaining hooks to hook.h Adrian Ratiu
2025-09-25 12:53 ` [PATCH 01/10] run-command: add stdin callback for parallelization Adrian Ratiu
2025-10-02 6:34 ` Patrick Steinhardt
2025-10-02 15:46 ` Junio C Hamano
2025-10-06 13:01 ` Adrian Ratiu
2025-10-06 12:59 ` Adrian Ratiu
2025-10-14 17:35 ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 02/10] hook: provide stdin via callback Adrian Ratiu
2025-09-25 20:05 ` Junio C Hamano
2025-09-26 12:03 ` Adrian Ratiu
2025-10-10 19:57 ` Emily Shaffer
2025-10-13 14:47 ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 03/10] hook: convert 'post-rewrite' hook in sequencer.c to hook.h Adrian Ratiu
2025-09-25 20:15 ` Junio C Hamano
2025-09-26 12:29 ` Adrian Ratiu
2025-09-26 14:12 ` Phillip Wood
2025-09-26 15:53 ` Adrian Ratiu
2025-09-29 10:11 ` Phillip Wood
2025-09-26 17:52 ` Junio C Hamano
2025-09-29 7:33 ` Adrian Ratiu
2025-10-02 6:34 ` Patrick Steinhardt
2025-10-08 7:04 ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 04/10] transport: convert pre-push hook " Adrian Ratiu
2025-09-25 18:58 ` D. Ben Knoble
2025-09-26 13:02 ` Adrian Ratiu
2025-09-26 14:11 ` Phillip Wood
2025-09-29 11:33 ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 05/10] reference-transaction: use hook.h to run hooks Adrian Ratiu
2025-09-25 21:45 ` Junio C Hamano
2025-09-26 13:03 ` Adrian Ratiu
2025-10-02 6:34 ` Patrick Steinhardt
2025-10-08 12:26 ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 06/10] run-command: allow capturing of collated output Adrian Ratiu
2025-09-25 21:52 ` Junio C Hamano
2025-09-26 14:14 ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 07/10] hooks: allow callers to capture output Adrian Ratiu
2025-09-25 12:53 ` [PATCH 08/10] receive-pack: convert 'update' hook to hook.h Adrian Ratiu
2025-09-25 21:53 ` Junio C Hamano
2025-10-10 19:57 ` Emily Shaffer
2025-10-17 8:27 ` Adrian Ratiu
2025-09-25 12:53 ` [PATCH 09/10] post-update: use hook.h library Adrian Ratiu
2025-09-25 18:02 ` [PATCH 10/10] receive-pack: convert receive hooks to hook.h Adrian Ratiu
2025-10-10 19:57 ` [PATCH 00/10] Convert remaining " Emily Shaffer
2025-10-17 14:15 ` [PATCH v2 " Adrian Ratiu
2025-10-17 14:15 ` [PATCH v2 01/10] run-command: add stdin callback for parallelization Adrian Ratiu
2025-10-21 7:40 ` Patrick Steinhardt
2025-10-17 14:15 ` [PATCH v2 02/10] hook: provide stdin via callback Adrian Ratiu
2025-10-21 7:41 ` Patrick Steinhardt
2025-10-21 7:41 ` Patrick Steinhardt
2025-10-21 14:44 ` Adrian Ratiu
2025-10-17 14:15 ` [PATCH v2 03/10] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2025-10-21 7:41 ` Patrick Steinhardt
2025-10-21 15:44 ` Adrian Ratiu
2025-10-17 14:15 ` [PATCH v2 04/10] transport: convert pre-push " Adrian Ratiu
2025-10-21 7:41 ` Patrick Steinhardt
2025-10-21 16:04 ` Adrian Ratiu
2025-10-17 14:15 ` [PATCH v2 05/10] reference-transaction: use hook API instead of run-command Adrian Ratiu
2025-10-17 14:15 ` [PATCH v2 06/10] hook: allow overriding the ungroup option Adrian Ratiu
2025-10-17 14:15 ` [PATCH v2 07/10] run-command: allow capturing of collated output Adrian Ratiu
2025-10-21 7:41 ` Patrick Steinhardt
2025-10-21 16:25 ` Adrian Ratiu
2025-10-17 14:15 ` [PATCH v2 08/10] hooks: allow callers to capture output Adrian Ratiu
2025-10-17 14:15 ` [PATCH v2 09/10] receive-pack: convert update hooks to new API Adrian Ratiu
2025-10-28 18:39 ` Kristoffer Haugsbakk
2025-10-17 14:15 ` [PATCH v2 10/10] receive-pack: convert receive hooks to hook API Adrian Ratiu
2025-10-21 7:41 ` Patrick Steinhardt
2025-10-28 18:42 ` Kristoffer Haugsbakk
2025-10-29 13:46 ` Adrian Ratiu
2025-10-29 13:50 ` Kristoffer Haugsbakk
2025-11-15 19:48 ` Junio C Hamano
2025-11-17 16:51 ` Adrian Ratiu
2025-10-21 7:40 ` [PATCH v2 00/10] Convert remaining hooks to hook.h Patrick Steinhardt
2025-10-21 16:34 ` Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 " Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 01/10] run-command: add stdin callback for parallelization Adrian Ratiu
2025-11-25 23:15 ` Junio C Hamano
2025-11-27 12:00 ` Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 02/10] hook: provide stdin via callback Adrian Ratiu
2025-11-29 13:03 ` Adrian Ratiu
2025-11-29 22:21 ` Junio C Hamano
2025-12-01 13:26 ` Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 03/10] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 04/10] transport: convert pre-push " Adrian Ratiu
2025-11-24 22:55 ` Junio C Hamano
2025-11-27 14:24 ` Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 05/10] reference-transaction: use hook API instead of run-command Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 06/10] hook: allow overriding the ungroup option Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 07/10] run-command: allow capturing of collated output Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 08/10] hooks: allow callers to capture output Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 09/10] receive-pack: convert update hooks to new API Adrian Ratiu
2025-11-24 17:20 ` [PATCH v3 10/10] receive-pack: convert receive hooks to hook API Adrian Ratiu
2025-12-04 14:15 ` [PATCH v4 00/11] Convert remaining hooks to hook.h Adrian Ratiu
2025-12-04 14:15 ` [PATCH v4 01/11] run-command: add first helper for pp child states Adrian Ratiu
2025-12-04 14:15 ` [PATCH v4 02/11] run-command: add stdin callback for parallelization Adrian Ratiu
2025-12-04 14:15 ` [PATCH v4 03/11] hook: provide stdin via callback Adrian Ratiu
2025-12-16 8:08 ` Patrick Steinhardt
2025-12-04 14:15 ` [PATCH v4 04/11] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2025-12-04 14:15 ` [PATCH v4 05/11] transport: convert pre-push " Adrian Ratiu
2025-12-16 8:08 ` Patrick Steinhardt
2025-12-16 9:09 ` Adrian Ratiu
2025-12-16 9:30 ` Patrick Steinhardt
2025-12-17 23:07 ` Junio C Hamano
2025-12-04 14:15 ` [PATCH v4 06/11] reference-transaction: use hook API instead of run-command Adrian Ratiu
2025-12-04 14:15 ` [PATCH v4 07/11] hook: allow overriding the ungroup option Adrian Ratiu
2025-12-04 14:15 ` [PATCH v4 08/11] run-command: allow capturing of collated output Adrian Ratiu
2025-12-04 14:15 ` [PATCH v4 09/11] hooks: allow callers to capture output Adrian Ratiu
2025-12-04 14:15 ` [PATCH v4 10/11] receive-pack: convert update hooks to new API Adrian Ratiu
2025-12-16 8:08 ` Patrick Steinhardt
2025-12-16 9:22 ` Adrian Ratiu
2025-12-04 14:15 ` [PATCH v4 11/11] receive-pack: convert receive hooks to hook API Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 00/11] Convert remaining hooks to hook.h Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 01/11] run-command: add first helper for pp child states Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 02/11] run-command: add stdin callback for parallelization Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 03/11] hook: provide stdin via callback Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 04/11] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 05/11] transport: convert pre-push " Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 06/11] reference-transaction: use hook API instead of run-command Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 07/11] hook: allow overriding the ungroup option Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 08/11] run-command: allow capturing of collated output Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 09/11] hooks: allow callers to capture output Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 10/11] receive-pack: convert update hooks to new API Adrian Ratiu
2025-12-18 17:11 ` [PATCH v5 11/11] receive-pack: convert receive hooks to hook API Adrian Ratiu
2025-12-19 12:38 ` Patrick Steinhardt
2025-12-20 10:40 ` Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 00/11] Convert remaining hooks to hook.h Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 01/11] run-command: add first helper for pp child states Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 02/11] run-command: add stdin callback for parallelization Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 03/11] hook: provide stdin via callback Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 04/11] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 05/11] transport: convert pre-push " Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 06/11] reference-transaction: use hook API instead of run-command Adrian Ratiu
2026-01-18 12:23 ` SZEDER Gábor
2026-01-18 18:30 ` Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 07/11] hook: allow overriding the ungroup option Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 08/11] run-command: allow capturing of collated output Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 09/11] hooks: allow callers to capture output Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 10/11] receive-pack: convert update hooks to new API Adrian Ratiu
2025-12-26 12:23 ` [PATCH v6 11/11] receive-pack: convert receive hooks to hook API Adrian Ratiu
2025-12-28 11:32 ` [PATCH v6 00/11] Convert remaining hooks to hook.h Junio C Hamano
2026-01-05 10:52 ` Adrian Ratiu
2026-01-05 12:13 ` Patrick Steinhardt
2026-01-21 21:54 ` [PATCH v7 00/12] " Adrian Ratiu
2026-01-21 21:54 ` [PATCH v7 01/12] t1800: add hook output stream tests Adrian Ratiu
2026-01-21 22:16 ` Junio C Hamano
2026-01-22 9:19 ` Adrian Ratiu
2026-01-21 21:54 ` [PATCH v7 02/12] run-command: add first helper for pp child states Adrian Ratiu
2026-01-21 23:01 ` Junio C Hamano
2026-01-22 9:21 ` Adrian Ratiu
2026-01-21 21:54 ` [PATCH v7 03/12] run-command: add stdin callback for parallelization Adrian Ratiu
2026-01-21 21:54 ` [PATCH v7 04/12] hook: provide stdin via callback Adrian Ratiu
2026-01-21 21:54 ` [PATCH v7 05/12] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2026-01-21 21:54 ` [PATCH v7 06/12] hook: allow separate std[out|err] streams Adrian Ratiu
2026-01-23 7:19 ` Patrick Steinhardt
2026-01-23 7:47 ` Adrian Ratiu
2026-01-21 21:54 ` [PATCH v7 07/12] transport: convert pre-push to hook API Adrian Ratiu
2026-01-21 21:54 ` [PATCH v7 08/12] reference-transaction: use hook API instead of run-command Adrian Ratiu
2026-01-21 21:54 ` Adrian Ratiu [this message]
2026-01-21 21:54 ` [PATCH v7 10/12] run-command: poll child stdin in addition to stdout Adrian Ratiu
2026-01-21 22:04 ` Kristoffer Haugsbakk
2026-01-22 9:57 ` Adrian Ratiu
2026-01-21 23:11 ` Junio C Hamano
2026-01-22 10:58 ` Adrian Ratiu
2026-01-22 17:20 ` Junio C Hamano
2026-01-26 23:20 ` Emily Shaffer
2026-01-27 0:11 ` Junio C Hamano
2026-01-27 10:10 ` Adrian Ratiu
2026-01-21 21:54 ` [PATCH v7 11/12] receive-pack: convert update hooks to new API Adrian Ratiu
2026-01-21 22:14 ` Kristoffer Haugsbakk
2026-01-22 9:26 ` Adrian Ratiu
2026-01-27 0:12 ` Emily Shaffer
2026-01-27 13:05 ` Adrian Ratiu
2026-01-21 21:54 ` [PATCH v7 12/12] receive-pack: convert receive hooks to hook API Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 00/12] Convert remaining hooks to hook.h Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 01/12] t1800: add hook output stream tests Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 02/12] run-command: add helper for pp child states Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 03/12] run-command: add stdin callback for parallelization Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 04/12] hook: provide stdin via callback Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 05/12] hook: convert 'post-rewrite' hook in sequencer.c to hook API Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 06/12] hook: allow separate std[out|err] streams Adrian Ratiu
2026-02-02 3:17 ` Chris Darroch
2026-02-02 16:32 ` Junio C Hamano
2026-01-28 21:39 ` [PATCH v8 07/12] transport: convert pre-push to hook API Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 08/12] reference-transaction: use hook API instead of run-command Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 09/12] hook: add jobs option Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 10/12] run-command: poll child input in addition to output Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 11/12] receive-pack: convert update hooks to new API Adrian Ratiu
2026-01-28 21:39 ` [PATCH v8 12/12] receive-pack: convert receive hooks to hook API Adrian Ratiu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260121215436.1473800-10-adrian.ratiu@collabora.com \
--to=adrian.ratiu@collabora$(echo .)com \
--cc=emilyshaffer@google$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=gitster@pobox$(echo .)com \
--cc=kristofferhaugsbakk@fastmail$(echo .)com \
--cc=peff@peff$(echo .)net \
--cc=ps@pks$(echo .)im \
--cc=steadmon@google$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox