From: "brian m. carlson" <sandals@crustytoothpaste•net>
To: Patrick Steinhardt <ps@pks•im>
Cc: git@vger•kernel.org, Karthik Nayak <karthik.188@gmail•com>,
Jeff King <peff@peff•net>, Junio C Hamano <gitster@pobox•com>,
Christian Couder <chriscool@tuxfamily•org>
Subject: Re: [PATCH 00/14] refs: batch refname availability checks
Date: Tue, 18 Feb 2025 17:10:22 +0000 [thread overview]
Message-ID: <Z7S-_iA73twblpvP@tapette.crustytoothpaste.net> (raw)
In-Reply-To: <20250217-pks-update-ref-optimization-v1-0-a2b6d87a24af@pks.im>
[-- Attachment #1: Type: text/plain, Size: 2995 bytes --]
On 2025-02-17 at 15:50:14, Patrick Steinhardt wrote:
> But more importantly, this refactoring also has a positive effect when
> updating references in a repository with preexisting refs, which I
> consider to be the more realistic scenario. The following benchmark
> creates 10k refs with 100k preexisting refs.
>
> With the "files" backend we see a modest improvement:
>
> Benchmark 1: update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = master)
> Time (mean ± σ): 470.1 ms ± 5.4 ms [User: 104.5 ms, System: 363.1 ms]
> Range (min … max): 465.7 ms … 484.3 ms 10 runs
>
> Benchmark 2: update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = HEAD)
> Time (mean ± σ): 407.8 ms ± 5.4 ms [User: 66.0 ms, System: 340.0 ms]
> Range (min … max): 399.9 ms … 417.6 ms 10 runs
>
> Summary
> update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = HEAD) ran
> 1.15 ± 0.02 times faster than update-ref: create many refs (refformat = files, preexisting = 100000, new = 10000, revision = master)
>
> But with the "reftable" backend we see an almost 5x improvement, where
> it's now ~15x faster than the "files" backend:
>
> Benchmark 1: update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = master)
> Time (mean ± σ): 153.9 ms ± 2.0 ms [User: 96.5 ms, System: 56.6 ms]
> Range (min … max): 150.5 ms … 158.4 ms 18 runs
>
> Benchmark 2: update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = HEAD)
> Time (mean ± σ): 32.2 ms ± 1.2 ms [User: 27.6 ms, System: 4.3 ms]
> Range (min … max): 29.8 ms … 38.6 ms 71 runs
>
> Summary
> update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = HEAD) ran
> 4.78 ± 0.19 times faster than update-ref: create many refs (refformat = reftable, preexisting = 100000, new = 10000, revision = master)
I'm glad to see this performance speedup. That's a really nice
improvement.
> The series is structured as follows:
>
> - Patches 1 to 4 implement the logic to skip ambiguity checks in
> git-update-ref(1).
>
> - Patch 5 and 6 introduce batched checks.
>
> - Patch 7 deduplicates the ref prefix checks.
>
> - Patch 8 to 14 implement the infrastructure to reseek iterators.
>
> - Patch 15 starts to reuse iterators for nested ref checks.
I took a look at this series and I didn't find anything that stood out
to me as a problem. I will say that the reftable code isn't my forte,
so please don't take this as a formal review, but I am definitely
positive on the series in the general sense.
--
brian m. carlson (they/them or he/him)
Toronto, Ontario, CA
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]
next prev parent reply other threads:[~2025-02-18 17:10 UTC|newest]
Thread overview: 169+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-02-17 15:50 [PATCH 00/14] refs: batch refname availability checks Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 01/14] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 02/14] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 03/14] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-02-18 16:04 ` Karthik Nayak
2025-02-17 15:50 ` [PATCH 04/14] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 05/14] refs/reftable: start using `refs_verify_refnames_available()` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 06/14] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-18 16:12 ` Karthik Nayak
2025-02-19 11:52 ` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 07/14] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-18 16:52 ` shejialuo
2025-02-19 11:52 ` Patrick Steinhardt
2025-02-19 12:41 ` shejialuo
2025-02-19 12:59 ` Patrick Steinhardt
2025-02-19 13:06 ` shejialuo
2025-02-19 13:17 ` Patrick Steinhardt
2025-02-19 13:20 ` Patrick Steinhardt
2025-02-19 13:23 ` shejialuo
2025-02-18 17:13 ` Karthik Nayak
2025-02-19 11:52 ` Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 08/14] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 09/14] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-19 20:10 ` Karthik Nayak
2025-02-17 15:50 ` [PATCH 10/14] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-02-19 20:13 ` Karthik Nayak
2025-02-17 15:50 ` [PATCH 11/14] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 12/14] refs/iterator: implement seeking for `packed-ref` iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 13/14] refs/iterator: implement seeking for "files" iterators Patrick Steinhardt
2025-02-17 15:50 ` [PATCH 14/14] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-02-18 17:10 ` brian m. carlson [this message]
2025-02-19 13:23 ` [PATCH v2 00/16] refs: batch refname availability checks Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-19 17:02 ` Justin Tobler
2025-02-19 13:23 ` [PATCH v2 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-02-21 8:00 ` Jeff King
2025-02-21 8:36 ` Patrick Steinhardt
2025-02-21 9:06 ` Jeff King
2025-02-19 13:23 ` [PATCH v2 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-02-19 18:21 ` Justin Tobler
2025-02-20 8:05 ` Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 05/16] refs/reftable: " Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-24 13:08 ` shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-24 13:37 ` shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-02-24 14:00 ` shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-24 14:49 ` shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 14/16] refs/iterator: implement seeking for `packed-ref` iterators Patrick Steinhardt
2025-02-24 15:09 ` shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-25 12:07 ` shejialuo
2025-02-19 13:23 ` [PATCH v2 15/16] refs/iterator: implement seeking for "files" iterators Patrick Steinhardt
2025-02-19 13:23 ` [PATCH v2 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-02-24 15:14 ` shejialuo
2025-02-24 15:18 ` [PATCH v2 00/16] refs: batch refname availability checks shejialuo
2025-02-25 7:39 ` Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 " Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-02-26 22:26 ` Junio C Hamano
2025-02-27 11:57 ` Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 05/16] refs/reftable: " Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-02-25 8:55 ` [PATCH v3 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-25 8:56 ` [PATCH v3 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-02-25 8:56 ` [PATCH v3 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-02-25 8:56 ` [PATCH v3 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 00/16] refs: batch refname availability checks Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-03-06 13:21 ` Karthik Nayak
2025-02-28 9:26 ` [PATCH v4 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-03-06 13:47 ` Karthik Nayak
2025-02-28 9:26 ` [PATCH v4 05/16] refs/reftable: " Patrick Steinhardt
2025-03-06 14:00 ` Karthik Nayak
2025-03-06 14:12 ` Karthik Nayak
2025-03-06 15:13 ` Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-03-06 14:10 ` Karthik Nayak
2025-02-28 9:26 ` [PATCH v4 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-03-06 14:16 ` Karthik Nayak
2025-02-28 9:26 ` [PATCH v4 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-02-28 9:26 ` [PATCH v4 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-03-06 14:20 ` [PATCH v4 00/16] refs: batch refname availability checks Karthik Nayak
2025-03-06 15:08 ` [PATCH v5 " Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-03-12 12:12 ` shejialuo
2025-03-06 15:08 ` [PATCH v5 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-03-12 12:36 ` shejialuo
2025-03-12 12:44 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 05/16] refs/reftable: " Patrick Steinhardt
2025-03-12 12:54 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-03-12 12:58 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-03-12 13:06 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-03-12 13:22 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-03-12 13:45 ` shejialuo
2025-03-12 15:36 ` Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-03-06 15:08 ` [PATCH v5 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-03-06 15:32 ` [PATCH v5 00/16] refs: batch refname availability checks Karthik Nayak
2025-03-12 14:03 ` shejialuo
2025-03-12 15:56 ` [PATCH v6 " Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 01/16] object-name: introduce `repo_get_oid_with_flags()` Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 02/16] object-name: allow skipping ambiguity checks in `get_oid()` family Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 03/16] builtin/update-ref: skip ambiguity checks when parsing object IDs Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 04/16] refs: introduce function to batch refname availability checks Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 05/16] refs/reftable: " Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 06/16] refs/files: batch refname availability checks for normal transactions Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 07/16] refs/files: batch refname availability checks for initial transactions Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 08/16] refs: stop re-verifying common prefixes for availability Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 09/16] refs/iterator: separate lifecycle from iteration Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 10/16] refs/iterator: provide infrastructure to re-seek iterators Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 11/16] refs/iterator: implement seeking for merged iterators Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 12/16] refs/iterator: implement seeking for reftable iterators Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 13/16] refs/iterator: implement seeking for ref-cache iterators Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 14/16] refs/iterator: implement seeking for packed-ref iterators Patrick Steinhardt
2025-04-03 19:56 ` Elijah Newren
2025-04-03 22:18 ` brian m. carlson
2025-04-04 7:18 ` shejialuo
2025-04-04 10:00 ` Patrick Steinhardt
2025-04-04 10:05 ` Patrick Steinhardt
2025-04-04 10:59 ` Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 15/16] refs/iterator: implement seeking for files iterators Patrick Steinhardt
2025-03-12 15:56 ` [PATCH v6 16/16] refs: reuse iterators when determining refname availability Patrick Steinhardt
2025-03-13 2:57 ` [PATCH v6 00/16] refs: batch refname availability checks shejialuo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Z7S-_iA73twblpvP@tapette.crustytoothpaste.net \
--to=sandals@crustytoothpaste$(echo .)net \
--cc=chriscool@tuxfamily$(echo .)org \
--cc=git@vger$(echo .)kernel.org \
--cc=gitster@pobox$(echo .)com \
--cc=karthik.188@gmail$(echo .)com \
--cc=peff@peff$(echo .)net \
--cc=ps@pks$(echo .)im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox