From: "brian m. carlson" <sandals@crustytoothpaste•net>
To: Martin Fick <mfick@nvidia•com>
Cc: "git@vger•kernel.org" <git@vger•kernel.org>
Subject: Re: Slow git pack-refs --all
Date: Thu, 25 Dec 2025 23:38:30 +0000 [thread overview]
Message-ID: <aU3K9lGbHw68Vv5U@fruit.crustytoothpaste.net> (raw)
In-Reply-To: <CH3PR12MB9026B5872FD42F031970074BC2B3A@CH3PR12MB9026.namprd12.prod.outlook.com>
[-- Attachment #1: Type: text/plain, Size: 1864 bytes --]
On 2025-12-25 at 22:13:54, Martin Fick wrote:
> Although the packed-refs file is large, copying it takes less than 1s,
> so there isn't a writing throughput issue with the filesystem.
> Additionally, jgit can pack-refs --all in under 20s on the same repo,
> so I don't believe there is an issue locking the 200 loose refs
> either. When observing the filesystem, I do see the packed-refs.new
> growing at a rate that seems slower than expected as if much more is
> happening while writing this file, than just writing the file.
>
> An strace shows about 200+ open("./objects..") calls interspersed
> between around ~26K write() calls. I am surprised to see pack-refs
> reading objects at all.
I think this is from `should_pack_ref`:
/* Do not pack broken refs: */
if (!ref_resolves_to_object(ref->name, refs->base.repo, ref->oid, ref->flags))
return 0;
So Git is going to need to verify that the object at least exists. I
don't know why we would need to _open_ them, however. Perhaps someone
else has ideas.
> Although the repository is not in terrible shape before packing refs
> (~1500 loose objects, 37pack files). Surprisingly, repacking the repo
> first does speed it up so that packing refs then takes under 20s.
>
> This repository is on NFS.
That's almost certainly part of your performance problem, too. Loading
a single pack file and index is going to be way, way faster than making
lots of network calls to open 37 pack file and 37 index files, plus at
least stat some loose objects.
I will note that at least some forges always have Git write pack files
and try to avoid loose objects altogether since that almost always
improves performance. You may want to set `receive.unpackLimit` to 1 to
see if that helps in the general case.
--
brian m. carlson (they/them)
Toronto, Ontario, CA
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]
next prev parent reply other threads:[~2025-12-25 23:38 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-12-25 22:13 Slow git pack-refs --all Martin Fick
2025-12-25 23:38 ` brian m. carlson [this message]
2025-12-26 4:45 ` Jeff King
2025-12-26 17:15 ` brian m. carlson
2025-12-27 7:36 ` Jeff King
2025-12-31 5:48 ` Martin Fick
2026-01-02 7:49 ` Jeff King
2026-01-05 23:45 ` Martin Fick
2026-01-06 6:53 ` Patrick Steinhardt
2026-01-06 23:02 ` Martin Fick
2026-01-07 11:42 ` Patrick Steinhardt
2026-01-07 22:58 ` Martin Fick
2026-01-08 6:33 ` Patrick Steinhardt
2026-01-15 21:09 ` Jeff King
2026-01-16 20:35 ` Martin Fick
2026-01-07 17:05 ` Martin Fick
2026-01-06 10:38 ` Jeff King
2026-01-06 23:03 ` Martin Fick
2025-12-31 5:39 ` Martin Fick
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aU3K9lGbHw68Vv5U@fruit.crustytoothpaste.net \
--to=sandals@crustytoothpaste$(echo .)net \
--cc=git@vger$(echo .)kernel.org \
--cc=mfick@nvidia$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox