* Slow git pack-refs --all
@ 2025-12-25 22:13 Martin Fick
2025-12-25 23:38 ` brian m. carlson
0 siblings, 1 reply; 19+ messages in thread
From: Martin Fick @ 2025-12-25 22:13 UTC (permalink / raw)
To: git@vger•kernel.org
I was hoping to get some help debugging a busy large repository where git pack-refs --all tends to regularly take over 5mins to run in production. As you can imagine this is particularly problematic on a busy Gerrit server since it tends to hold the packed-refs.lock file for most of this duration. Any help is greatly appreciated, see the details below.
-Martin
What did you do before the bug happened? (Steps to reproduce your issue)
I have a large repository (~90M objects, ~50GB, ~3M refs) which is regularly (every ~2hours) repacked and maintained, but generally gets at least 300+ updates per maintenance cycle.
What did you expect to happen? (Expected behavior)
git pack-refs --all to complete in under 20s when there are only 200 loose refs
What happened instead? (Actual behavior)
git pack-refs --all takes more than 3 minutes
What's different between what you expected and what actually happened?
This is much slower than expected
Anything else you want to add:
Although the packed-refs file is large, copying it takes less than 1s, so there isn't a writing throughput issue with the filesystem. Additionally, jgit can pack-refs --all in under 20s on the same repo, so I don't believe there is an issue locking the 200 loose refs either. When observing the filesystem, I do see the packed-refs.new growing at a rate that seems slower than expected as if much more is happening while writing this file, than just writing the file.
An strace shows about 200+ open("./objects..") calls interspersed between around ~26K write() calls. I am surprised to see pack-refs reading objects at all.
Although the repository is not in terrible shape before packing refs (~1500 loose objects, 37pack files). Surprisingly, repacking the repo first does speed it up so that packing refs then takes under 20s.
This repository is on NFS.
[System Info]
git version:
git version 2.45.2
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname: Linux 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64
compiler info: gnuc: 14.1
libc info: glibc: 2.17
$SHELL (typically, interactive shell): /bin/bash
[Enabled Hooks]
not run from a git repository - no hooks to show
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2025-12-25 22:13 Slow git pack-refs --all Martin Fick
@ 2025-12-25 23:38 ` brian m. carlson
2025-12-26 4:45 ` Jeff King
2025-12-31 5:39 ` Martin Fick
0 siblings, 2 replies; 19+ messages in thread
From: brian m. carlson @ 2025-12-25 23:38 UTC (permalink / raw)
To: Martin Fick; +Cc: git@vger•kernel.org
[-- Attachment #1: Type: text/plain, Size: 1864 bytes --]
On 2025-12-25 at 22:13:54, Martin Fick wrote:
> Although the packed-refs file is large, copying it takes less than 1s,
> so there isn't a writing throughput issue with the filesystem.
> Additionally, jgit can pack-refs --all in under 20s on the same repo,
> so I don't believe there is an issue locking the 200 loose refs
> either. When observing the filesystem, I do see the packed-refs.new
> growing at a rate that seems slower than expected as if much more is
> happening while writing this file, than just writing the file.
>
> An strace shows about 200+ open("./objects..") calls interspersed
> between around ~26K write() calls. I am surprised to see pack-refs
> reading objects at all.
I think this is from `should_pack_ref`:
/* Do not pack broken refs: */
if (!ref_resolves_to_object(ref->name, refs->base.repo, ref->oid, ref->flags))
return 0;
So Git is going to need to verify that the object at least exists. I
don't know why we would need to _open_ them, however. Perhaps someone
else has ideas.
> Although the repository is not in terrible shape before packing refs
> (~1500 loose objects, 37pack files). Surprisingly, repacking the repo
> first does speed it up so that packing refs then takes under 20s.
>
> This repository is on NFS.
That's almost certainly part of your performance problem, too. Loading
a single pack file and index is going to be way, way faster than making
lots of network calls to open 37 pack file and 37 index files, plus at
least stat some loose objects.
I will note that at least some forges always have Git write pack files
and try to avoid loose objects altogether since that almost always
improves performance. You may want to set `receive.unpackLimit` to 1 to
see if that helps in the general case.
--
brian m. carlson (they/them)
Toronto, Ontario, CA
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2025-12-25 23:38 ` brian m. carlson
@ 2025-12-26 4:45 ` Jeff King
2025-12-26 17:15 ` brian m. carlson
2025-12-31 5:48 ` Martin Fick
2025-12-31 5:39 ` Martin Fick
1 sibling, 2 replies; 19+ messages in thread
From: Jeff King @ 2025-12-26 4:45 UTC (permalink / raw)
To: brian m. carlson; +Cc: Martin Fick, git@vger•kernel.org
On Thu, Dec 25, 2025 at 11:38:30PM +0000, brian m. carlson wrote:
> I think this is from `should_pack_ref`:
>
> /* Do not pack broken refs: */
> if (!ref_resolves_to_object(ref->name, refs->base.repo, ref->oid, ref->flags))
> return 0;
>
> So Git is going to need to verify that the object at least exists. I
> don't know why we would need to _open_ them, however. Perhaps someone
> else has ideas.
The packed-refs file stores tag-peeling information. So pack-refs opens
the object for any newly written ref via peel_object(), which has to at
least read the header to get the type. That call happens via
write_with_updates() in packed-backend.c.
If we wanted to be really pedantic, anything in refs/heads/ should not
point to a non-commit and thus should never need to be peeled. I'm not
sure if we want to embed that assumption in this code path, though
(nor would it necessarily help Martin's case if the refs are not in
refs/heads anyway).
-Peff
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2025-12-26 4:45 ` Jeff King
@ 2025-12-26 17:15 ` brian m. carlson
2025-12-27 7:36 ` Jeff King
2025-12-31 5:48 ` Martin Fick
1 sibling, 1 reply; 19+ messages in thread
From: brian m. carlson @ 2025-12-26 17:15 UTC (permalink / raw)
To: Jeff King; +Cc: Martin Fick, git@vger•kernel.org
[-- Attachment #1: Type: text/plain, Size: 1579 bytes --]
On 2025-12-26 at 04:45:07, Jeff King wrote:
> On Thu, Dec 25, 2025 at 11:38:30PM +0000, brian m. carlson wrote:
>
> > I think this is from `should_pack_ref`:
> >
> > /* Do not pack broken refs: */
> > if (!ref_resolves_to_object(ref->name, refs->base.repo, ref->oid, ref->flags))
> > return 0;
> >
> > So Git is going to need to verify that the object at least exists. I
> > don't know why we would need to _open_ them, however. Perhaps someone
> > else has ideas.
>
> The packed-refs file stores tag-peeling information. So pack-refs opens
> the object for any newly written ref via peel_object(), which has to at
> least read the header to get the type. That call happens via
> write_with_updates() in packed-backend.c.
>
> If we wanted to be really pedantic, anything in refs/heads/ should not
> point to a non-commit and thus should never need to be peeled. I'm not
> sure if we want to embed that assumption in this code path, though
> (nor would it necessarily help Martin's case if the refs are not in
> refs/heads anyway).
I don't think that would be a good idea. I know that people definitely
do updates of the loose refs by hand (although they should not) and so
it's entirely possible for them to contain invalid values, such as
having branches contain non-commit objects.
I wonder if reftable would avoid the need for this kind of expensive
check since it would already have the data peeled if need be and
wouldn't need to recompute the values.
--
brian m. carlson (they/them)
Toronto, Ontario, CA
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2025-12-26 17:15 ` brian m. carlson
@ 2025-12-27 7:36 ` Jeff King
0 siblings, 0 replies; 19+ messages in thread
From: Jeff King @ 2025-12-27 7:36 UTC (permalink / raw)
To: brian m. carlson; +Cc: Martin Fick, git@vger•kernel.org
On Fri, Dec 26, 2025 at 05:15:31PM +0000, brian m. carlson wrote:
> > If we wanted to be really pedantic, anything in refs/heads/ should not
> > point to a non-commit and thus should never need to be peeled. I'm not
> > sure if we want to embed that assumption in this code path, though
> > (nor would it necessarily help Martin's case if the refs are not in
> > refs/heads anyway).
>
> I don't think that would be a good idea. I know that people definitely
> do updates of the loose refs by hand (although they should not) and so
> it's entirely possible for them to contain invalid values, such as
> having branches contain non-commit objects.
Yeah, that matches my inclination.
> I wonder if reftable would avoid the need for this kind of expensive
> check since it would already have the data peeled if need be and
> wouldn't need to recompute the values.
It does the same amount of peeling, but it's amortized across more
operations (i.e., whatever did those ref updates in the first place)
rather than during the pack operation. And of course there really is no
pack operation per se with reftables, but I believe it avoids re-peeling
when rewriting entries during compaction.
It might actually do fewer object accesses overall if the ref-writing
operations have already loaded the objects in question (and thus it
knows whether they're tags or not, and may even have parsed tags in
memory). It can also do more in some cases (e.g., two loose writes will
peel for each write, whereas the files backend only bothers to peel
during packing).
-Peff
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2025-12-25 23:38 ` brian m. carlson
2025-12-26 4:45 ` Jeff King
@ 2025-12-31 5:39 ` Martin Fick
1 sibling, 0 replies; 19+ messages in thread
From: Martin Fick @ 2025-12-31 5:39 UTC (permalink / raw)
To: brian m. carlson; +Cc: git@vger•kernel.org
>From: brian m. carlson
> Sent: Thursday, December 25, 2025 4:38 PM
> On 2025-12-25 at 22:13:54, Martin Fick wrote:
>> Although the repository is not in terrible shape before packing refs
>> (~1500 loose objects, 37pack files). Surprisingly, repacking the repo
>> first does speed it up so that packing refs then takes under 20s.
>>
>> This repository is on NFS.
> That's almost certainly part of your performance problem, too. Loading
> a single pack file and index is going to be way, way faster than making
> lots of network calls to open 37 pack file and 37 index files, plus at
> least stat some loose objects.
>
> I will note that at least some forges always have Git write pack files
> and try to avoid loose objects altogether since that almost always
> improves performance. You may want to set `receive.unpackLimit` to 1 to
> see if that helps in the general case.
This would not explain why jgit can pack-refs much faster, since it has to deal with these NFS latencies also. In my experience with jgit, we generally don't see performance issues unless packfile counts exceed 300 or so on NFS, definitely not with only 37 of them. It could be that git is doing some things less efficiently here, but I would be pretty surprised if git could not also perform well typically on NFS with only 37 packfiles. I don't think that 200 something object lookups, should ever take 3+mins, even on NFS, and even with way more packfiles than this. To be that slow, it would have to take about 1s per lookup! Something really seems fishy here to me. I have to think that somehow it isn't these reads that are slow, but I can't explain what else it would be?
-Martin
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2025-12-26 4:45 ` Jeff King
2025-12-26 17:15 ` brian m. carlson
@ 2025-12-31 5:48 ` Martin Fick
2026-01-02 7:49 ` Jeff King
1 sibling, 1 reply; 19+ messages in thread
From: Martin Fick @ 2025-12-31 5:48 UTC (permalink / raw)
To: Jeff King, brian m. carlson; +Cc: git@vger•kernel.org
> From: Jeff King <peff@peff•net>
> Sent: Thursday, December 25, 2025 9:45 PM
>
> On Thu, Dec 25, 2025 at 11:38:30PM +0000, brian m. carlson wrote:
>> I think this is from `should_pack_ref`:
>>
>> /* Do not pack broken refs: */
>> if (!ref_resolves_to_object(ref->name, refs->base.repo, ref->oid, ref->flags))
>> return 0;
>>
>> So Git is going to need to verify that the object at least exists. I
>> don't know why we would need to _open_ them, however. Perhaps someone
>> else has ideas.
>
>The packed-refs file stores tag-peeling information. So pack-refs opens
>the object for any newly written ref via peel_object(), which has to at
>least read the header to get the type. That call happens via
>write_with_updates() in packed-backend.c.
Thanks, this makes sense. However, since jgit needs to peel these objects also, it doesn't make sense to me that this would be the bottleneck unless git is doing something terribly inefficient here. :(
Except for the fact that repacking objects made it faster, my observations make it look like it's the writing that is actually slow, not the reads. Could there be too many small unbuffered writes, could this write path have missed being optimized (it likely isn't used elsewhere)?
-Martin
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2025-12-31 5:48 ` Martin Fick
@ 2026-01-02 7:49 ` Jeff King
2026-01-05 23:45 ` Martin Fick
0 siblings, 1 reply; 19+ messages in thread
From: Jeff King @ 2026-01-02 7:49 UTC (permalink / raw)
To: Martin Fick; +Cc: brian m. carlson, git@vger•kernel.org
On Wed, Dec 31, 2025 at 05:48:11AM +0000, Martin Fick wrote:
> >> I think this is from `should_pack_ref`:
> >>
> >> /* Do not pack broken refs: */
> >> if (!ref_resolves_to_object(ref->name, refs->base.repo, ref->oid, ref->flags))
> >> return 0;
> >>
> >> So Git is going to need to verify that the object at least exists. I
> >> don't know why we would need to _open_ them, however. Perhaps someone
> >> else has ideas.
> >
> >The packed-refs file stores tag-peeling information. So pack-refs opens
> >the object for any newly written ref via peel_object(), which has to at
> >least read the header to get the type. That call happens via
> >write_with_updates() in packed-backend.c.
>
> Thanks, this makes sense. However, since jgit needs to peel these
> objects also, it doesn't make sense to me that this would be the
> bottleneck unless git is doing something terribly inefficient here. :(
I'd expect both git and jgit to open each loose object once. I tried
running "git pack-refs --all --prune" under strace on a test repo. It
does seem to open the object once. Then I tried the same with jgit
(though it does not understand --prune), and got the same results.
So...I dunno.
> Except for the fact that repacking objects made it faster, my
> observations make it look like it's the writing that is actually slow,
> not the reads. Could there be too many small unbuffered writes, could
> this write path have missed being optimized (it likely isn't used
> elsewhere)?
All of the packed-refs writes are through fprintf(), which should be
fully buffered. You should be able to confirm with strace (I get
4096-byte writes on my system).
If writing were slow, I'd also expect that to scale with the total
number of refs, not the number of changed refs (since we have to rewrite
the whole file, but only new entries need to be peeled).
-Peff
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2026-01-02 7:49 ` Jeff King
@ 2026-01-05 23:45 ` Martin Fick
2026-01-06 6:53 ` Patrick Steinhardt
2026-01-06 10:38 ` Jeff King
0 siblings, 2 replies; 19+ messages in thread
From: Martin Fick @ 2026-01-05 23:45 UTC (permalink / raw)
To: Jeff King; +Cc: brian m. carlson, git@vger•kernel.org
> From: Jeff King <peff@peff•net>
> Sent: Friday, January 2, 2026 12:49 AM
> On Wed, Dec 31, 2025 at 05:48:11AM +0000, Martin Fick wrote:
> > Except for the fact that repacking objects made it faster, my
I have now confirmed that repacking does NOT actually make things
faster. I believe filesystem caching interfered with much of my
testing. Using echo 3 > /proc/sys/vm/drop_caches has helped to
get more consistent results.
By repacking to get one used, and one cruft pack only, and no loose
objects, I have confirmed that pack-refs it is still slow. This rules out the
idea that the loose object, or pack file counts were making things slow.
> > observations make it look like it's the writing that is actually slow,
> > not the reads. Could there be too many small unbuffered writes, could
> > this write path have missed being optimized (it likely isn't used
> > elsewhere)?
>
> All of the packed-refs writes are through fprintf(), which should be
> fully buffered. You should be able to confirm with strace (I get
> 4096-byte writes on my system).
I can confirm that my system is actually printing 8192 bytes at a time.
> If writing were slow, I'd also expect that to scale with the total
> number of refs, not the number of changed refs (since we have to rewrite
> the whole file, but only new entries need to be peeled).
OK, after discovering the strace -r and -T options, I have determined that
the 29K writes were all very fast in themselves. However, most of the
writes seem to follow each other with no other system calls in between.
This explains why it looks like the writes are slow, even though they aren't.
If I tally up the time between the previous system call, and each write(),
it adds up to the bulk of the time (4mins out of 4m15s) that it takes to
pack refs. This tells me that no visible I/O or system calls are the problem,
but rather that the program itself is taking a long time between writes.
I very much doubt that this is heavy CPU time, but rather I am going to
guess that this is hidden system time spent accessing mmaped memory.
Could it be really slow reading the packed-refs file? I can see the
packed-refs file is mmaped() before the writes start, and then
munmapped after the writes are completed. If I had to guess, that likely
means that the packed-refs file is being read in small increments by the
kernel via mmap, and that is what is making things very slow over NFS.
My alternative theory, is that each ref is being looked up via a binary
search, but I don't think git does this?
-Martin
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2026-01-05 23:45 ` Martin Fick
@ 2026-01-06 6:53 ` Patrick Steinhardt
2026-01-06 23:02 ` Martin Fick
2026-01-07 17:05 ` Martin Fick
2026-01-06 10:38 ` Jeff King
1 sibling, 2 replies; 19+ messages in thread
From: Patrick Steinhardt @ 2026-01-06 6:53 UTC (permalink / raw)
To: Martin Fick; +Cc: Jeff King, brian m. carlson, git@vger•kernel.org
Hi Martin,
On Mon, Jan 05, 2026 at 11:45:41PM +0000, Martin Fick wrote:
> OK, after discovering the strace -r and -T options, I have determined that
> the 29K writes were all very fast in themselves. However, most of the
> writes seem to follow each other with no other system calls in between.
> This explains why it looks like the writes are slow, even though they aren't.
>
> If I tally up the time between the previous system call, and each write(),
> it adds up to the bulk of the time (4mins out of 4m15s) that it takes to
> pack refs. This tells me that no visible I/O or system calls are the problem,
> but rather that the program itself is taking a long time between writes.
> I very much doubt that this is heavy CPU time, but rather I am going to
> guess that this is hidden system time spent accessing mmaped memory.
> Could it be really slow reading the packed-refs file? I can see the
> packed-refs file is mmaped() before the writes start, and then
> munmapped after the writes are completed. If I had to guess, that likely
> means that the packed-refs file is being read in small increments by the
> kernel via mmap, and that is what is making things very slow over NFS.
I wouldn't be surprised if NFS was the culprit. At GitLab we found it to
be a constant source of issues, which is why we eventually sunsetted the
use of it completely. Do you use any special flags for mounting the NFS
filesystem?
> My alternative theory, is that each ref is being looked up via a binary
> search, but I don't think git does this?
Did you try using perf(1) to profile the process and generate a flame
graph from it? That should likely make it immediately obvious where Git
is spending all of its time.
Patrick
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2026-01-05 23:45 ` Martin Fick
2026-01-06 6:53 ` Patrick Steinhardt
@ 2026-01-06 10:38 ` Jeff King
2026-01-06 23:03 ` Martin Fick
1 sibling, 1 reply; 19+ messages in thread
From: Jeff King @ 2026-01-06 10:38 UTC (permalink / raw)
To: Martin Fick; +Cc: brian m. carlson, git@vger•kernel.org
On Mon, Jan 05, 2026 at 11:45:41PM +0000, Martin Fick wrote:
> By repacking to get one used, and one cruft pack only, and no loose
> objects, I have confirmed that pack-refs it is still slow. This rules out the
> idea that the loose object, or pack file counts were making things slow.
OK, that is interesting. I'd still expect opening the objects to be the
dominating factor, but now the load would be on jumping around the
mmap'd packfile rather than open/read/close calls.
> OK, after discovering the strace -r and -T options, I have determined that
> the 29K writes were all very fast in themselves. However, most of the
> writes seem to follow each other with no other system calls in between.
> This explains why it looks like the writes are slow, even though they aren't.
>
> If I tally up the time between the previous system call, and each write(),
> it adds up to the bulk of the time (4mins out of 4m15s) that it takes to
> pack refs. This tells me that no visible I/O or system calls are the problem,
> but rather that the program itself is taking a long time between writes.
> I very much doubt that this is heavy CPU time, but rather I am going to
> guess that this is hidden system time spent accessing mmaped memory.
That would be consistent with reading object data from the packfile.
We'll jump around within the packfile to get that data.
> Could it be really slow reading the packed-refs file? I can see the
> packed-refs file is mmaped() before the writes start, and then
> munmapped after the writes are completed. If I had to guess, that likely
> means that the packed-refs file is being read in small increments by the
> kernel via mmap, and that is what is making things very slow over NFS.
The packed-refs file is mmap'd, but we'll be reading it sequentially. I
guess whether or not there is good read-ahead there may depend on the
NFS implementation.
> My alternative theory, is that each ref is being looked up via a binary
> search, but I don't think git does this?
Git does binary search within the packed-refs file, but it shouldn't be
doing so here. The write-out phase of packing refs is a straight merge
between two lists: the existing packed-refs entries and the new entries
we are adding.
I'd second Patrick's suggestion to use perf or similar to try to see
where the time is going.
You might also try building Git with NO_MMAP. That might make the I/O
costs more apparent via strace, because they'll be coming via pread().
-Peff
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2026-01-06 6:53 ` Patrick Steinhardt
@ 2026-01-06 23:02 ` Martin Fick
2026-01-07 11:42 ` Patrick Steinhardt
2026-01-07 17:05 ` Martin Fick
1 sibling, 1 reply; 19+ messages in thread
From: Martin Fick @ 2026-01-06 23:02 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: Jeff King, brian m. carlson, git@vger•kernel.org
> From: Patrick Steinhardt <ps@pks•im> Sent: Monday, January 5, 2026 11:53 PM
> On Mon, Jan 05, 2026 at 11:45:41PM +0000, Martin Fick wrote:
> > OK, after discovering the strace -r and -T options, I have determined that
> > the 29K writes were all very fast in themselves. However, most of the
> > writes seem to follow each other with no other system calls in between.
> > This explains why it looks like the writes are slow, even though they aren't.
> >
> > If I tally up the time between the previous system call, and each write(),
> > it adds up to the bulk of the time (4mins out of 4m15s) that it takes to
> > pack refs. This tells me that no visible I/O or system calls are the problem,
> > but rather that the program itself is taking a long time between writes.
> > I very much doubt that this is heavy CPU time, but rather I am going to
> > guess that this is hidden system time spent accessing mmaped memory.
> > Could it be really slow reading the packed-refs file? I can see the
> > packed-refs file is mmaped() before the writes start, and then
> > munmapped after the writes are completed. If I had to guess, that likely
> > means that the packed-refs file is being read in small increments by the
> > kernel via mmap, and that is what is making things very slow over NFS.
>
> I wouldn't be surprised if NFS was the culprit. At GitLab we found it to
> be a constant source of issues, which is why we eventually sunsetted the
> use of it completely. Do you use any special flags for mounting the NFS
> filesystem?
I am open to alternatives to NFS. Do you know of any NFS alternatives that
provides instantaneous replication to potentially hundreds of mirrors? I
have used Gerrit and git-daemon for many years on NFS, and it generally
has performed very well for us, and it solves many real performance issues
which I have yet to find a viable alternative able to even come close to
matching. NFS with all it warts it is for us (and likely will be for many) until
there is a viable enterprise ready alternative with low (zero) replication
latency and high throughput.
That being said, NFS can cause many issues. In this case, I would say that
something is particularly "broken" here with git, and I believe that it
would be helpful to the git community to be aware of this fairly specific
broken case which clearly has a lot of room for improvement (as seen
by the fact that jgit, in java, can do essentially the same thing more
than 10Xs faster). While I have been mostly assuming that this is a
particularly specific bad case since git daemon generally is fast for most
users, this might actually be something that if improved would greatly
improve many parts of git (not just this use case).
It would be nice to improve git to not hold the packed-refs.lock so long
to avoid this blocking behavior on servers. Of course, to be fair, this
likely only blocks Gerrit servers since Gerrit uses the packed-refs file to
perform atomic updates for many things, and most other servers use
loose refs instead. It would be great if git were optimized to avoid any
unnecessary reads while the lock is held. In theory, almost all of the
data that git needs to read here (including tags for peeling) could be
read before acquiring the lock, and it would only need to double
check certain reads after it acquires the lock in case things changed.
That wouldn't make git pack-refs faster, but it would drastically
reduce the impact of any problematic I/O by not holding the lock for
almost the entire operation.
> Did you try using perf(1) to profile the process and generate a flame
> graph from it? That should likely make it immediately obvious where Git
> is spending all of its time.
I will pursue this. Unfortunately this might be difficult on this
particular server.
Thanks for the feedback, and suggestions,
-Martin
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2026-01-06 10:38 ` Jeff King
@ 2026-01-06 23:03 ` Martin Fick
0 siblings, 0 replies; 19+ messages in thread
From: Martin Fick @ 2026-01-06 23:03 UTC (permalink / raw)
To: Jeff King; +Cc: brian m. carlson, git@vger•kernel.org
From: Jeff King <peff@peff•net> Sent: Tuesday, January 6, 2026 3:38 AM
> On Mon, Jan 05, 2026 at 11:45:41PM +0000, Martin Fick wrote:
> > By repacking to get one used, and one cruft pack only, and no loose
> > objects, I have confirmed that pack-refs it is still slow. This rules out the
> > idea that the loose object, or pack file counts were making things slow.
>
> OK, that is interesting. I'd still expect opening the objects to be the
> dominating factor, but now the load would be on jumping around the
> mmap'd packfile rather than open/read/close calls.
I believe I have confirmed this now with more testing...
By first dropping the system caches, and then catting the pack file to
/dev/null, it sped things up to under 20s!
Note that neither catting the idx, nor the packed-refs file helped to
noticeably speed things up.
> > OK, after discovering the strace -r and -T options, I have determined that
> > the 29K writes were all very fast in themselves. However, most of the
> > writes seem to follow each other with no other system calls in between.
> > This explains why it looks like the writes are slow, even though they aren't.
>
> > If I tally up the time between the previous system call, and each write(),
> > it adds up to the bulk of the time (4mins out of 4m15s) that it takes to
> > pack refs. This tells me that no visible I/O or system calls are the problem,
> > but rather that the program itself is taking a long time between writes.
> > I very much doubt that this is heavy CPU time, but rather I am going to
> > guess that this is hidden system time spent accessing mmaped memory.
>
> That would be consistent with reading object data from the packfile.
> We'll jump around within the packfile to get that data.
Agreed, but boy is that really bad performance!
> > Could it be really slow reading the packed-refs file? I can see the
> > packed-refs file is mmaped() before the writes start, and then
> > munmapped after the writes are completed. If I had to guess, that likely
> > means that the packed-refs file is being read in small increments by the
> > kernel via mmap, and that is what is making things very slow over NFS.
>
> The packed-refs file is mmap'd, but we'll be reading it sequentially. I
> guess whether or not there is good read-ahead there may depend on the
> NFS implementation.
Yeah, ruled out now by dropping the system caches, and then catting the
packed-refs file before running git pack-refs, which did NOT help speed
things up.
> > My alternative theory, is that each ref is being looked up via a binary
> > search, but I don't think git does this?
>
> Git does binary search within the packed-refs file, but it shouldn't be
> doing so here. The write-out phase of packing refs is a straight merge
> between two lists: the existing packed-refs entries and the new entries
> we are adding.
Agreed, and I should have ruled this out by realizing that this would likely
not have been affected by the system caches in my earlier tests.
> I'd second Patrick's suggestion to use perf or similar to try to see
> where the time is going.
Noted, thanks.
> You might also try building Git with NO_MMAP. That might make the I/O
> costs more apparent via strace, because they'll be coming via pread().
Agreed, I will try to do this. I think that the jgit results hint that this
this might even eliminate most of the I/O costs (jgit is not using
MMAP in my tests). It would be nice if this were a runtime config
instead of requiring a rebuild, as some use cases might be better
with, and some without MMAP.
Thanks for all the input,
-Martin
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2026-01-06 23:02 ` Martin Fick
@ 2026-01-07 11:42 ` Patrick Steinhardt
2026-01-07 22:58 ` Martin Fick
0 siblings, 1 reply; 19+ messages in thread
From: Patrick Steinhardt @ 2026-01-07 11:42 UTC (permalink / raw)
To: Martin Fick; +Cc: Jeff King, brian m. carlson, git@vger•kernel.org
On Tue, Jan 06, 2026 at 11:02:19PM +0000, Martin Fick wrote:
> > From: Patrick Steinhardt <ps@pks•im> Sent: Monday, January 5, 2026 11:53 PM
> > On Mon, Jan 05, 2026 at 11:45:41PM +0000, Martin Fick wrote:
> > > OK, after discovering the strace -r and -T options, I have determined that
> > > the 29K writes were all very fast in themselves. However, most of the
> > > writes seem to follow each other with no other system calls in between.
> > > This explains why it looks like the writes are slow, even though they aren't.
> > >
> > > If I tally up the time between the previous system call, and each write(),
> > > it adds up to the bulk of the time (4mins out of 4m15s) that it takes to
> > > pack refs. This tells me that no visible I/O or system calls are the problem,
> > > but rather that the program itself is taking a long time between writes.
> > > I very much doubt that this is heavy CPU time, but rather I am going to
> > > guess that this is hidden system time spent accessing mmaped memory.
> > > Could it be really slow reading the packed-refs file? I can see the
> > > packed-refs file is mmaped() before the writes start, and then
> > > munmapped after the writes are completed. If I had to guess, that likely
> > > means that the packed-refs file is being read in small increments by the
> > > kernel via mmap, and that is what is making things very slow over NFS.
> >
> > I wouldn't be surprised if NFS was the culprit. At GitLab we found it to
> > be a constant source of issues, which is why we eventually sunsetted the
> > use of it completely. Do you use any special flags for mounting the NFS
> > filesystem?
>
> I am open to alternatives to NFS. Do you know of any NFS alternatives that
> provides instantaneous replication to potentially hundreds of mirrors? I
> have used Gerrit and git-daemon for many years on NFS, and it generally
> has performed very well for us, and it solves many real performance issues
> which I have yet to find a viable alternative able to even come close to
> matching. NFS with all it warts it is for us (and likely will be for many) until
> there is a viable enterprise ready alternative with low (zero) replication
> latency and high throughput.
Yeah, agreed, NFS can get you a long way, until you eventually start to
hit some road blocks once you reach a certain scale. Unfortunately
though, there isn't really a ready-made alternative solution that serves
your needs, or at least none that I know of. That's why GitLab
eventually settled on Gitaly Cluster with Praefect handling replication,
and why GitHub has its Spokes architecture that does basically the same
thing.
> That being said, NFS can cause many issues. In this case, I would say that
> something is particularly "broken" here with git, and I believe that it
> would be helpful to the git community to be aware of this fairly specific
> broken case which clearly has a lot of room for improvement (as seen
> by the fact that jgit, in java, can do essentially the same thing more
> than 10Xs faster). While I have been mostly assuming that this is a
> particularly specific bad case since git daemon generally is fast for most
> users, this might actually be something that if improved would greatly
> improve many parts of git (not just this use case).
Chances are that if we can improve the case for NFS, other filesystems
might benefit, as well. So if this is something that we can improve I
agree that we should. It's too early to tell though, as we don't really
know what the actual root cause is just yet.
> It would be nice to improve git to not hold the packed-refs.lock so long
> to avoid this blocking behavior on servers. Of course, to be fair, this
> likely only blocks Gerrit servers since Gerrit uses the packed-refs file to
> perform atomic updates for many things, and most other servers use
> loose refs instead. It would be great if git were optimized to avoid any
> unnecessary reads while the lock is held. In theory, almost all of the
> data that git needs to read here (including tags for peeling) could be
> read before acquiring the lock, and it would only need to double
> check certain reads after it acquires the lock in case things changed.
> That wouldn't make git pack-refs faster, but it would drastically
> reduce the impact of any problematic I/O by not holding the lock for
> almost the entire operation.
It can probably be improved, true. I think that it's a bit of a wasted
effort, as I'd rather invest the time into improving reftables as a more
future-proof solution. But as you are well aware I'm quite biased here,
and I'd welcome any efforts to also improve the files backend. I am just
unlikely to work on it myself :)
> > Did you try using perf(1) to profile the process and generate a flame
> > graph from it? That should likely make it immediately obvious where Git
> > is spending all of its time.
>
> I will pursue this. Unfortunately this might be difficult on this
> particular server.
True, on the server side this can be a bit tricky.
Patrick
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2026-01-06 6:53 ` Patrick Steinhardt
2026-01-06 23:02 ` Martin Fick
@ 2026-01-07 17:05 ` Martin Fick
1 sibling, 0 replies; 19+ messages in thread
From: Martin Fick @ 2026-01-07 17:05 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: Jeff King, brian m. carlson, git@vger•kernel.org
> From: Patrick Steinhardt <ps@pks•im> Sent: Monday, January 5, 2026 11:53 PM
> On Mon, Jan 05, 2026 at 11:45:41PM +0000, Martin Fick wrote:
> > OK, after discovering the strace -r and -T options, I have determined that
> > the 29K writes were all very fast in themselves. However, most of the
> > writes seem to follow each other with no other system calls in between.
> > This explains why it looks like the writes are slow, even though they aren't.
>
> > If I tally up the time between the previous system call, and each write(),
> > it adds up to the bulk of the time (4mins out of 4m15s) that it takes to
> > pack refs. This tells me that no visible I/O or system calls are the problem,
> > but rather that the program itself is taking a long time between writes.
> > I very much doubt that this is heavy CPU time, but rather I am going to
> > guess that this is hidden system time spent accessing mmaped memory.
> > Could it be really slow reading the packed-refs file? I can see the
> > packed-refs file is mmaped() before the writes start, and then
> > munmapped after the writes are completed. If I had to guess, that likely
> > means that the packed-refs file is being read in small increments by the
> > kernel via mmap, and that is what is making things very slow over NFS.
>
> ... Do you use any special flags for mounting the NFS filesystem?
Oh sorry, I forgot to reply to this last time. We use the following mount flags:
rw,intr,retrans=10,timeo=600,hard,rsize=32768,wsize=32768,tcp,noacl,_netdev
-Martin
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2026-01-07 11:42 ` Patrick Steinhardt
@ 2026-01-07 22:58 ` Martin Fick
2026-01-08 6:33 ` Patrick Steinhardt
2026-01-15 21:09 ` Jeff King
0 siblings, 2 replies; 19+ messages in thread
From: Martin Fick @ 2026-01-07 22:58 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: Jeff King, brian m. carlson, git@vger•kernel.org
> From: Patrick Steinhardt <ps@pks•im> Sent: Wednesday, January 7, 2026 4:42 AM
On Tue, Jan 06, 2026 at 11:02:19PM +0000, Martin Fick wrote:
> > From: Patrick Steinhardt <ps@pks•im> Sent: Monday, January 5, 2026 11:53 PM
> > > Did you try using perf(1) to profile the process and generate a flame
> > > graph from it? That should likely make it immediately obvious where Git
> > > is spending all of its time.
> >
> > I will pursue this. Unfortunately this might be difficult on this
> > particular server.
>
> True, on the server side this can be a bit tricky.
I ran perf, and got a flame graph, I am not sure what the best way to share that
is, but I will try to summarize what looked important:
About one third of the time is in this section:
libc-2.17.so 32.5%
_memcmp_sse4_1 29.8%
page_fault 7.23%
...
I am not really sure what that is doing?
Another third is doing:
unpack_object_header_buffer 30%
page_fault 26.9%
...
nfs_read_page 10%
Which could very well be looking at the headers of objects to see if they are
tags needing to be peeled?
And the remaining third was a bit all over the place with small sections,
the largest two of those sections being:
packed_refs_store_create ~8.7%
unknown 4.4%
memchr 4.4%
page_fault 4.4%
nth_packed_object_offset 7%
page_fault 3.2%
This was way less informative (to me) then I would have hoped. :( Maybe
this means more to you?
It does look like a lot of page_faults, likely due to the use of mmap?
-Martin
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2026-01-07 22:58 ` Martin Fick
@ 2026-01-08 6:33 ` Patrick Steinhardt
2026-01-15 21:09 ` Jeff King
1 sibling, 0 replies; 19+ messages in thread
From: Patrick Steinhardt @ 2026-01-08 6:33 UTC (permalink / raw)
To: Martin Fick; +Cc: Jeff King, brian m. carlson, git@vger•kernel.org
On Wed, Jan 07, 2026 at 10:58:36PM +0000, Martin Fick wrote:
> > From: Patrick Steinhardt <ps@pks•im> Sent: Wednesday, January 7, 2026 4:42 AM
> On Tue, Jan 06, 2026 at 11:02:19PM +0000, Martin Fick wrote:
> > > From: Patrick Steinhardt <ps@pks•im> Sent: Monday, January 5, 2026 11:53 PM
> > > > Did you try using perf(1) to profile the process and generate a flame
> > > > graph from it? That should likely make it immediately obvious where Git
> > > > is spending all of its time.
> > >
> > > I will pursue this. Unfortunately this might be difficult on this
> > > particular server.
> >
> > True, on the server side this can be a bit tricky.
>
> I ran perf, and got a flame graph, I am not sure what the best way to share that
> is, but I will try to summarize what looked important:
>
> About one third of the time is in this section:
>
> libc-2.17.so 32.5%
> _memcmp_sse4_1 29.8%
> page_fault 7.23%
> ...
>
> I am not really sure what that is doing?
>
>
> Another third is doing:
>
> unpack_object_header_buffer 30%
> page_fault 26.9%
> ...
> nfs_read_page 10%
>
> Which could very well be looking at the headers of objects to see if they are
> tags needing to be peeled?
Both of these are lacking some information to be able to tell. Are you
by any chance able to share the whole flame graph? That'd make this a
bit easier to figure out.
> And the remaining third was a bit all over the place with small sections,
> the largest two of those sections being:
>
> packed_refs_store_create ~8.7%
> unknown 4.4%
> memchr 4.4%
> page_fault 4.4%
We spend ~9% of time in `packed_refs_store_create()`? That looks
seriously broken to me, the function shouldn't even do much.
> nth_packed_object_offset 7%
> page_fault 3.2%
>
> This was way less informative (to me) then I would have hoped. :( Maybe
> this means more to you?
>
> It does look like a lot of page_faults, likely due to the use of mmap?
Certainly looks like the page faults are to blame here overall. It's
still surprising to me it's _that_ slow. Quoting the other mail you sent:
On Wed, Jan 07, 2026 at 05:05:53PM +0000, Martin Fick wrote:
> rw,intr,retrans=10,timeo=600,hard,rsize=32768,wsize=32768,tcp,noacl,_netdev
I know that back when we still supported NFS we recommended to use an
rsize and wsize of 1MB to reduce the round trip times.
Patrick
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2026-01-07 22:58 ` Martin Fick
2026-01-08 6:33 ` Patrick Steinhardt
@ 2026-01-15 21:09 ` Jeff King
2026-01-16 20:35 ` Martin Fick
1 sibling, 1 reply; 19+ messages in thread
From: Jeff King @ 2026-01-15 21:09 UTC (permalink / raw)
To: Martin Fick; +Cc: Patrick Steinhardt, brian m. carlson, git@vger•kernel.org
On Wed, Jan 07, 2026 at 10:58:36PM +0000, Martin Fick wrote:
> I ran perf, and got a flame graph, I am not sure what the best way to share that
> is, but I will try to summarize what looked important:
>
> About one third of the time is in this section:
>
> libc-2.17.so 32.5%
> _memcmp_sse4_1 29.8%
> page_fault 7.23%
> ...
>
> I am not really sure what that is doing?
Probably this is the call to strcmp(iter->ref.name, update->refname) in
packed-backend.c:write_with_updates().
We have to write out the new packed-refs file with our updates in sorted
order. So it's a big O(n) merge between the existing ones (from the
"iter" side) and the new ones (from the "update" side).
It could also be caused by sorting of the packed-refs entries. We
generally shouldn't need to do that, but I think I may have found
something useful. See below.
> unpack_object_header_buffer 30%
> page_fault 26.9%
> ...
> nfs_read_page 10%
>
> Which could very well be looking at the headers of objects to see if they are
> tags needing to be peeled?
Yeah, that's what I'd expect here.
> And the remaining third was a bit all over the place with small sections,
> the largest two of those sections being:
>
> packed_refs_store_create ~8.7%
> unknown 4.4%
> memchr 4.4%
> page_fault 4.4%
Hmm, I don't think we have a function "packed_refs_store_create". Did
you typo while transferring the name over?
At any rate, we can assume this is poking through the packed-refs file
itself, looking for trailing newlines via memchr.
But why would we do that immediately when creating the packed-refs store
in memory? In modern versions of Git, we try to avoid reading the
packed-refs file as much as possible, binary-searching when we can. Of
course that means it has to be sorted, which was not something promised
by the original format. So we have a "sorted" tag that we write. E.g.,
this is from my clone of git, packed with git itself:
$ head -n 1 .git/packed-refs
# pack-refs with: peeled fully-peeled sorted
Now let's try something with jgit:
git init
git commit --allow-empty -m foo
git branch foo
git branch bar
jgit pack-refs --all
cat .git/packed-refs
That gives me this:
# pack-refs with: peeled
86054aaedc64c24aec8aaad988f6979a3cb82ee0 refs/heads/bar
86054aaedc64c24aec8aaad988f6979a3cb82ee0 refs/heads/foo
86054aaedc64c24aec8aaad988f6979a3cb82ee0 refs/heads/main
Aha! So jgit is not writing out the "sorted" tag. As a result, when git
reads the file, its logic is:
1. Check for the sorted tag. It's not here, so...
2. Check if the file is sorted by reading each entry linearly. If it's
not, then...
3. Read it all into memory and sort the result. We can then
binary-search that (and iterate it in sorted order, which is
important for pack-refs).
So when git reads the packed-refs file, we are ending up at least with
step 2, an extra pass through the whole file, and maybe to step 3
(depending on whether jgit actually sorts the file).
You mentioned that Gerrit writes the packed-refs file directly itself,
presumably using jgit. So it sounds like it is constantly undoing Git's
"sorted" marker, which causes git-pack-refs to spend extra effort
checking the sortedness, and rewrite the marker, which then gets hosed
again by jgit, and so on.
And that may explain why jgit is faster, if it is not doing the extra
sort check. If it is not even trying to maintain the sorted property
that it would be faster still (it takes one linear pass while writing
out the file, omitting entries that match our updates, and then appends
our updates at the end).
If jgit _is_ sorting the file but not writing out the sorted marker,
then it should start doing so. ;)
If it's not sorting the file, then probably it should start doing so
(and writing the marker). This will make subsequent reads much faster
(mmap + binary-search). It shouldn't even be slower to write (assuming
jgit's writes are doing the usual "rewrite the whole thing to a tempfile
and atomic-rename into place", and not taking some shortcut by appending
to the file).
Unrelated to your problem, but also jgit should support the fully-peeled
tag, another thing that makes readers faster. ;)
The jgit version I'm using is:
$ jgit version
jgit version 7.5.0.202512021534-r
One way you could test this theory is to sort and mark the file
yourself, before running "git pack-refs". One easy way to do that is to
convince git to rewrite it by removing an entry. I.e., find some ref
mentioned in the pack-refs file, and then "git update-ref -d $ref". And
check out the first line of .git/packed-refs before and after. If it
goes faster (and similarly fast to jgit) only when the "sorted" tag
appears, then that would be our culprit.
-Peff
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Slow git pack-refs --all
2026-01-15 21:09 ` Jeff King
@ 2026-01-16 20:35 ` Martin Fick
0 siblings, 0 replies; 19+ messages in thread
From: Martin Fick @ 2026-01-16 20:35 UTC (permalink / raw)
To: Jeff King; +Cc: Patrick Steinhardt, brian m. carlson, git@vger•kernel.org
> From: Jeff King <peff@peff•net> Sent: Thursday, January 15, 2026 2:09 PM
> > ...
> > And the remaining third was a bit all over the place with small sections,
> > the largest two of those sections being:
> >
> > packed_refs_store_create ~8.7%
> > unknown 4.4%
> > memchr 4.4%
> > page_fault 4.4%
>
> Hmm, I don't think we have a function "packed_refs_store_create". Did
> you typo while transferring the name over?
Yes, this should have been packed_ref_store_create (singular), sorry.
> At any rate, we can assume this is poking through the packed-refs file
> itself, looking for trailing newlines via memchr.
>
> But why would we do that immediately when creating the packed-refs store
> in memory? In modern versions of Git, we try to avoid reading the
> packed-refs file as much as possible, binary-searching when we can. Of
> course that means it has to be sorted, which was not something promised
> by the original format. So we have a "sorted" tag that we write. E.g.,
> this is from my clone of git, packed with git itself:
> ...
> # pack-refs with: peeled fully-peeled sorted
> ...
> jgit pack-refs --all
>
> That gives me this:
>
> # pack-refs with: peeled
> ...
> Aha! So jgit is not writing out the "sorted" tag. As a result, when git
> reads the file, its logic is:
>
> 1. Check for the sorted tag. It's not here, so...
> 2. Check if the file is sorted by reading each entry linearly. If it's
> not, then...
> 3. Read it all into memory and sort the result. We can then
> binary-search that (and iterate it in sorted order, which is
> important for pack-refs).
>
> So when git reads the packed-refs file, we are ending up at least with
> step 2, an extra pass through the whole file, and maybe to step 3
> (depending on whether jgit actually sorts the file).
>
> You mentioned that Gerrit writes the packed-refs file directly itself,
> presumably using jgit. So it sounds like it is constantly undoing Git's
> "sorted" marker, which causes git-pack-refs to spend extra effort
> checking the sortedness, and rewrite the marker, which then gets hosed
> again by jgit, and so on.
Agreed, this is likely the case, but not for the sorted marker, see below...
> And that may explain why jgit is faster, if it is not doing the extra
> sort check. If it is not even trying to maintain the sorted property
> that it would be faster still (it takes one linear pass while writing
> out the file, omitting entries that match our updates, and then appends
> our updates at the end).
Unfortunately, this does not actually seem to be the reason.
> If jgit _is_ sorting the file but not writing out the sorted marker,
> then it should start doing so. ;)
Agreed, I will see to it that this gets fixed. Unfortunately, adding the
sorted tag does not seem to speed things up. :(
> If it's not sorting the file, then probably it should start doing so
> (and writing the marker). This will make subsequent reads much faster
> (mmap + binary-search). It shouldn't even be slower to write (assuming
> jgit's writes are doing the usual "rewrite the whole thing to a tempfile
> and atomic-rename into place", and not taking some shortcut by appending
> to the file).
FYI, jgit does seem to order things, it does not append. The resulting output
from jgit after a repack with new refs add matches that from git for all but
the header.
> Unrelated to your problem, but also jgit should support the fully-peeled
> tag, another thing that makes readers faster. ;)
Ironically, this is not just related, it appears to be the trigger!!! When I add
this tag (and not the sorted tag), the cache flushed time drops down to
under 4s (from over 5mins)!
I will see to it that jgit fixes this too. That should help solve my problem.
That being said, it seems like something is still broken in git here
despite this tag being missing?
Thanks so much Peff for helping get to this point!
-Martin
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2026-01-16 20:35 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-12-25 22:13 Slow git pack-refs --all Martin Fick
2025-12-25 23:38 ` brian m. carlson
2025-12-26 4:45 ` Jeff King
2025-12-26 17:15 ` brian m. carlson
2025-12-27 7:36 ` Jeff King
2025-12-31 5:48 ` Martin Fick
2026-01-02 7:49 ` Jeff King
2026-01-05 23:45 ` Martin Fick
2026-01-06 6:53 ` Patrick Steinhardt
2026-01-06 23:02 ` Martin Fick
2026-01-07 11:42 ` Patrick Steinhardt
2026-01-07 22:58 ` Martin Fick
2026-01-08 6:33 ` Patrick Steinhardt
2026-01-15 21:09 ` Jeff King
2026-01-16 20:35 ` Martin Fick
2026-01-07 17:05 ` Martin Fick
2026-01-06 10:38 ` Jeff King
2026-01-06 23:03 ` Martin Fick
2025-12-31 5:39 ` Martin Fick
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox