From: Patrick Steinhardt <ps@pks•im>
To: Martin Fick <mfick@nvidia•com>
Cc: Jeff King <peff@peff•net>,
"brian m. carlson" <sandals@crustytoothpaste•net>,
"git@vger•kernel.org" <git@vger•kernel.org>
Subject: Re: Slow git pack-refs --all
Date: Thu, 8 Jan 2026 07:33:38 +0100 [thread overview]
Message-ID: <aV9PwnouO8652XQR@pks.im> (raw)
In-Reply-To: <CH3PR12MB9026C8C940270F02CEF83C4FC284A@CH3PR12MB9026.namprd12.prod.outlook.com>
On Wed, Jan 07, 2026 at 10:58:36PM +0000, Martin Fick wrote:
> > From: Patrick Steinhardt <ps@pks•im> Sent: Wednesday, January 7, 2026 4:42 AM
> On Tue, Jan 06, 2026 at 11:02:19PM +0000, Martin Fick wrote:
> > > From: Patrick Steinhardt <ps@pks•im> Sent: Monday, January 5, 2026 11:53 PM
> > > > Did you try using perf(1) to profile the process and generate a flame
> > > > graph from it? That should likely make it immediately obvious where Git
> > > > is spending all of its time.
> > >
> > > I will pursue this. Unfortunately this might be difficult on this
> > > particular server.
> >
> > True, on the server side this can be a bit tricky.
>
> I ran perf, and got a flame graph, I am not sure what the best way to share that
> is, but I will try to summarize what looked important:
>
> About one third of the time is in this section:
>
> libc-2.17.so 32.5%
> _memcmp_sse4_1 29.8%
> page_fault 7.23%
> ...
>
> I am not really sure what that is doing?
>
>
> Another third is doing:
>
> unpack_object_header_buffer 30%
> page_fault 26.9%
> ...
> nfs_read_page 10%
>
> Which could very well be looking at the headers of objects to see if they are
> tags needing to be peeled?
Both of these are lacking some information to be able to tell. Are you
by any chance able to share the whole flame graph? That'd make this a
bit easier to figure out.
> And the remaining third was a bit all over the place with small sections,
> the largest two of those sections being:
>
> packed_refs_store_create ~8.7%
> unknown 4.4%
> memchr 4.4%
> page_fault 4.4%
We spend ~9% of time in `packed_refs_store_create()`? That looks
seriously broken to me, the function shouldn't even do much.
> nth_packed_object_offset 7%
> page_fault 3.2%
>
> This was way less informative (to me) then I would have hoped. :( Maybe
> this means more to you?
>
> It does look like a lot of page_faults, likely due to the use of mmap?
Certainly looks like the page faults are to blame here overall. It's
still surprising to me it's _that_ slow. Quoting the other mail you sent:
On Wed, Jan 07, 2026 at 05:05:53PM +0000, Martin Fick wrote:
> rw,intr,retrans=10,timeo=600,hard,rsize=32768,wsize=32768,tcp,noacl,_netdev
I know that back when we still supported NFS we recommended to use an
rsize and wsize of 1MB to reduce the round trip times.
Patrick
next prev parent reply other threads:[~2026-01-08 6:33 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-12-25 22:13 Slow git pack-refs --all Martin Fick
2025-12-25 23:38 ` brian m. carlson
2025-12-26 4:45 ` Jeff King
2025-12-26 17:15 ` brian m. carlson
2025-12-27 7:36 ` Jeff King
2025-12-31 5:48 ` Martin Fick
2026-01-02 7:49 ` Jeff King
2026-01-05 23:45 ` Martin Fick
2026-01-06 6:53 ` Patrick Steinhardt
2026-01-06 23:02 ` Martin Fick
2026-01-07 11:42 ` Patrick Steinhardt
2026-01-07 22:58 ` Martin Fick
2026-01-08 6:33 ` Patrick Steinhardt [this message]
2026-01-15 21:09 ` Jeff King
2026-01-16 20:35 ` Martin Fick
2026-01-07 17:05 ` Martin Fick
2026-01-06 10:38 ` Jeff King
2026-01-06 23:03 ` Martin Fick
2025-12-31 5:39 ` Martin Fick
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aV9PwnouO8652XQR@pks.im \
--to=ps@pks$(echo .)im \
--cc=git@vger$(echo .)kernel.org \
--cc=mfick@nvidia$(echo .)com \
--cc=peff@peff$(echo .)net \
--cc=sandals@crustytoothpaste$(echo .)net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox