From: Eric Paris <eparis@redhat•com>
To: Nicolas Pitre <nico@fluxnic•net>
Cc: Jeff King <peff@peff•net>, git@vger•kernel.org
Subject: Re: git-reflog 70 minutes at 100% cpu and counting
Date: Wed, 16 Dec 2009 17:37:21 -0500 [thread overview]
Message-ID: <1261003041.2788.142.camel@localhost> (raw)
In-Reply-To: <alpine.LFD.2.00.0912161526080.23173@xanadu.home>
On Wed, 2009-12-16 at 16:06 -0500, Nicolas Pitre wrote:
> On Wed, 16 Dec 2009, Eric Paris wrote:
>
> > On Tue, 2009-12-15 at 22:03 -0500, Nicolas Pitre wrote:
> > > On Mon, 14 Dec 2009, Eric Paris wrote:
> > >
> > > > The alternative repo is slowing pushing up to that same location. That
> > > > tar is 855838982, so just a tad bit smaller.
> > >
> > > It doesn't appear to be complete yet, and not progressing either.
> >
> > The alternative repo is now available (but the original is down)
> >
> > I tried to run git gc --aggressive last night while I slept and got this
> > as output, maybe it helps point to a solution/problem? The git reflog
> > portion ran for 5 hours and 36 minutes and appears to have finished.
>
> Yes. I was able to reproduce your issue. And because of the *horrible*
> repository packing, the reflog expiration process is taking ages when
> determining object reachability at a rate of one reflog entry every 2
> seconds or so. With 4214 entries for the fsnotify-syscall branch, and
> 1352 entries for the fsnotify branch, this already takes up asignificant
> portion of the actual run time. I'm sure if your repository was
> properly packed this would take less than a minute.
I'm guessing this is a result of stgit.? These branches really should
be just a branch from a tag (which exists in kernel-1) and about 30-50
patches linearly applied on top. I don't know how I get that many
objects. I'm guessing many/most of them are crap that should be able to
be cleaned/deleted entirely as the rebasing/pushing/poping/updating that
stgit does under the covers should have rendered them pointless. Not
really sure when/how that should/could have happened.
Should I be running git-gc every night?
> Now, repacking doesn't work because...
>
> > $ git gc --aggressive
> > error: Could not read d936ff8a7b0841b51ddf96afa24a30b016824cb2
> > error: Could not read 29b6c2fb1390b4fd350a5ecc78f1156fc5d91e9f
/me is pretty git dumb, but is there some way to figure out the parents
or children of these? I just trolled through all of my directories
doing git show and didn't get any hits. I guess I'll just clean up and
start over....
> Those objects are indeed missing from the repository. Without them your
> repository is "broken". Either you can find them somewhere else and
> copy them over, or salvage as much as you can by fetching the
> interesting branches into another freshly made repository. This is
> unfortunate because I would have liked to see by how much this
> repository would have shrunk after a successful repack.
>
> Of course, usage of alternates is recommended _only_ with repositories
> that are stable, i.e. don't ever add repositories to
> .git/objects/info/alternates if those repositories are rewinded/rebased
> and/or branches in them are deleted/replaced. That could be a reason
> why some objects are now missing from the repository using alternates.
So I'm not sure how I did things wrong. my kernel-1 has those bunch of
remotes. The linux-next remote, like I said, basically rebases to
linus' tree, then merges 150 random branches. It tags that tree every
day and I pull those tags. So I would never expect any objects from
those remote trees to ever disappear.
Now I created branches in kernel-1 and I certainly have done lots of
things like so
git checkout -b testing remotes/linux-next/master
[edit]
git commit -a
git checkout -b testing1 remotes/linux-next/master
git branch -D testing
My assumption though was that this wouldn't ever affect my other
repositories. My other repository branches always started by checking
out a branch with remotes/*/* as the base.
My understanding was that I would only run into problems if I used
something on a branch I created myself in the alternatives repo in other
repos (and I didn't remove remotes)
I guess it's not impossible to believe that at some point in time i
would have exported patches to and mbox from kernel-1 and applied them
to kernel-2 or vice versa. I guess this would end up with the same
objects, right? Then if I deleted the branch in kernel-1 I would have
problems in kernel-2?
I guess I'll rebuild my setup
new kernel-alt has just the remotes, and my kernel-1,2,3 all alt to it
I'll never have local branches in my kernel-alt
I'll run git-gc every night
I'll hope to never have problem again.
Sound good?
next prev parent reply other threads:[~2009-12-16 22:37 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-12-14 20:28 git-reflog 70 minutes at 100% cpu and counting Eric Paris
2009-12-14 20:41 ` Sverre Rabbelier
2009-12-14 21:11 ` Jeff King
2009-12-14 21:20 ` Eric Paris
2009-12-14 21:23 ` Jeff King
2009-12-14 21:56 ` Eric Paris
2009-12-14 22:03 ` Sverre Rabbelier
2009-12-15 0:29 ` Nicolas Pitre
2009-12-14 22:14 ` Jeff King
2009-12-15 0:26 ` Nicolas Pitre
2009-12-15 0:36 ` Junio C Hamano
2009-12-15 3:58 ` Nicolas Pitre
2009-12-15 2:11 ` Eric Paris
2009-12-15 3:44 ` Nicolas Pitre
2009-12-15 2:39 ` Jeff King
2009-12-15 3:50 ` Nicolas Pitre
2009-12-15 4:26 ` Eric Paris
2009-12-16 3:03 ` Nicolas Pitre
2009-12-16 3:31 ` Eric Paris
2009-12-16 13:41 ` Eric Paris
2009-12-16 21:06 ` Nicolas Pitre
2009-12-16 22:37 ` Eric Paris [this message]
2009-12-17 5:38 ` Nicolas Pitre
2009-12-17 16:29 ` Eric Paris
2009-12-18 3:33 ` Nicolas Pitre
2009-12-18 3:44 ` Steven Noonan
2009-12-18 3:52 ` Eric Paris
2009-12-18 3:57 ` Nicolas Pitre
2009-12-18 4:26 ` Steven Noonan
2009-12-18 3:55 ` Eric Paris
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1261003041.2788.142.camel@localhost \
--to=eparis@redhat$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=nico@fluxnic$(echo .)net \
--cc=peff@peff$(echo .)net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox