From: "R. Tyler Ballance" <tyler@slide•com>
To: Nicolas Pitre <nico@cam•org>
Cc: "Jan Krüger" <jk@jk•gs>, "Git ML" <git@vger•kernel.org>
Subject: Re: [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file
Date: Tue, 06 Jan 2009 17:39:20 -0800 [thread overview]
Message-ID: <1231292360.8870.61.camel@starfruit> (raw)
In-Reply-To: <alpine.LFD.2.00.0901062005290.26118@xanadu.home>
[-- Attachment #1: Type: text/plain, Size: 3010 bytes --]
On Tue, 2009-01-06 at 20:25 -0500, Nicolas Pitre wrote:
> On Tue, 6 Jan 2009, R. Tyler Ballance wrote:
>
> > On Tue, 2008-12-09 at 09:36 +0100, Jan Krüger wrote:
> > > For fixing a corrupted repository by using backup copies of individual
> > > files, allow write_sha1_file() to write loose files even if the object
> > > already exists in a pack file, but only if the existing entry is marked
> > > as corrupted.
> >
> > I figured I'd reply to this again, since the issue cropped up again.
> >
> > We started experiencing *large* numbers of corruptions like the ones
> > that started the thread (one developer was receiving them once or twice
> > a day) with v1.6.0.4
> >
> > We went ahead and upgraded to a custom build of v1.6.1 with Jan's patch
> > (below) and the issues /seem/ to have resolved themselves. I'm not
> > certain whether Jan's patch was really responsible, or if there was
> > another issue that caused this to correct itself in v1.6.1.
I'll back the patch out and redeploy, it's worth mentioning that a
coworker of mine just got the issue as well (on 1.6.1). He was able to
`git pull` and the error went away, but I doubt that it "magically fixed
itself"
> Please back it out. As it stands, that patch is a no op because of the
> way git is used, and even if the patch was to work as intended, its
> purpose is not to magically fix corruptions without special action from
> your part. If you have corruption problems coming back only because of
> the removal of this patch then something is really really fishy and I
> would really like to know about it.
>
> There were indeed many changes between v1.6.0.4 and v1.6.1: the exact
> number is 1029. A couple of them are especially addressing increased
> robustness against some kind of pack corruptions. But in any case you
> still should see error messages appearing about them.
>
> And don't underestimate the power of disk corruptions. I started to
> work on git corruption resilience simply because I ended up with a
> corrupted pack at some point. Then a while later I got another
> corrupted pack. Then another while later I lost my filesystem entirely
> and had to reinstall my system (after buying a new disk). Turns out
> that my old disk is silently corrupting data without signaling any
> errors to the host.
I highly doubt this, I've got the issue appearing on at least 7
different development boxes (not workstations, 2U quad-core ECC RAM, etc
machines), while that doesn't mean that they all don't have issues, the
probability of them *all* having disk issues, and it somehow only
manifesting itself with Git usage, is low ;)
I've tarred one of the repositories that had it in a reproducible state
so I can create a build and extract the tar and run against that to
verify any patches anybody might have, but unfortunately at 7GB of
company code and assets, I can't exactly share ;)
Cheers
--
-R. Tyler Ballance
Slide, Inc.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]
next prev parent reply other threads:[~2009-01-07 1:40 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-12-09 8:36 [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file Jan Krüger
2008-12-09 9:02 ` R. Tyler Ballance
2008-12-09 16:24 ` Shawn O. Pearce
2009-01-06 22:52 ` R. Tyler Ballance
2009-01-07 1:25 ` Nicolas Pitre
2009-01-07 1:39 ` R. Tyler Ballance [this message]
2009-01-07 2:09 ` Nicolas Pitre
2009-01-07 2:47 ` R. Tyler Ballance
2009-01-07 3:21 ` Nicolas Pitre
2009-01-07 4:54 ` Linus Torvalds
2009-01-07 7:41 ` R. Tyler Ballance
2009-01-07 8:16 ` Junio C Hamano
2009-01-07 8:32 ` R. Tyler Ballance
2009-01-07 9:42 ` Junio C Hamano
2009-01-07 9:05 ` R. Tyler Ballance
2009-01-07 15:31 ` Nicolas Pitre
2009-01-07 16:07 ` Linus Torvalds
2009-01-07 16:08 ` Linus Torvalds
2009-01-07 22:55 ` R. Tyler Ballance
2009-01-07 23:29 ` Linus Torvalds
2009-01-08 0:28 ` Public repro case! " R. Tyler Ballance
2009-01-08 0:48 ` Linus Torvalds
2009-01-08 0:57 ` R. Tyler Ballance
2009-01-08 1:08 ` Linus Torvalds
2009-01-08 1:29 ` Linus Torvalds
2009-01-08 1:46 ` Shawn O. Pearce
2009-01-08 2:21 ` James Pickens
2009-01-08 2:43 ` Shawn O. Pearce
2009-01-08 5:40 ` Junio C Hamano
2009-01-08 6:04 ` Shawn O. Pearce
2009-01-08 2:52 ` Boyd Stephen Smith Jr.
2009-01-08 2:52 ` Linus Torvalds
2009-01-08 3:01 ` Shawn O. Pearce
2009-01-08 3:06 ` Linus Torvalds
2009-01-08 3:13 ` Shawn O. Pearce
2009-01-08 3:16 ` [PATCH] Wrap inflateInit to retry allocation after releasing pack memory Shawn O. Pearce
2009-01-08 3:54 ` Linus Torvalds
2009-01-08 5:23 ` Junio C Hamano
2009-01-08 15:35 ` Linus Torvalds
2009-01-08 15:34 ` Shawn O. Pearce
2009-01-08 16:14 ` Linus Torvalds
2009-01-08 18:15 ` R. Tyler Ballance
2009-01-08 20:22 ` Linus Torvalds
2009-01-08 20:37 ` R. Tyler Ballance
2009-01-09 1:43 ` Junio C Hamano
2009-01-08 0:37 ` [PATCH/RFC] Allow writing loose objects that are corrupted in a pack file Linus Torvalds
2009-01-08 0:49 ` R. Tyler Ballance
2009-01-08 1:01 ` Linus Torvalds
2009-01-08 1:06 ` R. Tyler Ballance
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1231292360.8870.61.camel@starfruit \
--to=tyler@slide$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=jk@jk$(echo .)gs \
--cc=nico@cam$(echo .)org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox