From: Junio C Hamano <gitster@pobox•com>
To: Doug Kelly <dougk.ff7@gmail•com>
Cc: Git List <git@vger•kernel.org>
Subject: Re: Question: .idx without .pack causes performance issues?
Date: Tue, 21 Jul 2015 12:15:01 -0700 [thread overview]
Message-ID: <xmqqzj2puxu2.fsf@gitster.dls.corp.google.com> (raw)
In-Reply-To: <xmqq4mkxwd77.fsf@gitster.dls.corp.google.com> (Junio C. Hamano's message of "Tue, 21 Jul 2015 11:57:48 -0700")
Junio C Hamano <gitster@pobox•com> writes:
> I however do not think that we mark the in-core structure that
> corresponds to an open ".idx" file in any way when such a failure
> happens. If we really cared enough, we could do so, saying "we know
> there is .idx file, but do not bother looking at it again, as we
> know the corresponding .pack is missing", and that would speed things
> up a bit, essentially bringing us back to a sane situation without
> any ".idx" without corresponding ".pack".
>
> I do not think it is worth the effort, though. It would be more
> fruitful to find out how you end up with ".idx exists but not
> corresponding .pack" and if that is some systemic failure, see if
> there is a way to prevent that from happening in the first place.
While I still think that it is more important to prevent such a
situation from occurring in the first place, ignoring .idx that lack
corresponding .pack should be fairly simple, perhaps like this.
Note that if we wanted to do this for real, I think such an ".idx"
file should also be added to the "garbage" list in the loop in which
the second hunk of the following patch appears.
sha1_file.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/sha1_file.c b/sha1_file.c
index 1cee438..b69298e 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1240,6 +1240,19 @@ static void report_pack_garbage(struct string_list *list)
report_helper(list, seen_bits, first, list->nr);
}
+static int packfile_exists(const char *base, size_t base_len)
+{
+ struct strbuf path = STRBUF_INIT;
+ int status;
+
+ strbuf_add(&path, base, base_len);
+ strbuf_addstr(&path, ".pack");
+ status = file_exists(path.buf);
+
+ strbuf_release(&path);
+ return status;
+}
+
static void prepare_packed_git_one(char *objdir, int local)
{
struct strbuf path = STRBUF_INIT;
@@ -1281,6 +1294,7 @@ static void prepare_packed_git_one(char *objdir, int local)
break;
}
if (p == NULL &&
+ packfile_exists(path.buf, base_len) &&
/*
* See if it really is a valid .idx file with
* corresponding .pack file that we can map.
next prev parent reply other threads:[~2015-07-21 19:15 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-07-21 18:41 Question: .idx without .pack causes performance issues? Doug Kelly
2015-07-21 18:57 ` Junio C Hamano
2015-07-21 19:15 ` Junio C Hamano [this message]
2015-07-21 20:48 ` Junio C Hamano
2015-07-21 21:37 ` Doug Kelly
2015-08-03 22:17 ` Doug Kelly
2015-08-04 1:27 ` Junio C Hamano
2015-08-07 21:36 ` Doug Kelly
2015-08-07 22:27 ` Junio C Hamano
2015-08-13 18:02 ` [PATCH 1/2] prepare_packed_git(): refactor garbage reporting in pack directory Doug Kelly
2015-08-13 18:02 ` [PATCH 2/2] gc: Remove garbage .idx files from pack dir Doug Kelly
2015-08-17 16:35 ` Junio C Hamano
2015-08-17 20:30 ` Junio C Hamano
2015-08-13 18:46 ` [PATCH 1/2] prepare_packed_git(): refactor garbage reporting in pack directory Eric Sunshine
2015-08-17 16:53 ` Junio C Hamano
2015-10-28 17:48 ` Junio C Hamano
2015-10-28 22:43 ` Doug Kelly
2015-11-04 3:05 ` [PATCH 1/3] " Doug Kelly
2015-11-04 3:05 ` [PATCH 2/3] t5304: Add test for cleaning pack garbage Doug Kelly
2015-11-04 3:05 ` [PATCH 3/3] gc: Remove garbage .idx files from pack dir Doug Kelly
2015-11-04 3:12 ` [PATCH 1/2] prepare_packed_git(): refactor garbage reporting in pack directory Doug Kelly
2015-11-04 19:35 ` Junio C Hamano
2015-11-04 19:56 ` Doug Kelly
2015-11-04 20:02 ` Jeff King
2015-11-04 20:08 ` Doug Kelly
2015-11-04 20:15 ` Jeff King
2015-12-30 7:37 ` Jeff King
2016-01-13 17:14 ` Doug Kelly
2016-01-13 20:08 ` Junio C Hamano
2016-01-13 20:19 ` Doug Kelly
2016-01-13 20:23 ` Jeff King
2015-11-04 19:56 ` Jeff King
[not found] ` <CABYiQpn7r2Vcf=S5RaWHBN85eBYGPV_e02+BY=4L98qfUzDT1Q@mail.gmail.com>
2015-11-11 14:58 ` Fwd: Question: .idx without .pack causes performance issues? Thomas Berg
2015-07-21 19:49 ` Doug Kelly
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqzj2puxu2.fsf@gitster.dls.corp.google.com \
--to=gitster@pobox$(echo .)com \
--cc=dougk.ff7@gmail$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox