From: "Nguyễn Thái Ngọc Duy" <pclouds@gmail•com>
To: git@vger•kernel.org
Cc: "Junio C Hamano" <gitster@pobox•com>,
michelbach94@gmail•com,
"Nguyễn Thái Ngọc Duy" <pclouds@gmail•com>
Subject: [PATCH v2 0/7] Number truncation with 4+ GB files on 32-bit systems
Date: Wed, 13 Jul 2016 17:43:57 +0200 [thread overview]
Message-ID: <20160713154404.22909-1-pclouds@gmail.com> (raw)
In-Reply-To: <20160705170558.10906-1-pclouds@gmail.com>
A diff from nd/pack-ofs-4gb-limit can explain the changes better than
me.
I did not add PRIdMAX or similar because that carries a risk to exotic
platforms that people rarely test. Just casting to unsigned should be
fine.
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 55eac75..b08bc8b 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -356,6 +356,10 @@ static int fsck_sha1(const unsigned char *sha1)
static int fsck_obj_buffer(const unsigned char *sha1, enum object_type type,
unsigned long size, void *buffer, int *eaten)
{
+ /*
+ * Note, buffer may be NULL if type is OBJ_BLOB. See
+ * verify_packfile(), data_valid variable for details.
+ */
struct object *obj;
obj = parse_object_buffer(sha1, type, size, buffer, eaten);
if (!obj) {
diff --git a/pack-check.c b/pack-check.c
index 14e8cb0..d123846 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -106,7 +106,7 @@ static int verify_packfile(struct packed_git *p,
enum object_type type;
unsigned long size;
off_t curpos;
- int data_valid = 0;
+ int data_valid;
if (p->index_version > 1) {
off_t offset = entries[i].offset;
@@ -130,6 +130,7 @@ static int verify_packfile(struct packed_git *p,
* the data in-core only to discard.
*/
data = NULL;
+ data_valid = 0;
} else {
data = unpack_entry(p, entries[i].offset, &type, &size);
data_valid = 1;
diff --git a/t/t1050-large.sh b/t/t1050-large.sh
index f9f3d13..096dbff 100755
--- a/t/t1050-large.sh
+++ b/t/t1050-large.sh
@@ -177,10 +177,9 @@ test_expect_success 'zip achiving, deflate' '
git archive --format=zip HEAD >/dev/null
'
-test_expect_success 'fsck' '
- test_must_fail git fsck 2>err &&
- n=$(grep "error: attempting to allocate .* over limit" err | wc -l) &&
- test "$n" -gt 1
+test_expect_success 'fsck large blobs' '
+ git fsck 2>err &&
+ test_must_be_empty err
'
test_done
Nguyễn Thái Ngọc Duy (7):
pack-objects: pass length to check_pack_crc() without truncation
sha1_file.c: use type off_t* for object_info->disk_sizep
index-pack: correct "len" type in unpack_data()
index-pack: report correct bad object offsets even if they are large
index-pack: correct "offset" type in unpack_entry_data()
pack-objects: do not truncate result in-pack object size on 32-bit systems
fsck: use streaming interface for large blobs in pack
builtin/cat-file.c | 4 ++--
builtin/fsck.c | 4 ++++
builtin/index-pack.c | 23 ++++++++++++-----------
builtin/pack-objects.c | 17 +++++++++--------
cache.h | 2 +-
pack-check.c | 23 +++++++++++++++++++++--
pack.h | 1 +
sha1_file.c | 2 +-
t/t1050-large.sh | 7 +++----
9 files changed, 54 insertions(+), 29 deletions(-)
--
2.9.1.564.gb2f7278
next prev parent reply other threads:[~2016-07-13 15:45 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-24 22:38 [bug] Reliably Reproducible Bad Packing of Objects Christoph Michelbach
2016-07-02 9:10 ` Duy Nguyen
2016-07-02 14:35 ` Duy Nguyen
2016-07-05 17:05 ` [PATCH 0/5] Number truncation with 4+ GB files on 32-bit systems Nguyễn Thái Ngọc Duy
2016-07-05 17:05 ` [PATCH 1/5] pack-objects: pass length to check_pack_crc() without truncation Nguyễn Thái Ngọc Duy
2016-07-12 17:16 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 2/5] sha1_file.c: use type off_t* for object_info->disk_sizep Nguyễn Thái Ngọc Duy
2016-07-12 17:20 ` Junio C Hamano
2016-07-12 19:26 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 3/5] index-pack: correct "len" type in unpack_data() Nguyễn Thái Ngọc Duy
2016-07-05 20:25 ` Johannes Sixt
2016-07-06 15:25 ` Duy Nguyen
2016-07-06 16:04 ` Junio C Hamano
2016-07-06 16:08 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 4/5] index-pack: report correct bad object offsets even if they are large Nguyễn Thái Ngọc Duy
2016-07-12 17:24 ` Junio C Hamano
2016-07-12 19:27 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 5/5] index-pack: correct "offset" type in unpack_entry_data() Nguyễn Thái Ngọc Duy
2016-07-05 18:11 ` [PATCH 0/5] Number truncation with 4+ GB files on 32-bit systems Christoph Michelbach
[not found] ` <1467756891.4798.1.camel@gmail.com>
[not found] ` <CACsJy8BDQbanGsf=3z3K-OuH0++EuqQFEB22udXJT+WZnFKSBg@mail.gmail.com>
2016-07-06 18:02 ` Christoph Michelbach
2016-07-06 18:54 ` Duy Nguyen
2016-07-10 10:41 ` Duy Nguyen
2016-07-10 10:42 ` [PATCH 6/5] pack-objects: do not truncate result in-pack object size " Nguyễn Thái Ngọc Duy
2016-07-10 10:45 ` [PATCH 7/5] fsck: use streaming interface for large blobs in pack Nguyễn Thái Ngọc Duy
2016-07-12 18:39 ` Junio C Hamano
2016-07-12 19:06 ` Junio C Hamano
2016-07-12 17:07 ` [PATCH 0/5] Number truncation with 4+ GB files on 32-bit systems Junio C Hamano
2016-07-12 18:48 ` Junio C Hamano
2016-07-12 20:38 ` Junio C Hamano
2016-07-13 6:01 ` Duy Nguyen
2016-07-13 15:43 ` Nguyễn Thái Ngọc Duy [this message]
2016-07-13 15:43 ` [PATCH v2 1/7] pack-objects: pass length to check_pack_crc() without truncation Nguyễn Thái Ngọc Duy
2016-07-13 15:43 ` [PATCH v2 2/7] sha1_file.c: use type off_t* for object_info->disk_sizep Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 3/7] index-pack: correct "len" type in unpack_data() Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 4/7] index-pack: report correct bad object offsets even if they are large Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 5/7] index-pack: correct "offset" type in unpack_entry_data() Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 6/7] pack-objects: do not truncate result in-pack object size on 32-bit systems Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 7/7] fsck: use streaming interface for large blobs in pack Nguyễn Thái Ngọc Duy
2016-07-13 16:16 ` [PATCH v2 0/7] Number truncation with 4+ GB files on 32-bit systems Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160713154404.22909-1-pclouds@gmail.com \
--to=pclouds@gmail$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=gitster@pobox$(echo .)com \
--cc=michelbach94@gmail$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox