From: Junio C Hamano <gitster@pobox•com>
To: atousa.p@gmail•com
Cc: git@vger•kernel.org, Atousa Pahlevan Duprat <apahlevan@ieee•org>
Subject: Re: [PATCH v4 2/3] Limit the size of the data block passed to SHA1_Update()
Date: Thu, 05 Nov 2015 10:29:28 -0800 [thread overview]
Message-ID: <xmqqtwp05mgn.fsf@gitster.mtv.corp.google.com> (raw)
In-Reply-To: 1446705523-30701-2-git-send-email-apahlevan@ieee.org
atousa.p@gmail•com writes:
> +#ifndef git_SHA_CTX
> +
> +#ifdef SHA1_MAX_BLOCK_SIZE
> +#include "compat/sha1-chunked.h"
> +#define git_SHA_CTX platform_SHA_CTX
> +#define git_SHA1_Init platform_SHA1_Init
> +#define git_SHA1_Update git_SHA1_Update_Chunked
> +#define git_SHA1_Final platform_SHA1_Final
> +#else
> #define git_SHA_CTX platform_SHA_CTX
> #define git_SHA1_Init platform_SHA1_Init
> #define git_SHA1_Update platform_SHA1_Update
> #define git_SHA1_Final platform_SHA1_Final
> -
> #endif
>
> +#endif
> +
Adjusting to the proposed change to 1/3, this step would become the
attached patch. Note that I thought the above would not scale well
so I did it a bit differently.
-- >8 --
From: Atousa Pahlevan Duprat <apahlevan@ieee•org>
Subject: sha1: allow limiting the size of the data passed to SHA1_Update()
Using the previous commit's inredirection mechanism for SHA1,
support a chunked implementation of SHA1_Update() that limits the
amount of data in the chunk passed to SHA1_Update().
This is enabled by using the Makefile variable SHA1_MAX_BLOCK_SIZE
to specify chunk size. When using Apple's CommonCrypto library this
is set to 1GiB (the implementation cannot handle more 4GiB).
Signed-off-by: Atousa Pahlevan Duprat <apahlevan@ieee•org>
Signed-off-by: Junio C Hamano <gitster@pobox•com>
---
Makefile | 13 +++++++++++++
cache.h | 6 ++++++
compat/apple-common-crypto.h | 4 ++++
compat/sha1-chunked.c | 19 +++++++++++++++++++
compat/sha1-chunked.h | 2 ++
5 files changed, 44 insertions(+)
create mode 100644 compat/sha1-chunked.c
create mode 100644 compat/sha1-chunked.h
diff --git a/Makefile b/Makefile
index 04c2231..6a4ca59 100644
--- a/Makefile
+++ b/Makefile
@@ -141,6 +141,10 @@ all::
# Define PPC_SHA1 environment variable when running make to make use of
# a bundled SHA1 routine optimized for PowerPC.
#
+# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed
+# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO
+# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.
+#
# Define NEEDS_CRYPTO_WITH_SSL if you need -lcrypto when using -lssl (Darwin).
#
# Define NEEDS_SSL_WITH_CRYPTO if you need -lssl when using -lcrypto (Darwin).
@@ -1335,6 +1339,11 @@ ifdef NO_POSIX_GOODIES
BASIC_CFLAGS += -DNO_POSIX_GOODIES
endif
+ifdef APPLE_COMMON_CRYPTO
+ # Apple CommonCrypto requires chunking
+ SHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L
+endif
+
ifdef BLK_SHA1
SHA1_HEADER = "block-sha1/sha1.h"
LIB_OBJS += block-sha1/sha1.o
@@ -1353,6 +1362,10 @@ endif
endif
endif
+ifdef SHA1_MAX_BLOCK_SIZE
+ LIB_OBJS += compat/sha1-chunked.o
+ BASIC_CFLAGS += -DSHA1_MAX_BLOCK_SIZE="$(SHA1_MAX_BLOCK_SIZE)"
+endif
ifdef NO_PERL_MAKEMAKER
export NO_PERL_MAKEMAKER
endif
diff --git a/cache.h b/cache.h
index 2f697c4..19c966d 100644
--- a/cache.h
+++ b/cache.h
@@ -30,6 +30,12 @@
#define git_SHA1_Update platform_SHA1_Update
#define git_SHA1_Final platform_SHA1_Final
+#ifdef SHA1_MAX_BLOCK_SIZE
+#include "compat/sha1-chunked.h"
+#undef git_SHA1_Update
+#define git_SHA1_Update git_SHA1_Update_Chunked
+#endif
+
#include <zlib.h>
typedef struct git_zstream {
z_stream z;
diff --git a/compat/apple-common-crypto.h b/compat/apple-common-crypto.h
index c8b9b0e..d3fb264 100644
--- a/compat/apple-common-crypto.h
+++ b/compat/apple-common-crypto.h
@@ -16,6 +16,10 @@
#undef TYPE_BOOL
#endif
+#ifndef SHA1_MAX_BLOCK_SIZE
+#error Using Apple Common Crypto library requires setting SHA1_MAX_BLOCK_SIZE
+#endif
+
#ifdef APPLE_LION_OR_NEWER
#define git_CC_error_check(pattern, err) \
do { \
diff --git a/compat/sha1-chunked.c b/compat/sha1-chunked.c
new file mode 100644
index 0000000..6adfcfd
--- /dev/null
+++ b/compat/sha1-chunked.c
@@ -0,0 +1,19 @@
+#include "cache.h"
+
+int git_SHA1_Update_Chunked(platform_SHA_CTX *c, const void *data, size_t len)
+{
+ size_t nr;
+ size_t total = 0;
+ const char *cdata = (const char*)data;
+
+ while (len) {
+ nr = len;
+ if (nr > SHA1_MAX_BLOCK_SIZE)
+ nr = SHA1_MAX_BLOCK_SIZE;
+ platform_SHA1_Update(c, cdata, nr);
+ total += nr;
+ cdata += nr;
+ len -= nr;
+ }
+ return total;
+}
diff --git a/compat/sha1-chunked.h b/compat/sha1-chunked.h
new file mode 100644
index 0000000..7b2df28
--- /dev/null
+++ b/compat/sha1-chunked.h
@@ -0,0 +1,2 @@
+
+int git_SHA1_Update_Chunked(platform_SHA_CTX *c, const void *data, size_t len);
--
2.6.2-535-ga9e37b0
next prev parent reply other threads:[~2015-11-05 18:29 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-28 23:10 git fsck failure on OS X with files >= 4 GiB Rafael Espíndola
2015-10-29 6:46 ` Filipe Cabecinhas
[not found] ` <CAEDE8505fXAwVXx=EZwxPHvXpMByzpnXJ9LBgfx3U6VUaFbPHw@mail.gmail.com>
2015-10-29 10:46 ` Rafael Espíndola
2015-10-29 15:15 ` Filipe Cabecinhas
2015-10-29 16:02 ` Atousa Duprat
2015-10-29 17:19 ` Junio C Hamano
2015-10-30 2:15 ` Atousa Duprat
2015-10-30 22:12 ` [PATCH] Limit the size of the data block passed to SHA1_Update() Atousa Pahlevan Duprat
2015-10-30 22:22 ` Junio C Hamano
2015-11-01 6:41 ` Atousa Duprat
2015-11-01 18:31 ` Junio C Hamano
2015-11-01 1:32 ` Eric Sunshine
2015-11-01 6:32 ` atousa.p
2015-11-01 8:30 ` Eric Sunshine
2015-11-01 18:37 ` Junio C Hamano
2015-11-02 20:52 ` Atousa Duprat
2015-11-02 21:21 ` Junio C Hamano
2015-11-03 6:58 ` [PATCH 1/2] " atousa.p
2015-11-03 11:51 ` Torsten Bögershausen
2015-11-04 4:24 ` [PATCH] " atousa.p
2015-11-04 19:51 ` Eric Sunshine
2015-11-05 6:38 ` [PATCH v4 1/3] Provide another level of abstraction for the SHA1 utilities atousa.p
2015-11-05 18:29 ` Junio C Hamano
2015-11-05 6:38 ` [PATCH v4 2/3] Limit the size of the data block passed to SHA1_Update() atousa.p
2015-11-05 18:29 ` Junio C Hamano [this message]
2015-11-11 23:46 ` Atousa Duprat
2015-11-05 6:38 ` [PATCH v4 3/3] Move all the SHA1 implementations into one directory atousa.p
2015-11-05 18:29 ` Junio C Hamano
2015-11-04 4:27 ` [PATCH 1/2] Limit the size of the data block passed to SHA1_Update() Atousa Duprat
2015-11-04 17:09 ` [PATCH] " Junio C Hamano
2015-10-30 22:18 ` Atousa Pahlevan Duprat
2015-10-30 22:26 ` Randall S. Becker
2015-10-31 17:35 ` Junio C Hamano
2015-11-01 6:37 ` Atousa Duprat
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqtwp05mgn.fsf@gitster.mtv.corp.google.com \
--to=gitster@pobox$(echo .)com \
--cc=apahlevan@ieee$(echo .)org \
--cc=atousa.p@gmail$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox