From: Junio C Hamano <gitster@pobox•com>
To: Eric Sunshine <sunshine@sunshineco•com>
Cc: Joachim Schmitz <jojo@schmitz-digital•de>,
Git List <git@vger•kernel.org>
Subject: Re: read() MAX_IO_SIZE bytes, more than SSIZE_MAX?
Date: Wed, 11 Feb 2015 13:13:10 -0800 [thread overview]
Message-ID: <xmqqmw4ktamx.fsf@gitster.dls.corp.google.com> (raw)
In-Reply-To: <CAPig+cTbLOV-0yFKp8wwVLSr4OJz7LUaLZgVGHUdFhj7xZEzrw@mail.gmail.com> (Eric Sunshine's message of "Sun, 8 Feb 2015 12:09:20 -0500")
Eric Sunshine <sunshine@sunshineco•com> writes:
> A bit cleaner:
>
> #ifndef(MAX_IO_SIZE)
> # define MAX_IO_SIZE_DEFAULT (8*1024*1024)
> # if defined(SSIZE_MAX) && (SSIZE_MAX < MAX_IO_SIZE_DEFAULT)
> # define MAX_IO_SIZE SSIZE_MAX
> # else
> # define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT
> # endif
> #endif
OK, then let's do this.
-- >8 --
Subject: xread/xwrite: clip MAX_IO_SIZE to SSIZE_MAX
Since 0b6806b9 (xread, xwrite: limit size of IO to 8MB, 2013-08-20),
we chomp our calls to read(2) and write(2) into chunks of
MAX_IO_SIZE bytes (8 MiB), because a large IO results in a bad
latency when the program needs to be killed. This also brought our
IO below SSIZE_MAX, which is a limit POSIX allows read(2) and
write(2) to fail when the IO size exceeds it, for OS X, where a
problem was originally reported.
However, there are other systems that define SSIZE_MAX smaller than
our default X-<. Make sure we clip our calls to this as well.
Reported-by: Joachim Schmitz <jojo@schmitz-digital•de>
Helped-by: Torsten Bögershausen <tboegi@web•de>
Helped-by: Eric Sunshine <sunshine@sunshineco•com>
Signed-off-by: Junio C Hamano <gitster@pobox•com>
---
wrapper.c | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/wrapper.c b/wrapper.c
index 007ec0d..50e6697 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -172,8 +172,21 @@ void *xcalloc(size_t nmemb, size_t size)
* 64-bit is buggy, returning EINVAL if len >= INT_MAX; and even in
* the absence of bugs, large chunks can result in bad latencies when
* you decide to kill the process.
+ *
+ * We pick 8 MiB as our default, but if the platform defines SSIZE_MAX
+ * that is smaller than that, clip it to SSIZE_MAX, as a call to
+ * read(2) or write(2) larger than taht is allowed to fail. As the last
+ * resort, we allow a port to pass via CFLAGS e.g. "-DMAX_IO_SIZE=value"
+ * to override this, if the definition of SSIZE_MAX platform is broken.
*/
-#define MAX_IO_SIZE (8*1024*1024)
+#ifndef(MAX_IO_SIZE)
+# define MAX_IO_SIZE_DEFAULT (8*1024*1024)
+# if defined(SSIZE_MAX) && (SSIZE_MAX < MAX_IO_SIZE_DEFAULT)
+# define MAX_IO_SIZE SSIZE_MAX
+# else
+# define MAX_IO_SIZE MAX_IO_SIZE_DEFAULT
+# endif
+#endif
/*
* xread() is the same a read(), but it automatically restarts read()
next prev parent reply other threads:[~2015-02-11 21:13 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-02-07 16:45 read() MAX_IO_SIZE bytes, more than SSIZE_MAX? Joachim Schmitz
2015-02-07 16:48 ` Joachim Schmitz
2015-02-07 17:19 ` Torsten Bögershausen
2015-02-07 17:29 ` Joachim Schmitz
2015-02-07 18:03 ` Joachim Schmitz
2015-02-07 20:32 ` Torsten Bögershausen
2015-02-07 22:13 ` Junio C Hamano
2015-02-07 22:31 ` Joachim Schmitz
2015-02-08 2:13 ` Junio C Hamano
2015-02-08 2:32 ` Randall S. Becker
2015-02-08 12:05 ` Joachim Schmitz
2015-02-08 17:09 ` Eric Sunshine
2015-02-11 21:13 ` Junio C Hamano [this message]
2015-02-11 21:29 ` Joachim Schmitz
2015-02-11 22:05 ` Joachim Schmitz
2015-02-11 23:15 ` Junio C Hamano
2015-02-07 18:06 ` Randall S. Becker
2015-02-07 18:20 ` Randall S. Becker
2015-02-07 18:36 ` Joachim Schmitz
2015-02-07 19:14 ` Joachim Schmitz
-- strict thread matches above, loose matches on Subject: below --
2015-02-12 7:46 Joachim Schmitz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqmw4ktamx.fsf@gitster.dls.corp.google.com \
--to=gitster@pobox$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
--cc=jojo@schmitz-digital$(echo .)de \
--cc=sunshine@sunshineco$(echo .)com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox