public inbox for git@vger.kernel.org 
 help / color / mirror / Atom feed

From: Junio C Hamano <gitster@pobox•com>
To: Steffen Prohaska <prohaska@zib•de>
Cc: git@vger•kernel.org, peff@peff•net, pclouds@gmail•com,
	john@keeping•me.uk, schacon@gmail•com
Subject: Re: [PATCH v3 2/3] Introduce GIT_MMAP_LIMIT to allow testing expected mmap size
Date: Thu, 21 Aug 2014 15:26:27 -0700	[thread overview]
Message-ID: <xmqq1ts9qy24.fsf@gitster.dls.corp.google.com> (raw)
In-Reply-To: <1408637110-15669-3-git-send-email-prohaska@zib.de> (Steffen Prohaska's message of "Thu, 21 Aug 2014 18:05:09 +0200")

Steffen Prohaska <prohaska@zib•de> writes:

> Similar to testing expectations about malloc with GIT_ALLOC_LIMIT (see
> commit d41489), it can be useful to test expectations about mmap.
>
> This introduces a new environment variable GIT_MMAP_LIMIT to limit the
> largest allowed mmap length (in KB).  xmmap() is modified to check the
> limit.  Together with GIT_ALLOC_LIMIT tests can now easily confirm
> expectations about memory consumption.
>
> GIT_ALLOC_LIMIT will be used in the next commit to test that data will

I smell the need for s/ALLOC/MMAP/ here, but perhaps you did mean
ALLOC (I won't know until I check 3/3 ;-)

> be streamed to an external filter without mmaping the entire file.
>
> [commit d41489]: d41489a6424308dc9a0409bc2f6845aa08bd4f7d Add more large
>     blob test cases
>
> Signed-off-by: Steffen Prohaska <prohaska@zib•de>
> ---
>  sha1_file.c | 17 ++++++++++++++++-
>  1 file changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/sha1_file.c b/sha1_file.c
> index 00c07f2..88d64c0 100644
> --- a/sha1_file.c
> +++ b/sha1_file.c
> @@ -663,10 +663,25 @@ void release_pack_memory(size_t need)
>  		; /* nothing */
>  }
>  
> +static void mmap_limit_check(size_t length)
> +{
> +	static int limit = -1;

Perhaps you want ssize_t here?  I see mmap() as a tool to handle a
lot more data than a single malloc() typically would ;-) so previous
mistakes by other people would not be a good excuse.

> +	if (limit == -1) {
> +		const char *env = getenv("GIT_MMAP_LIMIT");
> +		limit = env ? atoi(env) * 1024 : 0;
> +	}
> +	if (limit && length > limit)
> +		die("attempting to mmap %"PRIuMAX" over limit %d",
> +		    (intmax_t)length, limit);
> +}
> +
>  void *xmmap(void *start, size_t length,
>  	int prot, int flags, int fd, off_t offset)
>  {
> -	void *ret = mmap(start, length, prot, flags, fd, offset);
> +	void *ret;
> +
> +	mmap_limit_check(length);
> +	ret = mmap(start, length, prot, flags, fd, offset);
>  	if (ret == MAP_FAILED) {
>  		if (!length)
>  			return NULL;

next prev parent reply	other threads:[~2014-08-21 22:26 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-08-21 16:05 [PATCH v3 0/3] Stream fd to clean filter, GIT_MMAP_LIMIT Steffen Prohaska
2014-08-21 16:05 ` [PATCH v3 1/3] convert: Refactor would_convert_to_git() to single arg 'path' Steffen Prohaska
2014-08-21 16:05 ` [PATCH v3 2/3] Introduce GIT_MMAP_LIMIT to allow testing expected mmap size Steffen Prohaska
2014-08-21 22:26   ` Junio C Hamano [this message]
2014-08-22 13:46     ` Steffen Prohaska
2014-08-22 15:57       ` Junio C Hamano
2014-08-22 16:31         ` Junio C Hamano
2014-08-24 16:03           ` Steffen Prohaska
2014-08-21 16:05 ` [PATCH v3 3/3] convert: Stream from fd to required clean filter instead of mmap Steffen Prohaska

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqq1ts9qy24.fsf@gitster.dls.corp.google.com \
    --to=gitster@pobox$(echo .)com \
    --cc=git@vger$(echo .)kernel.org \
    --cc=john@keeping$(echo .)me.uk \
    --cc=pclouds@gmail$(echo .)com \
    --cc=peff@peff$(echo .)net \
    --cc=prohaska@zib$(echo .)de \
    --cc=schacon@gmail$(echo .)com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox