From: Anders Darander <anders.darander@gmail•com>
To: git@vger•kernel.org
Subject: Re: git stash takes excessively long when many untracked files present
Date: Tue, 13 Aug 2013 10:11:11 +0000 (UTC) [thread overview]
Message-ID: <loom.20130813T120243-481@post.gmane.org> (raw)
In-Reply-To: 20130810214453.GA5719@jtriplet-mobl1
Josh Triplett <josh <at> joshtriplett.org> writes:
> [CCing folks involved in the recent "stash-refuse-to-kill" merge.]
>
> I keep portions of my home directory in git. I tried to "git stash"
> some local changes, and it ran for several minutes with no progress. ps
> showed that it was running "git ls-files --killed", which was taking
> 100% CPU, and occasionally reading the disk very slowly.
I've recently got the same problem, though in this case it's my
openembedded directory that's giving me those problems. (Having an
untracked build-directory of quiet a few GB takes some time).
I worked around it by locally patching git-stash:
-------------------------------------------
diff --git a/git-stash.sh b/git-stash.sh
index 85c9e2c..e5a2043 100755
--- a/git-stash.sh
+++ b/git-stash.sh
@@ -263,7 +263,7 @@ save_stash () {
exit 0
fi
if test -z "$untracked$force" &&
- test -n "$(git ls-files --killed | head -n 1)"
+ test -n "$(git ls-files --killed --directory | head -n 1)"
then
say "$(gettext "The following untracked files would NOT be
saved
test -n "$GIT_QUIET" || git ls-files --killed | sed
's/^/\t/'
-------------------------------------------
It seems to work in my extremely limited testing. Though, I'm pretty sure
that there'll be quite a few error cases... (Especially, as I just made
a naive attempt at patching git-stash, so I could go on with a few other
things).
Do anyone have any better idea on how to approach this?
> strace shows that git ls-files --killed is doing a full recursive
> enumeration of my entire home directory. That's a Really Bad Idea:
>
> ~$ find | wc -l
> 3248997
> ~$ find -type d | wc -l
> 350680
>
> Not only that, but it also appears to be attempting to stat and open
> several files in every single directory; for instance:
>
> stat(".ccache/1/3/.git", 0x7fff254bc7a0) = -1 ENOENT (No such file or
directory)
> open(".ccache/1/3/.git/HEAD", O_RDONLY) = -1 ENOENT (No such file or
directory)
> stat(".ccache/1/3/.git", 0x7fff254bc770) = -1 ENOENT (No such file or
directory)
> open(".ccache/1/3/.git/packed-refs", O_RDONLY) = -1 ENOENT (No such file
or directory)
>
> (Yes, in that order.)
>
> I see a lot of room for optimization here. Most importantly, git
> ls-files --killed really doesn't need to look at any directory entry
> unless something in the index would conflict with it.
I guess that this would be a good optimization. Or, are ls-files --killed
used in other cases where the current behaviour would be requiered?
Cheers,
Anders
next prev parent reply other threads:[~2013-08-13 10:15 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-08-10 21:44 git stash takes excessively long when many untracked files present Josh Triplett
2013-08-13 10:11 ` Anders Darander [this message]
2013-08-13 17:07 ` Junio C Hamano
2013-08-13 17:36 ` Anders Darander
2013-08-13 17:52 ` Junio C Hamano
2013-08-13 21:47 ` Junio C Hamano
2013-08-15 17:52 ` Junio C Hamano
2013-08-15 18:07 ` Josh Triplett
2013-08-15 18:58 ` Junio C Hamano
2013-08-15 19:47 ` Junio C Hamano
2013-08-15 21:28 ` [PATCH 0/3] Optimizing "ls-files -k" Junio C Hamano
2013-08-15 21:28 ` [PATCH 1/3] dir.c: use the cache_* macro to access the current index Junio C Hamano
2013-08-15 21:28 ` [PATCH 2/3] ls-files -k: a directory only can be killed if the index has a non-directory Junio C Hamano
2013-08-15 21:28 ` [PATCH 3/3] t3010: update to demonstrate "ls-files -k" optimization pitfalls Junio C Hamano
2013-08-15 23:30 ` [PATCH 4/3] git stash: avoid data loss when "git stash save" kills a directory Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=loom.20130813T120243-481@post.gmane.org \
--to=anders.darander@gmail$(echo .)com \
--cc=git@vger$(echo .)kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox