* Why is git submodule slow under windows @ 2014-07-18 10:14 Armbruster Joachim (BEG/EMS1) 2014-07-18 12:08 ` Thomas Braun 0 siblings, 1 reply; 4+ messages in thread From: Armbruster Joachim (BEG/EMS1) @ 2014-07-18 10:14 UTC (permalink / raw) To: git@vger•kernel.org Hello, We split a monolithic repository into ~50 submodules. The stored data has the same size. In the 1:1 comparison to the monolithic repository, the submodule handling is very slow. Under Linux everything remains fast, but windows is slow. So, why is git getting slow when it has to deal with a lot of submodules? I read something about the lack of the underlying cygwin to handle NTFS in a efficient way. Is this the root cause, or are there other causes also? ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Why is git submodule slow under windows 2014-07-18 10:14 Why is git submodule slow under windows Armbruster Joachim (BEG/EMS1) @ 2014-07-18 12:08 ` Thomas Braun 2014-07-18 13:02 ` Fredrik Gustafsson 0 siblings, 1 reply; 4+ messages in thread From: Thomas Braun @ 2014-07-18 12:08 UTC (permalink / raw) To: Armbruster Joachim (BEG/EMS1), git@vger•kernel.org Am 18.07.2014 12:14, schrieb Armbruster Joachim (BEG/EMS1): > Hello, > > We split a monolithic repository into ~50 submodules. The stored data > has the same size. In the 1:1 comparison to the monolithic > repository, the submodule handling is very slow. Under Linux > everything remains fast, but windows is slow. > > So, why is git getting slow when it has to deal with a lot of > submodules? I read something about the lack of the underlying cygwin > to handle NTFS in a efficient way. Is this the root cause, or are > there other causes also? > Hi, I assume you are using the latetst git from https://msysgit.github.io on windows. I would guess that submodules on windows are slow because git-submodules.sh is a shell script, and bash on windows is not really that fast. There has been some (albeit older) discussion on the msysgit mailinglist, see [1]. You can play around with core.fscache [2] maybe that helps. Thomas [1]: https://groups.google.com/d/msg/msysgit/AuPA4EbwchU/N42tsb6GousJ [2]: https://github.com/msysgit/msysgit/releases/tag/Git-1.8.5.2-preview20131230 ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Why is git submodule slow under windows 2014-07-18 12:08 ` Thomas Braun @ 2014-07-18 13:02 ` Fredrik Gustafsson 2014-07-18 18:01 ` Jens Lehmann 0 siblings, 1 reply; 4+ messages in thread From: Fredrik Gustafsson @ 2014-07-18 13:02 UTC (permalink / raw) To: Thomas Braun; +Cc: Armbruster Joachim (BEG/EMS1), git@vger•kernel.org On Fri, Jul 18, 2014 at 02:08:36PM +0200, Thomas Braun wrote: > Am 18.07.2014 12:14, schrieb Armbruster Joachim (BEG/EMS1): > > Hello, > > > > We split a monolithic repository into ~50 submodules. The stored data > > has the same size. In the 1:1 comparison to the monolithic > > repository, the submodule handling is very slow. Under Linux > > everything remains fast, but windows is slow. > > > > So, why is git getting slow when it has to deal with a lot of > > submodules? I read something about the lack of the underlying cygwin > > to handle NTFS in a efficient way. Is this the root cause, or are > > there other causes also? > > > > Hi, > > I assume you are using the latetst git from https://msysgit.github.io on > windows. > > I would guess that submodules on windows are slow because > git-submodules.sh is a shell script, and bash on windows is not really > that fast. My guess is that because the shell script uses fork() heavily and fork() is an expensive operation on Windows, that alone causes the slowddown. I did a quick test a while back when I rewrote part of git-submodule.sh in lua and runned it on my repo with ~45 submodules. The speedup was significant and should be even bigger on windows. -- Med vänlig hälsning Fredrik Gustafsson tel: 0733-608274 e-post: iveqy@iveqy•com ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Why is git submodule slow under windows 2014-07-18 13:02 ` Fredrik Gustafsson @ 2014-07-18 18:01 ` Jens Lehmann 0 siblings, 0 replies; 4+ messages in thread From: Jens Lehmann @ 2014-07-18 18:01 UTC (permalink / raw) To: Fredrik Gustafsson, Thomas Braun Cc: Armbruster Joachim (BEG/EMS1), git@vger•kernel.org, Heiko Voigt Am 18.07.2014 15:02, schrieb Fredrik Gustafsson: > On Fri, Jul 18, 2014 at 02:08:36PM +0200, Thomas Braun wrote: >> Am 18.07.2014 12:14, schrieb Armbruster Joachim (BEG/EMS1): >>> Hello, >>> >>> We split a monolithic repository into ~50 submodules. The stored data >>> has the same size. In the 1:1 comparison to the monolithic >>> repository, the submodule handling is very slow. Under Linux >>> everything remains fast, but windows is slow. >>> >>> So, why is git getting slow when it has to deal with a lot of >>> submodules? I read something about the lack of the underlying cygwin >>> to handle NTFS in a efficient way. Is this the root cause, or are >>> there other causes also? >>> >> >> Hi, >> >> I assume you are using the latetst git from https://msysgit.github.io on >> windows. >> >> I would guess that submodules on windows are slow because >> git-submodules.sh is a shell script, and bash on windows is not really >> that fast. > > My guess is that because the shell script uses fork() heavily and fork() > is an expensive operation on Windows, that alone causes the slowddown. > > I did a quick test a while back when I rewrote part of git-submodule.sh > in lua and runned it on my repo with ~45 submodules. The speedup was > significant and should be even bigger on windows. Without having looked at your Lua rewrite I suspect my recursive submodule checkout series could speed that up even more, as it only needs to fork a "git status" followed by a "git checkout" or "git read-tree" for each submodule. Now that my submodule test harness and Heiko's submodule config lookup API did hit next, this is the thing I'm currently working on. ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-07-18 18:02 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-07-18 10:14 Why is git submodule slow under windows Armbruster Joachim (BEG/EMS1) 2014-07-18 12:08 ` Thomas Braun 2014-07-18 13:02 ` Fredrik Gustafsson 2014-07-18 18:01 ` Jens Lehmann
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox