Subject: Re: Git Submodule Problem - Bug?

From: Fredrik Gustafsson <[email-masked]>
Date: Tue, 29 Nov 2011 11:15:46 +0100

On Tue, Nov 29, 2011 at 10:24:01AM +0100, Thomas Rast wrote:
> Manuel Koller wrote:
> > 
> > The problem arises when I pull a commit that switches a submodule with another one and then run git submodule update. Say I have a repo "super" that has one submodule "sub1" in the folder "sub" and a clone "super2". Now I remove the submodule in "super2" and add another one ("sub2") again named "sub", commit this and push it. Now after pulling the commit to "super", I need to run git submodule sync and then git submodule update. 
> > 
> > I expect to end up with the submodule "sub2" in sub. But the log clearly shows that the commits from "sub1" are still there (the master branch belongs to "sub1" while origin/master comes from "sub2").  I get the following output:
> > 
> > > ...
> > > commit 77d8d11fed3b07e1d4e47b3df9fc44c278694a39
> > > Author: Manuel Koller <[email-masked]>
> > > Date:   Mon Nov 28 17:46:45 2011 +0100
> > > 
> > >     initial commit sub1
> > > commit 346fe6bd9e7957f10c5e833bb1155153379da41c
> > > Author: Manuel Koller <[email-masked]>
> > > Date:   Mon Nov 28 17:46:45 2011 +0100
> > > 
> > >     initial commit sub2
> > 
> > I think it should be twice the same, and "sub2". I checked also with Charon, on his machine, the two log messages reported are "sub1" which completely baffles me.
> 
> I used the test script at the end (just a testification of your
> attachment) to bisect this to the following commit:
> 
>     commit 501770e1bb5d132ae4f79aa96715f07f6b84e1f6
>     Author: Fredrik Gustafsson <[email-masked]>
>     Date:   Mon Aug 15 23:17:47 2011 +0200
> 
>         Move git-dir for submodules
>         
>         Move git-dir for submodules into $GIT_DIR/modules/[name_of_submodule] of
>         the superproject. This is a step towards being able to delete submodule
>         directories without loosing the information from their .git directory
>         as that is now stored outside the submodules work tree.
>         
>         This is done relying on the already existent .git-file functionality.
>         When adding or updating a submodule whose git directory is found under
>         $GIT_DIR/modules/[name_of_submodule], don't clone it again but simply
>         point the .git-file to it and remove the now stale index file from it.
>         The index will be recreated by the following checkout.
>         
>         This patch will not affect already cloned submodules at all.
>         
>         Tests that rely on .git being a directory have been fixed.
>         
>         Signed-off-by: Fredrik Gustafsson <[email-masked]>
>         Mentored-by: Jens Lehmann <[email-masked]>
>         Mentored-by: Heiko Voigt <[email-masked]>
>         Signed-off-by: Junio C Hamano <[email-masked]>
> 
> That is, before 501770 I get sub1/sub2 as Manuel said above.  After
> 501770 I get sub1/sub1 (!).  I have not been able to reproduce the
> sub2/sub2 behavior with any version I tried (1.7.0, 1.7.3.4, 1.7.6,
> etc.).  Perhaps there is a configuration setting that changes this?
> 
> In any case, since it's -rc4 time and 501770 is set to go into the
> release, it would be nice if you could check whether this is indeed
> correct/intended.
> 
> 
> 
> ---- 8< ----
> #!/bin/sh
> 
> test_description='submodule change bug'
> . ./test-lib.sh
> 
> ## set current directory as working directory
> wd=`pwd`
> 
> test_expect_success 'set up submodules' '
> ## create repositories to be used as submodules
> mkdir sub1 sub2 remote &&
> (cd sub1 &&
>     git init &&
>     echo "test sub1" >> file &&
>     git add file &&
>     git commit -m "initial commit sub1"
> ) &&
> git clone --bare sub1 remote/sub1.git &&
> (cd sub2 &&
>     git init &&
>     echo "test sub2" >> file &&
>     git add file &&
>     git commit -m "initial commit sub2"
> ) &&
> git clone --bare sub2 remote/sub2.git
> '
> 
> test_expect_success 'set up super-repo' '
> ## create super repository
> git init --bare remote/super.git &&
> git clone remote/super.git super &&
> (cd super &&
>     git submodule add "$wd"/remote/sub1.git sub &&
>     git commit -m "Added submodule sub1 as sub" &&
>     git push -u origin master
> )'
> 
> test_expect_success 'make super-repo with sub1->sub2' '
> ## clone super repository again
> ## and switch submodule sub1 by sub2
> git clone --recursive remote/super.git super2 &&
> (cd super2 &&
>     ## remote submodule sub
>     git config --remove-section submodule.sub &&
>     git rm .gitmodules &&
>     rm -rf sub &&
>     git rm sub &&
>     git commit -m "Removed submodule sub" &&
>     ## add submodule sub2 as sub
>     git submodule add "$wd"/remote/sub2.git sub &&
>     git commit -m "Added submodule sub2 as sub" &&
>     git push
> )
> '
> 
> test_expect_success 'pull from super2' '
> ## now pull super
> (cd super &&
>     git pull &&
>     ## this will fail
>     if ! git submodule update --init; then
>     ## so sync first und update again
>         git submodule sync &&
>         git submodule update --init
>     fi &&
>     ## now sub is corrupt
>     (cd sub &&
>         git log master >log1 && ## this is from sub1
>         echo "# next line should be: initial commit from sub1" &&
>         grep sub1 log1 &&
>         echo "# next line should be: initial commit from sub2" &&
>         git log origin/master >log2 && ## this is from sub2
>         grep sub2 log2
>     )
> )
> '
> 
> test_done

This is something I did not thought about when writing that patch. The
reason that this fails is that the part when the first submodule is
removed is no longer complete (as intended). Before this patch
     git config --remove-section submodule.sub &&
     git rm .gitmodules &&
     rm -rf sub &&
Did remove all the data associated with the submodule. That's no longer
the case. The submodule repository is stored in .git/modules and is
still there. When a new submodule (at the same path) is added, the
repository already associated with that path will be used.

If to the three lines above from the test a line that removes the
repository also is added, the test will work:
     git config --remove-section submodule.sub &&
     git rm .gitmodules &&
     rm -rf sub &&
     rm -rf .git/modules/sub &&
     git rm sub &&

The point beeing that 'rm -rf sub' is not longer sufficient to remove a
submodule and its repository. As 'rm -rf' is not a git command, I'm not
sure how this should be solved (or if it should at all).

* Note: the .git/modules/sub is only valid if .git is a git-dir, if it's
  a gitfile, it's not...

-- 
Med vänliga hälsningar
Fredrik Gustafsson

E-post: [email-masked]
Tel. nr.: 0733 60 82 74