[PATCH/RFC] filter-branch: Fix to allow replacing submodules with another content

[PATCH/RFC] filter-branch: Fix to allow replacing submodules with another content

[Michal Sojka]

When git filter-branch is used to replace a submodule with another
content, it always fails on the first commit. Consider a repository with
directory submodule containing a submodule. If I want to remove the
submodule and replace it with a file, the following command fails.

git filter-branch --tree-filter 'rm -rf submodule &&
				 git rm -q submodule &&
				 mkdir submodule &&
				 touch submodule/file'

The error message is:
error: submodule: is a directory - add files inside instead

The reason is that git diff-index, which generates a part of the list of
files to update-index, emits also the removed submodule even if it was
replaced by a real directory.

Adding --ignored-submodules solves the problem for me and
tests in t7003-filter-branch.sh passes correctly.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
---
 git-filter-branch.sh |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 195b5ef..d4ac7fb 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -331,7 +331,7 @@ while read commit parents; do
 			die "tree filter failed: $filter_tree"
 
 		(
-			git diff-index -r --name-only $commit &&
+			git diff-index -r --name-only --ignore-submodules $commit && 
 			git ls-files --others
 		) > "$tempdir"/tree-state || exit
 		git update-index --add --replace --remove --stdin \
-- 
1.6.6

Re: [PATCH/RFC] filter-branch: Fix to allow replacing submodules with another content

[Johannes Schindelin]

Hi,

On Mon, 11 Jan 2010, Michal Sojka wrote:

> When git filter-branch is used to replace a submodule with another
> content, it always fails on the first commit. Consider a repository with
> directory submodule containing a submodule. If I want to remove the
> submodule and replace it with a file, the following command fails.
> 
> git filter-branch --tree-filter 'rm -rf submodule &&
> 				 git rm -q submodule &&
> 				 mkdir submodule &&
> 				 touch submodule/file'
> 
> The error message is:
> error: submodule: is a directory - add files inside instead
> 
> The reason is that git diff-index, which generates a part of the list of
> files to update-index, emits also the removed submodule even if it was
> replaced by a real directory.
> 
> Adding --ignored-submodules solves the problem for me and
> tests in t7003-filter-branch.sh passes correctly.

Have you tested replacing one revision of a submodule with another?

Ciao,
Dscho

Re: [PATCH/RFC] filter-branch: Fix to allow replacing submodules with another content

[Michal Sojka]

On Monday 11 of January 2010 19:02:06 Johannes Schindelin wrote:
> Hi,
> 
> On Mon, 11 Jan 2010, Michal Sojka wrote:
> > When git filter-branch is used to replace a submodule with another
> > content, it always fails on the first commit. Consider a repository with
> > directory submodule containing a submodule. If I want to remove the
> > submodule and replace it with a file, the following command fails.
> >
> > git filter-branch --tree-filter 'rm -rf submodule &&
> > 				 git rm -q submodule &&
> > 				 mkdir submodule &&
> > 				 touch submodule/file'
> >
> > The error message is:
> > error: submodule: is a directory - add files inside instead
> >
> > The reason is that git diff-index, which generates a part of the list of
> > files to update-index, emits also the removed submodule even if it was
> > replaced by a real directory.
> >
> > Adding --ignored-submodules solves the problem for me and
> > tests in t7003-filter-branch.sh passes correctly.
> 
> Have you tested replacing one revision of a submodule with another?

Hi,

no, I didn't try it. I wanted to test it now but it seems to me that
it cannot work even without my patch. I may be trying wrong
--tree-filter. If anybody has a better idea, let me know.

I suppose that in order to change the revision of the submodule within
the tree filter, I need to checkout the original revision first. Then
I could use e.g. git reset HEAD^ to change it. Unfortunately even
checkout of the submodule fails:

$ git filter-branch --tree-filter "git submodule update --init" HEAD
Clone of '/tmp/submod' into submodule path 'submod' failed
tree filter failed: git submodule update --init

This is because filter-branch sets GIT_WORKING_TREE to "." which
causes clone to fail.

Replacing a revision of a submodule can be done only by manipulating
index, but for this case you would use index-filter rather than
tree-filter. Right?

I have created a few tests for testing filter-branch with submodules.
See the patch bellow.

Michal

>From 9aa38185d795061a2f00204d181244f906280b5a Mon Sep 17 00:00:00 2001
From: Michal Sojka <sojkam1@fel.cvut.cz>
Date: Wed, 13 Jan 2010 15:15:28 +0100
Subject: [PATCH] filter-branch: Add tests for submodules


Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
---
 t/t7003-filter-branch.sh |   26 ++++++++++++++++++++++++++
 1 files changed, 26 insertions(+), 0 deletions(-)

diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 9503875..daebd17 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -306,4 +306,30 @@ test_expect_success '--remap-to-ancestor with filename filters' '
 	test $orig_invariant = $(git rev-parse invariant)
 '
 
+test_expect_success 'setup submodule' '
+	rm -rf * .*
+	git init &&
+	test_commit file &&
+	mkdir submod &&
+	submodurl="$PWD/submod"
+	( cd submod &&
+	  git init &&
+	  test_commit file-in-submod ) &&
+	git submodule add "$submodurl"
+	git commit -m "added submodule" &&
+	test_commit add-file &&
+	( cd submod && test_commit add-in-submodule ) &&
+	git add submod &&
+	git commit -m "changed submodule"
+'
+
+test_expect_failure 'rewrite submodule with another content' '
+	git filter-branch --tree-filter "test -d submod && {
+					 rm -rf submod &&
+					 git rm -rf --quiet submod &&
+					 mkdir submod &&
+					 : > submod/file
+					 } || :"
+'
+
 test_done
-- 
1.6.6