[PATCH 0/3] git-svn-externals PoC (in a sh script)

[PATCH 0/3] git-svn-externals PoC (in a sh script)

[Eddy Petrișor]

Hello,

I have started a while back working on support for svn:externals
support for git-svn, but since I'm not that satisfied with the current
status of the patch, I haven't modified git-svn itself and just left
the sh script I made as a PoC as it was.

There's still work to be done to it, but I the current version is
functional enough to be probably found useful by more people than
myself.


Current status follows:

    Current functionality:
     - fetches all the externals of an already svn-fetched repo
     - support for svn:externals refresh
     - if the location of the external has changed, the current working
       copy will be placed aside and a new directory will be created
       instead
     - if the remote URI is the same (maybe a verison bump, there will
       be a 'git svn rebase'
     - remove support (useful for testing purposes or clean restarts)
     - avoid zombie externals at all costs - in some repos empty
       svn:externals might exist; svn ignores such externals, so git should
       do the same

    TODO:
     - take into account the revision of an external, if it exists
     - do not do deep svn cloning, to avoid legthy operations, just pull HEAD
       (this actually needs changes in git-svn itself)
     - use/create shallow copies to git svn repos (one revision should be enough
       for most externals)
     - use submodules for externals




Any comments are welcome.


-- 
Regards,
EddyP
=============================================
"Imagination is more important than knowledge" A.Einstein

[PATCH] git svn: should not display zombie externals

[Eddy Petrișor]

subversion does not process in any way empty svn:externals properties
and doesn't list them in any way; git svn externals shouldn't do that
either

Signed-off-by: Eddy Petrișor <eddy.petrisor@gmail.com>
---
 t/t9101-git-svn-props.sh |   24 +++++++++++++++++++++++-
 1 files changed, 23 insertions(+), 1 deletions(-)

diff --git a/t/t9101-git-svn-props.sh b/t/t9101-git-svn-props.sh
index f420796..e174b30 100755
--- a/t/t9101-git-svn-props.sh
+++ b/t/t9101-git-svn-props.sh
@@ -211,7 +211,29 @@ EOF
 
 test_expect_success 'test proplist' "
 	git-svn proplist . | cmp - prop.expect &&
-	git-svn proplist nested/directory/.keep | cmp - prop2.expect
+	git-svn proplist nested/directory/.keep | cmp - prop2.expect &&
+	cd ..
+	"
+
+test_expect_success 'show external' "
+	cd test_wc &&
+		svn mkdir zombie &&
+		svn propset svn:externals 'externaldir file:///fake/external' zombie &&
+		svn ci -m 'added a fake svn:external' &&
+	cd .. &&
+	git-svn fetch &&
+	git-merge git-svn &&
+	git-svn show-externals | grep -q 'externaldir'
+	"
+
+test_expect_failure 'remove external' "
+	cd test_wc &&
+		svn propset svn:externals '' zombie &&
+		svn ci -m 'deleted the fake external' &&
+	cd .. &&
+	git-svn fetch &&
+	git-merge git-svn &&
+	git-svn show-externals | grep -v ^# | grep -v ^$ | grep -q -v '^/zombie'
 	"
 
 test_done
-- 
1.5.6.3

[PATCH] First crude implementation of git-svn-externals

[Eddy Petrișor]

From: Eddy Petrisor <eddy@epetrisor.dsd.ro>

Current functionality:
 - fetches all the externals of an already svn-fetched repo
 - support for svn:externals refresh
 - if the location of the external has changed, the current working
   copy will be placed aside and a new directory will be created
   instead
 - if the remote URI is the same (maybe a verison bump, there will
   be a 'git svn rebase'
 - remove support (useful for testing purposes or clean restarts)
 - avoid zombie externals at all costs - in some repos empty
   svn:externals might exist; svn ignores such externals, so git should
   do the same

TODO:
 - take into account the revision of an external, if it exists
 - do not do deep svn cloning, to avoid legthy operations, just pull HEAD
 - add shallow copies to git svn repos (one revision should be enough
   for most externals)
 - use submodules for externals
---
 .gitignore           |    1 +
 Makefile             |    1 +
 git-svn-externals.sh |  158 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 160 insertions(+), 0 deletions(-)
 create mode 100755 git-svn-externals.sh

diff --git a/.gitignore b/.gitignore
index bbaf9de..cd2c47d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -123,6 +123,7 @@ git-status
 git-stripspace
 git-submodule
 git-svn
+git-svn-externals
 git-symbolic-ref
 git-tag
 git-tar-tree
diff --git a/Makefile b/Makefile
index bf400e6..b130244 100644
--- a/Makefile
+++ b/Makefile
@@ -265,6 +265,7 @@ SCRIPT_SH += git-sh-setup.sh
 SCRIPT_SH += git-stash.sh
 SCRIPT_SH += git-submodule.sh
 SCRIPT_SH += git-web--browse.sh
+SCRIPT_SH += git-svn-externals.sh
 
 SCRIPT_PERL += git-add--interactive.perl
 SCRIPT_PERL += git-archimport.perl
diff --git a/git-svn-externals.sh b/git-svn-externals.sh
new file mode 100755
index 0000000..38b7af6
--- /dev/null
+++ b/git-svn-externals.sh
@@ -0,0 +1,158 @@
+#!/bin/sh
+
+set -e
+
+GIT_SVN_LAST_DIR="$(pwd)"
+trap "cd '$GIT_SVN_LAST_DIR'" INT EXIT KILL
+
+
+USAGE='[help|init|refresh|show]'
+
+LONG_USAGE='git svn externals help
+	print this long help message.
+git svn externals init [<pathspec>...]
+	pull all externals according to current repo state.
+git svn externals refresh [<pathspec>...]
+	removes current externals and repulls them.
+'
+
+
+SUBDIRECTORY_OK=Sometimes
+OPTIONS_SPEC=
+. git-sh-setup
+cd_to_toplevel
+# this should never fail
+require_work_tree
+
+
+GIT_SVN_TOP_DIR="$(pwd)"
+
+svn_revision_to_numrevision ()
+#$1 - repository URI
+#$2 - the revision to be transformed
+{
+	local REVSPEC
+	[ "$2" ] && REVSPEC="-r $2" || REVSPEC=''
+	[ "$1" ] || die "Internal error: no repository given in svn_revision_to_numrevision"
+	LANG=C svn info "$1" "$REVSPEC" | grep 'Revision' | cut -f 2 -d ':' | sed 's#^\ \ *##g'
+}
+
+prune_inital_slashes ()
+{
+	echo "$1" | sed 's#^//*##g'
+}
+
+git_svn_externals_remove ()
+{
+	local LASTDIR="$(pwd)"
+	cd_to_toplevel && TOPLEVELDIR="$(pwd)"
+	git svn show-externals | grep -vE '^(#.*|)$' | while read EXT_DIR REV EXT_SRC ; do
+		if [ -z "$REV" ] ; then
+			echo "Skipping illegal external '$EXT_DIR' which has no value" >&2
+			continue
+		fi
+		EXT_DIR="$(prune_inital_slashes "$EXT_DIR")"
+
+		if [ -z "$EXT_SRC" ] ; then
+			EXT_SRC="$REV"
+			REV="-rHEAD"
+		fi
+		REV=${REV#-r}
+
+		echo "Removing local copy for external '$EXT_DIR' ( $EXT_SRC@$REV )"
+		rm -fr "$EXT_DIR"
+	done
+
+	cd "$LASTDIR"
+}
+
+git_svn_init_dir_repo ()
+# assumes is ran from the parent dir of the repo
+# $1 EXT_DIR
+# $2 EXT_SRC
+# $3 REV
+{
+	local LASTDIR="$(pwd)"
+
+	mkdir -p "$1"
+	cd "$1"
+	git svn init "$2"
+	git svn fetch
+
+	cd "$LASTDIR"
+}
+
+git_svn_externals_refresh_external ()
+#$1 directory where the external is locally
+#$2 remote url
+#$3 revision at which the external is pinned at
+{
+	local LASTDIR="$GIT_SVN_TOP_DIR"
+
+	local EXT_SRC="$2"
+	local EXT_DIR="$1"
+
+	if cd "$EXT_DIR" 2>/dev/null; then
+		local OLDDIREXISTS=yes
+		local OLD_EXT_SRC=$(git config --get svn-remote.svn.url)
+		local OLD_EXT_REV=$(git config --get svn-remote.svn.revision || svn_revision_to_numrevision $OLD_EXT_SRC HEAD)
+		cd -
+	else
+		local OLDDIREXISTS=no
+		local OLD_EXT_SRC="${EXT_SRC}"
+		local OLD_EXT_REV=$(svn_revision_to_numrevision $EXT_SRC HEAD)
+	fi
+
+	if [ "$OLD_EXT_SRC" != "$EXT_SRC" ] ; then
+		echo "External URI definition changed. Moving away the old repo and pulling a new one."
+		cd "$LASTDIR"
+		local SUBDIR="$(basename "$EXT_DIR")"
+		local DIRNAME="$(dirname "$EXT_DIR")"
+		NEWDIR="$(mktemp -p "$DIRNAME" "$SUBDIR.obsolete.XXXXXX")" && rm -f "$NEWDIR"
+		mv "$EXT_DIR" "$NEWDIR"
+		echo "  Old copy moved to '$NEWDIR'."
+		git_svn_init_dir_repo "$EXT_DIR" "$EXT_SRC" "$REV"
+	elif [ "$OLDDIREXISTS" = "no" ] ; then
+		git_svn_init_dir_repo "$EXT_DIR" "$EXT_SRC" "$REV"
+	else
+		cd "$EXT_DIR"
+		git svn rebase
+	fi
+
+	cd "$LASTDIR"
+	return 0
+}
+
+git_svn_externals_refresh ()
+{
+	git svn show-externals | grep -vE '^(#.*|)$' | while read EXT_DIR REV EXT_SRC ; do
+		if [ -z "$REV" ] ; then
+			echo "Skipping illegal external '$EXT_DIR' which has no value" >&2
+			continue
+		fi
+		EXT_DIR="$(prune_inital_slashes "$EXT_DIR")"
+
+		if [ -z "$EXT_SRC" ] ; then
+			EXT_SRC="$REV"
+			REV="-rHEAD"
+		fi
+		REV=${REV#-r}
+
+		echo "Refreshing local copy for external '$EXT_DIR' ( $EXT_SRC@$REV )"
+		cd "$GIT_SVN_TOP_DIR"
+		git_svn_externals_refresh_external "$EXT_DIR" "$EXT_SRC" "$REV"
+	done
+
+	cd "$GIT_SVN_TOP_DIR"
+}
+
+
+case $1 in
+	--remove)
+		git_svn_externals_remove
+		;;
+	-r|--refresh|*)
+		git_svn_externals_refresh
+		;;
+esac
+
-- 
1.5.6.3

[PATCH] added a test frame for git-svn-externals.sh

[Eddy Petrișor]

From: Eddy Petrisor <eddy@epetrisor.dsd.ro>

---
 t/t9130-git-svn-externals-fetch.sh |   76 ++++++++++++++++++++++++++++++++++++
 t/t9130/clean.dump                 |   74 +++++++++++++++++++++++++++++++++++
 2 files changed, 150 insertions(+), 0 deletions(-)
 create mode 100755 t/t9130-git-svn-externals-fetch.sh
 create mode 100644 t/t9130/clean.dump

diff --git a/t/t9130-git-svn-externals-fetch.sh b/t/t9130-git-svn-externals-fetch.sh
new file mode 100755
index 0000000..70335ad
--- /dev/null
+++ b/t/t9130-git-svn-externals-fetch.sh
@@ -0,0 +1,76 @@
+#!/bin/sh
+#
+# Copyright (c) 2008 Eddy Petrișor
+
+
+test_description='git svn properties pulls and updates externals'
+. ./lib-git-svn.sh
+
+test_expect_success 'load repository with base directory' '
+	svnadmin load -q "$rawsvnrepo" < ../t9130/clean.dump
+	'
+
+test_expect_success 'checkout the simple repo' '
+	mkdir -p gitrepo &&
+	(
+		cd gitrepo &&
+		git svn init "$svnrepo/trunk" &&
+		git svn fetch &&
+		cd ..
+	)
+	'
+
+test_expect_success 'add an external in the repo' '
+	svn checkout "$svnrepo/trunk" work.svn &&
+	(
+		cd work.svn &&
+		svn propset svn:externals "external $(echo $svnrepo|sed "s#\ #%20#")/external" . &&
+		svn commit -m "added external directory to repo" &&
+		svn update &&
+		cd .. &&
+		rm -fr work.svn
+	)
+	'
+
+test_expect_success 'git svn fetch the property changes and fetch externals' '
+	cd gitrepo &&
+	git svn rebase &&
+	git svn show-externals | grep -q "^/external" &&
+	git svn-externals &&
+	cd ..
+	'
+
+test_expect_success 'create an external pinned to a version' '
+	svn checkout "$svnrepo/trunk" work.svn &&
+	(
+		cd work.svn &&
+		svn propset svn:externals "external -r2 $(echo $svnrepo|sed "s#\ #%20#")/external" . &&
+		svn commit -m "pin the external to -r2" &&
+		svn update &&
+		cd .. &&
+		rm -fr work.svn
+	) &&
+	svn checkout "$svnrepo/external" ext.svn &&
+	(
+		cd ext.svn &&
+		echo "0" > externalfile &&
+		svn commit -m "mark the bad floating revision for the external" &&
+		cd .. &&
+		rm -fr ext.svn
+	)
+	'
+
+test_expect_failure 'refresh revisioned externals in git' '
+	cd gitrepo &&
+	git config --list >> /tmp/test &&
+	git svn rebase &&
+	git svn-externals &&
+	grep 1 external/externalfile &&
+	cd ..
+	'
+
+#TODO: add test_expect_success for a URI change for the external
+
+
+
+test_done
diff --git a/t/t9130/clean.dump b/t/t9130/clean.dump
new file mode 100644
index 0000000..9e26b66
--- /dev/null
+++ b/t/t9130/clean.dump
@@ -0,0 +1,74 @@
+SVN-fs-dump-format-version: 2
+
+UUID: 13066bdd-4590-4b1e-8a3f-4379dfa77d1b
+
+Revision-number: 0
+Prop-content-length: 56
+Content-length: 56
+
+K 8
+svn:date
+V 27
+2008-07-10T11:26:12.351202Z
+PROPS-END
+
+Revision-number: 1
+Prop-content-length: 125
+Content-length: 125
+
+K 7
+svn:log
+V 26
+bare structure of the repo
+K 10
+svn:author
+V 4
+eddy
+K 8
+svn:date
+V 27
+2008-07-10T11:31:04.711302Z
+PROPS-END
+
+Node-path: external
+Node-kind: dir
+Node-action: add
+Prop-content-length: 10
+Content-length: 10
+
+PROPS-END
+
+
+Node-path: external/externalfile
+Node-kind: file
+Node-action: add
+Prop-content-length: 10
+Text-content-length: 2
+Text-content-md5: b026324c6904b2a9cb4b88d6d61c81d1
+Content-length: 12
+
+PROPS-END
+1
+
+
+Node-path: trunk
+Node-kind: dir
+Node-action: add
+Prop-content-length: 10
+Content-length: 10
+
+PROPS-END
+
+
+Node-path: trunk/dummyfile
+Node-kind: file
+Node-action: add
+Prop-content-length: 10
+Text-content-length: 1
+Text-content-md5: 68b329da9893e34099c7d8ad5cb9c940
+Content-length: 11
+
+PROPS-END
+
+
+
-- 
1.5.6.3

Re: [PATCH 0/3] git-svn-externals PoC (in a sh script)

[Eddy Petrișor]

Please CC me, I am not listed.

2008/8/29 Eddy Petrișor <eddy.petrisor@gmail.com>:
> Hello,
>
> I have started a while back working on support for svn:externals
> support for git-svn, but since I'm not that satisfied with the current
> status of the patch, I haven't modified git-svn itself and just left
> the sh script I made as a PoC as it was.
>
> There's still work to be done to it, but I the current version is
> functional enough to be probably found useful by more people than
> myself.
>
>
> Current status follows:
>
>    Current functionality:
>     - fetches all the externals of an already svn-fetched repo
>     - support for svn:externals refresh
>     - if the location of the external has changed, the current working
>       copy will be placed aside and a new directory will be created
>       instead
>     - if the remote URI is the same (maybe a verison bump, there will
>       be a 'git svn rebase'
>     - remove support (useful for testing purposes or clean restarts)
>     - avoid zombie externals at all costs - in some repos empty
>       svn:externals might exist; svn ignores such externals, so git should
>       do the same
>
>    TODO:
>     - take into account the revision of an external, if it exists
>     - do not do deep svn cloning, to avoid legthy operations, just pull HEAD
>       (this actually needs changes in git-svn itself)
>     - use/create shallow copies to git svn repos (one revision should be enough
>       for most externals)
>     - use submodules for externals
>
>
>
>
> Any comments are welcome.
>
>
> --
> Regards,
> EddyP
> =============================================
> "Imagination is more important than knowledge" A.Einstein
>



-- 
Regards,
EddyP
=============================================
"Imagination is more important than knowledge" A.Einstein

Re: [PATCH 0/3] git-svn-externals PoC (in a sh script)

[Eric Wong]

[-- Attachment #1: Type: text/plain, Size: 2898 bytes --]

Eddy Petrișor <eddy.petrisor@gmail.com> wrote:
> Hello,

Hi Eddy,

> I have started a while back working on support for svn:externals
> support for git-svn, but since I'm not that satisfied with the current
> status of the patch, I haven't modified git-svn itself and just left
> the sh script I made as a PoC as it was.
> 
> There's still work to be done to it, but I the current version is
> functional enough to be probably found useful by more people than
> myself.

Cool.

I definitely like the separate script approach.  Not sure if you read my
posts, your PoC seems inline with my thoughts on handling externals be
seen here:

http://article.gmane.org/gmane.comp.version-control.git/91283
http://article.gmane.org/gmane.comp.version-control.git/91293

> Current status follows:
> 
>     Current functionality:
>      - fetches all the externals of an already svn-fetched repo
>      - support for svn:externals refresh
>      - if the location of the external has changed, the current working
>        copy will be placed aside and a new directory will be created
>        instead
>      - if the remote URI is the same (maybe a verison bump, there will
>        be a 'git svn rebase'
>      - remove support (useful for testing purposes or clean restarts)
>      - avoid zombie externals at all costs - in some repos empty
>        svn:externals might exist; svn ignores such externals, so git should
>        do the same
> 
>     TODO:
>      - take into account the revision of an external, if it exists
>      - do not do deep svn cloning, to avoid legthy operations, just pull HEAD
>        (this actually needs changes in git-svn itself)

git svn clone -r<latest_revision_number> URL should work if you extract
the revision number easily.  Specifying "-rHEAD" will only work if the
branch of the external you're tracking was the last modified revision in
the repository, so it's not very useful.  "svn log" seems to have the
same semantics as git-svn as far as -rHEAD being useful or not...

>      - use/create shallow copies to git svn repos (one revision should be enough
>        for most externals)
>      - use submodules for externals

I'm not sure if mapping submodules to externals is a good idea
because externals don't require exact revisions and submodules do.
There's also an issue I was just made aware of two days ago with
submodules and git-svn that I haven't had time to work on.

Another user also privately reported a bug to me about git-svn having
trouble dcommitting when using submodules.  I've attached the test case
here in case you have any thoughts on how to handle this (I think the
easiest would be to ignore submodules on dcommit entirely).

> Any comments are welcome.

Also some small portability issues: "grep -q" is definitely unportable
in my experience.  There are probably some more that I am missing my eye
at this time of night...

-- 
Eric Wong

[-- Attachment #2: t9126-git-svn-submodule.sh --]
[-- Type: application/x-sh, Size: 988 bytes --]

RFH: git-svn and submodules

[Eric Wong]

Eric Wong <normalperson@yhbt.net> wrote:
> Eddy Petrișor <eddy.petrisor@gmail.com> wrote:
> >      - use submodules for externals
> 
> I'm not sure if mapping submodules to externals is a good idea
> because externals don't require exact revisions and submodules do.
> There's also an issue I was just made aware of two days ago with
> submodules and git-svn that I haven't had time to work on.

Definitely a complex problem.  A new submodule from git mapping to an
new svn:external probably doesn't work very well, either.  There's also
UI/policy issues with creating a new directory/branch in SVN to house
the external.  It will also will not work for (the few) projects that
are using the root of the SVN repository directly.

> Another user also privately reported a bug to me about git-svn having
> trouble dcommitting when using submodules.  I've attached the test case
> here in case you have any thoughts on how to handle this (I think the
> easiest would be to ignore submodules on dcommit entirely).

Well, I'm stuck on this problem.  Since that user that reported this
emailed me privately, I'm bcc-ing him on this issue, too.

I've tried the following, and it gets the commit into SVN without the
submodules using dcommit.

--- a/git-svn.perl
+++ b/git-svn.perl
@@ -3364,7 +3364,7 @@ sub new {
 
 sub generate_diff {
 	my ($tree_a, $tree_b) = @_;
-	my @diff_tree = qw(diff-tree -z -r);
+	my @diff_tree = qw(diff-tree --ignore-submodules -z -r);
 	if ($_cp_similarity) {
 		push @diff_tree, "-C$_cp_similarity";
 	} else {

However, all is not well.  The rebase/reset step in dcommit gets
triggered with a rebase because the submodule is dropped from the tree
when we refetch the commit from SVN.


The simplest would be to say we don't support mixing submodules with
git-svn.  I'm fine with this statement.


The clever way would be introducing a custom revprop for git-svn
to use to declare submodule dependencies.  Then git-svn can
recreate the information needed for a revprop on fetch.

That would mean regular SVN users would not get access to submodules.

Cutting SVN users off from being able to effectively use an SVN repo
doesn't make sense to me.  The project might as well be using git
entirely and ditch SVN.  Of course, some PHB somewhere will want to keep
the project on SVN even when all the developers are using git-svn...



Since gmane seems to have broken the attachment I previously sent,
here again is the patch inlined:

From bddc57078188d4798a9a74401aa0da8ca35a4db1 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Fri, 29 Aug 2008 02:09:50 -0700
Subject: [PATCH] git-svn: Add submodule ignore test

---
 t/t9126-git-svn-submodule.sh |   44 ++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 44 insertions(+), 0 deletions(-)
 create mode 100644 t/t9126-git-svn-submodule.sh

diff --git a/t/t9126-git-svn-submodule.sh b/t/t9126-git-svn-submodule.sh
new file mode 100644
index 0000000..c4b3489
--- /dev/null
+++ b/t/t9126-git-svn-submodule.sh
@@ -0,0 +1,44 @@
+#!/bin/sh
+# Copyright (c) 2008 Eric Wong
+
+test_description='git-svn submodule'
+
+. ./lib-git-svn.sh
+
+test_expect_success 'setup repo' 'svn co "$svnrepo" svnfoo'
+
+test_expect_success 'add files to svn repo' '
+	touch svnfoo/foo.txt
+	(cd svnfoo && svn add * && svn commit -m "test commit")
+	'
+
+test_expect_success 'git-svn clone the svn repo' '
+	git-svn clone "$svnrepo" gitfoo
+	'
+
+test_expect_success 'create a git repo to include as a submodule' '
+	mkdir git_to_include
+	(
+		cd git_to_include &&
+		git init &&
+		echo "This is another file" > another.txt &&
+		git add another.txt &&
+		git commit -a -m "Adding file to submodule" &&
+		git log
+	)
+	'
+
+test_expect_success 'add the submodule' '
+	(
+		cd gitfoo &&
+		git submodule add "$remove_trash"/git_to_include \
+		  git_as_submodule &&
+		git commit -a -m "Adding submodule"
+	)
+	'
+
+test_expect_success 'try to rebase and commit' '
+	( cd gitfoo && git svn rebase && git svn dcommit )
+	'
+
+test_done
-- 
Eric Wong

Re: [PATCH 0/3] git-svn-externals PoC (in a sh script)

[Eddy Petrișor]

(Please keep the CC. Thanks)

2008/8/29 Eric Wong <normalperson@yhbt.net>:
> Eddy Petrișor <eddy.petrisor@gmail.com> wrote:
>> Hello,
>
> Hi Eddy,

Hello and sorry for the late reply.

(I was on a small vacation away from the computer in the last two weeks.)

>> I have started a while back working on support for svn:externals
>> support for git-svn, but since I'm not that satisfied with the current
>> status of the patch, I haven't modified git-svn itself and just left
>> the sh script I made as a PoC as it was.
>>
>> There's still work to be done to it, but I the current version is
>> functional enough to be probably found useful by more people than
>> myself.
>
> Cool.
>
> I definitely like the separate script approach.  Not sure if you read my
> posts, your PoC seems inline with my thoughts on handling externals be
> seen here:
>
> http://article.gmane.org/gmane.comp.version-control.git/91283
> http://article.gmane.org/gmane.comp.version-control.git/91293

WRT the revision pinning, it seems to me that is enough to locate that
revision on the URI in question and checkout that revision. Still I am
unsure if it would be wise to (stash +) svn rebase + checkout the
pinned version (+ stash pop), since one would needlessly pull newer
stuff as the remote svn HEAD advances, but the pinned version might
simply stagnate.


I already have/wrote some code that follows the remote HEAD or a
specific for the necessary, but I am unsure if is still present in the
PoC script, is not that hard (in sh - a "svn info" on the URI, not on
the local copy would reveal the real revision of the HEAD).

>> Current status follows:
>>
>>     Current functionality:
>>      - fetches all the externals of an already svn-fetched repo
>>      - support for svn:externals refresh
>>      - if the location of the external has changed, the current working
>>        copy will be placed aside and a new directory will be created
>>        instead
>>      - if the remote URI is the same (maybe a verison bump, there will
>>        be a 'git svn rebase'
>>      - remove support (useful for testing purposes or clean restarts)
>>      - avoid zombie externals at all costs - in some repos empty
>>        svn:externals might exist; svn ignores such externals, so git should
>>        do the same
>>
>>     TODO:
>>      - take into account the revision of an external, if it exists
>>      - do not do deep svn cloning, to avoid legthy operations, just pull HEAD
>>        (this actually needs changes in git-svn itself)
>
> git svn clone -r<latest_revision_number> URL should work if you extract
> the revision number easily.

Why was I under the impression that this wasn't working? Or was I
expecting a shallow repo?

> Specifying "-rHEAD" will only work if the
> branch of the external you're tracking was the last modified revision in
> the repository, so it's not very useful.

as I already said, "svn info URI" can return the real revision, no
need to ness with the pseudo-revision HEAD.

>  "svn log" seems to have the
> same semantics as git-svn as far as -rHEAD being useful or not...
>
>>      - use/create shallow copies to git svn repos (one revision should be enough
>>        for most externals)
>>      - use submodules for externals
>
> I'm not sure if mapping submodules to externals is a good idea
> because externals don't require exact revisions and submodules do.

I don't think I can follow you. Externals actually require exact
revisions or can be made to pretend as if they do in git-svn context
with continuous HEAD refresh.

> There's also an issue I was just made aware of two days ago with
> submodules and git-svn that I haven't had time to work on.
>
> Another user also privately reported a bug to me about git-svn having
> trouble dcommitting when using submodules.  I've attached the test case
> here in case you have any thoughts on how to handle this (I think the
> easiest would be to ignore submodules on dcommit entirely).

Probably, and try later to tackle the problem.

>> Any comments are welcome.
>
> Also some small portability issues: "grep -q" is definitely unportable
> in my experience.  There are probably some more that I am missing my eye
> at this time of night...

Will fix it.

-- 
Regards,
EddyP
=============================================
"Imagination is more important than knowledge" A.Einstein

Re: [PATCH 0/3] git-svn-externals PoC (in a sh script)

[Eddy Petrișor]

2008/9/10 Eddy Petrișor <eddy.petrisor@gmail.com>:
> (Please keep the CC. Thanks)
> I already have/wrote some code that follows the remote HEAD or a
> specific for the necessary, but I am unsure if is still present in the

That was supposed to be:

> specific revision for such cases, but I am unsure if is still present in the


-- 
Regards,
EddyP
=============================================
"Imagination is more important than knowledge" A.Einstein

[PATCH] git svn: should not display zombie externals

[Eddy Petrișor]

during the history of a repo a svn:external could be added and later
removed; such externals shouldn't be displayed since only the current
state is relevant

I have encountered the broken behaviour on a git-svn repo with the svn
origin a repo served via https (git 1:1.5.6-1~bpo40+1 from Debian Etch
backports.) I hope this bug doesn't have to do with the svn repo being
served via https.

Signed-off-by: Eddy Petrișor <eddy.petrisor@gmail.com>
---
  t/t9101-git-svn-props.sh |   23 ++++++++++++++++++++++-
  1 files changed, 22 insertions(+), 1 deletions(-)

diff --git a/t/t9101-git-svn-props.sh b/t/t9101-git-svn-props.sh
index f420796..e5ab748 100755
--- a/t/t9101-git-svn-props.sh
+++ b/t/t9101-git-svn-props.sh
@@ -211,7 +211,28 @@ EOF

  test_expect_success 'test proplist' "
  	git-svn proplist . | cmp - prop.expect &&
-	git-svn proplist nested/directory/.keep | cmp - prop2.expect
+	git-svn proplist nested/directory/.keep | cmp - prop2.expect &&
+	cd ..
+	"
+
+test_expect_success 'show external' "
+	cd test_wc &&
+		svn propset svn:externals 'zombie file:///fake/external' . &&
+		svn ci -m 'added a fake svn:external' &&
+	cd .. &&
+	git-svn fetch &&
+	git-merge git-svn &&
+	git-svn show-externals | grep -q 'zombie'
+	"
+
+test_expect_success 'remove external' "
+	cd test_wc &&
+		svn propdel svn:externals . &&
+		svn ci -m 'deleted the fake external' &&
+	cd .. &&
+	git-svn fetch &&
+	git-merge git-svn &&
+	git-svn show-externals | grep -q -v 'zombie'
  	"

  test_done
-- 
1.5.6.3

[PATCH] git svn: should not display zombie externals

[Eddy Petrișor]

during the history of a repo a svn:external could be added and later
removed; such externals shouldn't be displayed since only the current
state is relevant

I have encountered the broken behaviour on a git-svn repo with the svn
origin a repo served via https (git 1:1.5.6-1~bpo40+1 from Debian Etch
backports.) I hope this bug doesn't have to do with the svn repo being
served via https.

Signed-off-by: Eddy Petrișor <eddy.petrisor@gmail.com>
---
  t/t9101-git-svn-props.sh |   23 ++++++++++++++++++++++-
  1 files changed, 22 insertions(+), 1 deletions(-)

diff --git a/t/t9101-git-svn-props.sh b/t/t9101-git-svn-props.sh
index f420796..e5ab748 100755
--- a/t/t9101-git-svn-props.sh
+++ b/t/t9101-git-svn-props.sh
@@ -211,7 +211,28 @@ EOF

  test_expect_success 'test proplist' "
  	git-svn proplist . | cmp - prop.expect &&
-	git-svn proplist nested/directory/.keep | cmp - prop2.expect
+	git-svn proplist nested/directory/.keep | cmp - prop2.expect &&
+	cd ..
+	"
+
+test_expect_success 'show external' "
+	cd test_wc &&
+		svn propset svn:externals 'zombie file:///fake/external' . &&
+		svn ci -m 'added a fake svn:external' &&
+	cd .. &&
+	git-svn fetch &&
+	git-merge git-svn &&
+	git-svn show-externals | grep -q 'zombie'
+	"
+
+test_expect_success 'remove external' "
+	cd test_wc &&
+		svn propdel svn:externals . &&
+		svn ci -m 'deleted the fake external' &&
+	cd .. &&
+	git-svn fetch &&
+	git-merge git-svn &&
+	git-svn show-externals | grep -q -v 'zombie'
  	"

  test_done
-- 
1.5.6.3


This one is properly formated (the previous one was a copy/paste from the output of the console).

Re: [PATCH] git svn: should not display zombie externals

[Eddy Petrișor]

2008/8/1 Eddy Petrișor <eddy.petrisor@gmail.com>:
>
> during the history of a repo a svn:external could be added and later
> removed; such externals shouldn't be displayed since only the current
> state is relevant
>
> I have encountered the broken behaviour on a git-svn repo with the svn
> origin a repo served via https (git 1:1.5.6-1~bpo40+1 from Debian Etch
> backports.) I hope this bug doesn't have to do with the svn repo being
> served via https.

I have finally found the root of this breakage. It seems that the
older versions of subversion allowed a svn:external property to be
empty, thus generating the zombie externals. Newer versions of
subversion (I checked this with 1.4.2 (r22196) from Debian Etch) do
not allow such empty svn:externals.

> Signed-off-by: Eddy Petrișor <eddy.petrisor@gmail.com>
> ---
>  t/t9101-git-svn-props.sh |   23 ++++++++++++++++++++++-
>  1 files changed, 22 insertions(+), 1 deletions(-)
>
> diff --git a/t/t9101-git-svn-props.sh b/t/t9101-git-svn-props.sh
> index f420796..e5ab748 100755
> --- a/t/t9101-git-svn-props.sh
> +++ b/t/t9101-git-svn-props.sh
> @@ -211,7 +211,28 @@ EOF
>
>  test_expect_success 'test proplist' "
>        git-svn proplist . | cmp - prop.expect &&
> -       git-svn proplist nested/directory/.keep | cmp - prop2.expect
> +       git-svn proplist nested/directory/.keep | cmp - prop2.expect &&
> +       cd ..
> +       "
> +
> +test_expect_success 'show external' "
> +       cd test_wc &&
> +               svn propset svn:externals 'zombie file:///fake/external' .
> &&
> +               svn ci -m 'added a fake svn:external' &&
> +       cd .. &&
> +       git-svn fetch &&
> +       git-merge git-svn &&
> +       git-svn show-externals | grep -q 'zombie'
> +       "
> +
> +test_expect_success 'remove external' "
> +       cd test_wc &&
> +               svn propdel svn:externals . &&
> +               svn ci -m 'deleted the fake external' &&
> +       cd .. &&
> +       git-svn fetch &&
> +       git-merge git-svn &&
> +       git-svn show-externals | grep -q -v 'zombie'
>        "
>
>  test_done
> --
> 1.5.6.3
>
>
> This one is properly formated (the previous one was a copy/paste from the
> output of the console).
>



-- 
Regards,
EddyP
=============================================
"Imagination is more important than knowledge" A.Einstein