git-svn: cloning an SVN repo with sub-branches and sub-tags

git-svn: cloning an SVN repo with sub-branches and sub-tags

[Daniel Pocock]

I'm trying to use git-svn to clone a repo that has sub-branches and sub-tags

The SVN layout is like this:

branches/user1/workitem1
branches/user1/workitem2
branches/user2/workitem1
branches/user2/workitem2

tags/v1.0
tags/v1.1
tags/debian/v1.0-1
tags/debian/v1.1-1
tags/debian/v1.1-2

In git, I would like to have branches with the same names, e.g.

branches/user1/workitem1
branches/user1/workitem2
branches/user2/workitem1
branches/user2/workitem2

and tags like this:

1.0  (notice I don't want to keep the v1.0, drop the v)
1.1
debian/1.0-1
debian/1.1-1
debian/1.1-2

However, if I run

git svn clone \
   -T trunk \
   -t tags \
   -b branches \
   https://flactag.svn.sourceforge.net/svnroot/flactag \
   flactag-git1

I don't get any of the branches or tags, and I get numerous warnings 
like this:

W: -empty_dir: branches/d_pocock/autotools/HTTPFetch.cc
W: -empty_dir: branches/d_pocock/autotools/HTTPFetch.h
W: Cannot find common ancestor between 
f475b655c10b55b36df8376a396c0d9534418617 and 
7e277bfd626a1caa84681e93a7af73e0160eb2eb. Ignoring merge info.
W: Cannot find common ancestor between 
f475b655c10b55b36df8376a396c0d9534418617 and 
a82cd9bd2197a89303558af9aad43fa0c23d1ddb. Ignoring merge info.

I have the impression that git-svn does not like the repository layout, 
and it is not clear from the manual page how to make it work.  Can 
anyone share any advice about this?

Re: git-svn: cloning an SVN repo with sub-branches and sub-tags

[Dave Zarzycki]

You can make this work, but it is very fragile, doesn't scale well, and it only works on recent versions of git. For example, from the LLVM clang project:

ignore-refs = ^refs/remotes/svn/(Apple|ggreif|misc/Apple/williamson-llvm|misc/Apple/Necropolis|tags/Apple|tags/RELEASE_29|tags/RELEASE_30|tags/checker|tags/cremebrulee)
fetch = cfe/trunk:refs/remotes/svn/trunk
branches = cfe/branches/*:refs/remotes/svn/*
branches = cfe/branches/Apple/*:refs/remotes/svn/misc/Apple/*
branches = cfe/branches/Apple/Necropolis/*:refs/remotes/svn/misc/AppleNecropolis/*
branches = cfe/branches/ggreif/*:refs/remotes/svn/misc/ggreif/*
tags = cfe/tags/*:refs/remotes/svn/tags/*
tags = cfe/tags/Apple/*:refs/remotes/svn/tags2/Apple/*
tags = cfe/tags/RELEASE_29/*:refs/remotes/svn/tags2/RELEASE_29/*
tags = cfe/tags/RELEASE_30/*:refs/remotes/svn/tags2/RELEASE_30/*
tags = cfe/tags/checker/*:refs/remotes/svn/tags2/checker/*
tags = cfe/tags/cremebrulee/*:refs/remotes/svn/tags2/cremebrulee/*



On Mar 29, 2012, at 6:47 AM, Daniel Pocock <daniel@pocock.com.au> wrote:

> 
> 
> 
> I'm trying to use git-svn to clone a repo that has sub-branches and sub-tags
> 
> The SVN layout is like this:
> 
> branches/user1/workitem1
> branches/user1/workitem2
> branches/user2/workitem1
> branches/user2/workitem2
> 
> tags/v1.0
> tags/v1.1
> tags/debian/v1.0-1
> tags/debian/v1.1-1
> tags/debian/v1.1-2
> 
> In git, I would like to have branches with the same names, e.g.
> 
> branches/user1/workitem1
> branches/user1/workitem2
> branches/user2/workitem1
> branches/user2/workitem2
> 
> and tags like this:
> 
> 1.0  (notice I don't want to keep the v1.0, drop the v)
> 1.1
> debian/1.0-1
> debian/1.1-1
> debian/1.1-2
> 
> However, if I run
> 
> git svn clone \
>  -T trunk \
>  -t tags \
>  -b branches \
>  https://flactag.svn.sourceforge.net/svnroot/flactag \
>  flactag-git1
> 
> I don't get any of the branches or tags, and I get numerous warnings like this:
> 
> W: -empty_dir: branches/d_pocock/autotools/HTTPFetch.cc
> W: -empty_dir: branches/d_pocock/autotools/HTTPFetch.h
> W: Cannot find common ancestor between f475b655c10b55b36df8376a396c0d9534418617 and 7e277bfd626a1caa84681e93a7af73e0160eb2eb. Ignoring merge info.
> W: Cannot find common ancestor between f475b655c10b55b36df8376a396c0d9534418617 and a82cd9bd2197a89303558af9aad43fa0c23d1ddb. Ignoring merge info.
> 
> I have the impression that git-svn does not like the repository layout, and it is not clear from the manual page how to make it work.  Can anyone share any advice about this?
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git-svn: cloning an SVN repo with sub-branches and sub-tags

[Ramkumar Ramachandra]

Hi Daniel,

Daniel Pocock wrote:
> git svn clone \
>  -T trunk \
>  -t tags \
>  -b branches \
>  https://flactag.svn.sourceforge.net/svnroot/flactag \
>  flactag-git1
>
> I don't get any of the branches or tags, and I get numerous warnings like
> this:
>
> W: -empty_dir: branches/d_pocock/autotools/HTTPFetch.cc
> W: -empty_dir: branches/d_pocock/autotools/HTTPFetch.h
> W: Cannot find common ancestor between
> f475b655c10b55b36df8376a396c0d9534418617 and
> 7e277bfd626a1caa84681e93a7af73e0160eb2eb. Ignoring merge info.
> W: Cannot find common ancestor between
> f475b655c10b55b36df8376a396c0d9534418617 and
> a82cd9bd2197a89303558af9aad43fa0c23d1ddb. Ignoring merge info.

These look like genuine warnings; git-svn probably doesn't like your
repository (nested branches/ tags, missing mergeinfo etc)

> I have the impression that git-svn does not like the repository layout, and
> it is not clear from the manual page how to make it work.  Can anyone share
> any advice about this?

Off the top of my head, you could try doing two things:
1. Try exporting the branches and tags by hand one-by-one.
2. You could try using reposurgeon [1] to fix the repository.

[1]: http://esr.ibiblio.org/?p=4071

    Ram

Re: git-svn: cloning an SVN repo with sub-branches and sub-tags

[dag]

Daniel Pocock <daniel@pocock.com.au> writes:

> branches/user1/workitem1
> branches/user1/workitem2
> branches/user2/workitem1
> branches/user2/workitem2
> git svn clone \
>   -T trunk \
>   -t tags \
>   -b branches \
>   https://flactag.svn.sourceforge.net/svnroot/flactag \
>   flactag-git1

I think you need to add --branches=branches/user1/workitem1
--branches=branches/user1/workitem1, etc.  Ditto for tags.

                               -Dave

Re: git-svn: cloning an SVN repo with sub-branches and sub-tags

[Stephen Bash]

----- Original Message -----
> From: "Daniel Pocock" <daniel@pocock.com.au>
> To: git@vger.kernel.org
> Sent: Thursday, March 29, 2012 9:47:13 AM
> Subject: git-svn: cloning an SVN repo with sub-branches and sub-tags
> 
> I'm trying to use git-svn to clone a repo that has sub-branches and
> sub-tags
> 
> The SVN layout is like this:
> 
> branches/user1/workitem1
> branches/user1/workitem2
> branches/user2/workitem1
> branches/user2/workitem2
> 
> tags/v1.0
> tags/v1.1
> tags/debian/v1.0-1
> tags/debian/v1.1-1
> tags/debian/v1.1-2
> 
> In git, I would like to have branches with the same names, e.g.
> 
> branches/user1/workitem1
> branches/user1/workitem2
> branches/user2/workitem1
> branches/user2/workitem2
> 
> and tags like this:
> 
> 1.0  (notice I don't want to keep the v1.0, drop the v)
> 1.1
> debian/1.0-1
> debian/1.1-1
> debian/1.1-2
> 
> However, if I run
> 
> git svn clone \
>    -T trunk \
>    -t tags \
>    -b branches \
>    https://flactag.svn.sourceforge.net/svnroot/flactag \
>    flactag-git1
> 
> I don't get any of the branches or tags, and I get numerous warnings
> like this:
> 
> W: -empty_dir: branches/d_pocock/autotools/HTTPFetch.cc
> W: -empty_dir: branches/d_pocock/autotools/HTTPFetch.h
> W: Cannot find common ancestor between
> f475b655c10b55b36df8376a396c0d9534418617 and
> 7e277bfd626a1caa84681e93a7af73e0160eb2eb. Ignoring merge info.
> W: Cannot find common ancestor between
> f475b655c10b55b36df8376a396c0d9534418617 and
> a82cd9bd2197a89303558af9aad43fa0c23d1ddb. Ignoring merge info.
> 
> I have the impression that git-svn does not like the repository
> layout and it is not clear from the manual page how to make it 
> work.  Can anyone share any advice about this?

Complicated layouts can require a special git-svn configuration.  See the configuration section at the end of the git-svn manpage:

  http://schacon.github.com/git/git-svn.html

My usual work flow is:

  git svn init /path/to/git/repo
  cd /path/to/git/repo
  edit .git/config
  git svn fetch

In the [svn-remote] section of Git config you can have multiple fetch, branches, and tag entries as long as each entry maps to a unique Git ref.

HTH,
Stephen

Re: git-svn: cloning an SVN repo with sub-branches and sub-tags

[Daniel Pocock]

On 29/03/2012 15:40, Dave Zarzycki wrote:
> You can make this work, but it is very fragile, doesn't scale well, and it only works on recent versions of git. For example, from the LLVM clang project:

...

> branches = cfe/branches/Apple/*:refs/remotes/svn/misc/Apple/*
> branches = cfe/branches/Apple/Necropolis/*:refs/remotes/svn/misc/AppleNecropolis/*


In the first line, (Apple/*), can it accidentally think that Necropolis 
is a branch, when it actually appears to be a directory containing branches?

Or does it consider all the lines in the file together?

Re: git-svn: cloning an SVN repo with sub-branches and sub-tags

[Daniel Pocock]

On 29/03/2012 16:32, Ramkumar Ramachandra wrote:
> Hi Daniel,
>
> Daniel Pocock wrote:
>> git svn clone \
>>   -T trunk \
>>   -t tags \
>>   -b branches \
>>   https://flactag.svn.sourceforge.net/svnroot/flactag \
>>   flactag-git1
>>
>> I don't get any of the branches or tags, and I get numerous warnings like
>> this:
>>
>> W: -empty_dir: branches/d_pocock/autotools/HTTPFetch.cc
>> W: -empty_dir: branches/d_pocock/autotools/HTTPFetch.h
>> W: Cannot find common ancestor between
>> f475b655c10b55b36df8376a396c0d9534418617 and
>> 7e277bfd626a1caa84681e93a7af73e0160eb2eb. Ignoring merge info.
>> W: Cannot find common ancestor between
>> f475b655c10b55b36df8376a396c0d9534418617 and
>> a82cd9bd2197a89303558af9aad43fa0c23d1ddb. Ignoring merge info.
>
> These look like genuine warnings; git-svn probably doesn't like your
> repository (nested branches/ tags, missing mergeinfo etc)
>
>> I have the impression that git-svn does not like the repository layout, and
>> it is not clear from the manual page how to make it work.  Can anyone share
>> any advice about this?
>
> Off the top of my head, you could try doing two things:
> 1. Try exporting the branches and tags by hand one-by-one.
> 2. You could try using reposurgeon [1] to fix the repository.
>
> [1]: http://esr.ibiblio.org/?p=4071


The flactag project is not something of a commercial nature, so we were 
really looking at quick solutions

One other factor I didn't mention in the original email, some of the 
branches have been renamed/moved within the SVN repo - is that a problem 
for git-svn?

One compromise I have thought about would be just extracting trunk, and 
forgetting about the branches, and manually tagging the official 
releases (as there were very few of them, this is quick)

However, based on the errors/warnings, is it possible that there is any 
corruption in the gitified version of trunk (or the history of trunk)?

Re: git-svn: cloning an SVN repo with sub-branches and sub-tags

[Ramkumar Ramachandra]

Hi again,

Daniel Pocock wrote:
> One other factor I didn't mention in the original email, some of the
> branches have been renamed/moved within the SVN repo - is that a problem for
> git-svn?

Yes, I think so.

> One compromise I have thought about would be just extracting trunk, and
> forgetting about the branches, and manually tagging the official releases
> (as there were very few of them, this is quick)

Just use contrib/svn-fe in that case: it's really quick and
lightweight.  You can then filter-branch the history and pull out
branches by hand.

> However, based on the errors/warnings, is it possible that there is any
> corruption in the gitified version of trunk (or the history of trunk)?

No corruption is possible on the Git side of things- it has
cryptographic hashes for everything.

    Ram

Re: git-svn: cloning an SVN repo with sub-branches and sub-tags

[Dave Zarzycki]

On Mar 30, 2012, at 5:16 AM, Daniel Pocock <daniel@pocock.com.au> wrote:

> On 29/03/2012 15:40, Dave Zarzycki wrote:
>> You can make this work, but it is very fragile, doesn't scale well, and it only works on recent versions of git. For example, from the LLVM clang project:
> 
> ...
> 
>> branches = cfe/branches/Apple/*:refs/remotes/svn/misc/Apple/*
>> branches = cfe/branches/Apple/Necropolis/*:refs/remotes/svn/misc/AppleNecropolis/*
> 
> 
> In the first line, (Apple/*), can it accidentally think that Necropolis is a branch, when it actually appears to be a directory containing branches?

Correct.

> Or does it consider all the lines in the file together?

It does not. That is why the "ignore-refs" setting is necessary. The "ignore-refs" setting tells git that "Necropolis" etc are not valid branches (because they are SVN subdirectories). The "ignore-refs" setting is useful for working around other SVN user errors too.

davez