Git / Subversion Interoperability

Git / Subversion Interoperability

[Bruno Cesar Ribas]

Hello,

I'm going to apply for the Git / Subversion Interoperability project.

I saw that there is no mentor yet assigned for the "job". So i'm sending this
mail to get some help to start the project by submiting to GSOC and begin to
work :)

My idea on this project is to create:
1.  git-svnserver
2.  write a backend for Subversion

And make it easy to work with SSH using those "common" flags in
authorized_keys like:
command="svnserve -t -r /home/svn --tunnel-user=bruno" ssh-dss bla bla

And as an idea i would like to make the same funcionality to git, as it is
not as easy today to do something like above :)

Well, that's it for now! And i'm waiting for replys

Regards,
Bruno

-- 
Bruno Ribas - ribas@c3sl.ufpr.br
http://web.inf.ufpr.br/ribas
C3SL: http://www.c3sl.ufpr.br 

Re: Git / Subversion Interoperability

[Shawn O. Pearce]

Bruno Cesar Ribas <ribas@c3sl.ufpr.br> wrote:
> I'm going to apply for the Git / Subversion Interoperability project.
> 
> I saw that there is no mentor yet assigned for the "job". So i'm sending this
> mail to get some help to start the project by submiting to GSOC and begin to
> work :)

I'll consider being a mentor for this project, but I know very
little of the SVN protocol or how its server behaves.  I also
don't really have the time to learn those nitty-gritty details
myself, nor do I have any burning desire to.
 
> My idea on this project is to create:
> 1.  git-svnserver
> 2.  write a backend for Subversion

These are two different approaches to the same problem.  I think
what was meant here was:

> 1.  git-svnserver

Here we create a new program that can be invoked via SSH that runs
the server-side of the SVN protocol.  Or we create a CGI program that
acts as the extended-WebDAV server for SVN.  Sam Vilain (mugwump
on #git) is suggesting using this approach as it probably will be
easier to debug.

> 2.  write a backend for Subversion

In this case we try to reuse the existing SVN server code by
creating a library module that plugs into that system and uses a
Git repository to store data, rather than say the existing bdb or
fsfs stores.  Git should be much faster than fsfs, use a lot less
disk space, and be just as atomic.

By using this approach you avoid the need to reimplement their
network protocol.  Which is a nice part of this approach.  But the
downside is you have to write code to run within their library and
address space, and that conforms to their storage API.


But either approach has a few key issues:

- Assigning repository-wide revision numbers.  Git doesn't have
such a concept, but its key to SVN.  These would need to be stored
in a file so the server can quickly map from revision number to
Git commit SHA-1.  The reflogs may help here, but currently they
also expire.  Any reflog that is being used to do this mapping
cannot be expired, ever.

- Branches.  In SVN these are in the repository wide namespace,
but in Git they aren't.  I imagine we'd want to just enforce the
standard layout that the SVN people recommened:

  /trunk/    --> refs/heads/master
  /branches/ --> refs/heads/ (minus master)
  /tags/     --> refs/tags/

That's all I can think of right now.  But I'm sure there are more.

> And make it easy to work with SSH using those "common" flags in
> authorized_keys like:
> command="svnserve -t -r /home/svn --tunnel-user=bruno" ssh-dss bla bla

Not following you...
 
> And as an idea i would like to make the same funcionality to git, as it is
> not as easy today to do something like above :)

Again, not following you...

-- 
Shawn.

Re: Git / Subversion Interoperability

[Julian Phillips]

On Thu, 22 Mar 2007, Shawn O. Pearce wrote:

> Bruno Cesar Ribas <ribas@c3sl.ufpr.br> wrote:
>> 2.  write a backend for Subversion
>
> In this case we try to reuse the existing SVN server code by
> creating a library module that plugs into that system and uses a
> Git repository to store data, rather than say the existing bdb or
> fsfs stores.  Git should be much faster than fsfs, use a lot less
> disk space, and be just as atomic.
>
> By using this approach you avoid the need to reimplement their
> network protocol.  Which is a nice part of this approach.  But the
> downside is you have to write code to run within their library and
> address space, and that conforms to their storage API.

This might run headlong into some of the issues facing the libification 
project - in particular the tendancy of git code to die as a primary error 
handling mechanism.  This approach may not viable without access to a 
libified git?

> But either approach has a few key issues:
>
> - Assigning repository-wide revision numbers.  Git doesn't have
> such a concept, but its key to SVN.  These would need to be stored
> in a file so the server can quickly map from revision number to
> Git commit SHA-1.  The reflogs may help here, but currently they
> also expire.  Any reflog that is being used to do this mapping
> cannot be expired, ever.

You could probably do something similar to the system used by 
git-svnimport to record the Subversion revision to git SHA1 mapping - 
perhaps with a hook script to update?

>
> - Branches.  In SVN these are in the repository wide namespace,
> but in Git they aren't.  I imagine we'd want to just enforce the
> standard layout that the SVN people recommened:
>
>  /trunk/    --> refs/heads/master
>  /branches/ --> refs/heads/ (minus master)
>  /tags/     --> refs/tags/

That would probably be good enough for the majority of 
one-project-per-repo Subversion projects at least.  Though there is still 
the issue that Subversion will actually let you create a "tag" simply by 
committing whatever you currently have in your working copy (including 
localally modified files ... yeuch).

>
> That's all I can think of right now.  But I'm sure there are more.

Properties are probably the next biggest headache.  Subversion allows you 
to associate arbitrary keyword value pairs with files (which are 
versioned) and with revisions (which are not versioned).  You would need 
to find someway to support this in git.  Since revision properties are 
disabled by default in Subversion you may be able to simply disallow them 
permanently - but I don't know anything about the Subversion protocol ...

-- 
Julian

  ---
Practical politics consists in ignoring facts.
 		-- Henry Adams

Re: Git / Subversion Interoperability

[Shawn O. Pearce]

Julian Phillips <julian@quantumfyre.co.uk> wrote:
> On Thu, 22 Mar 2007, Shawn O. Pearce wrote:
> >Bruno Cesar Ribas <ribas@c3sl.ufpr.br> wrote:
> >>2.  write a backend for Subversion
> >
> >In this case we try to reuse the existing SVN server code by
> >creating a library module that plugs into that system
...
> >downside is you have to write code to run within their library and
> >address space, and that conforms to their storage API.
> 
> This might run headlong into some of the issues facing the libification 
> project - in particular the tendancy of git code to die as a primary error 
> handling mechanism.  This approach may not viable without access to a 
> libified git?

Yes, exactly.  The libification won't be ready early enough for
this project.  So that does make it more difficult.  But this just
points out another reason why libification might be useful. ;-)

> >- Branches.  In SVN these are in the repository wide namespace,
> >but in Git they aren't.  I imagine we'd want to just enforce the
> >standard layout that the SVN people recommened:
...
> That would probably be good enough for the majority of 
> one-project-per-repo Subversion projects at least.  Though there is still 
> the issue that Subversion will actually let you create a "tag" simply by 
> committing whatever you currently have in your working copy (including 
> localally modified files ... yeuch).

Heh.  People do weird things.  ;-)
 
> >That's all I can think of right now.  But I'm sure there are more.
> 
> Properties are probably the next biggest headache.  Subversion allows you 
> to associate arbitrary keyword value pairs with files (which are 
> versioned) and with revisions (which are not versioned).  You would need 
> to find someway to support this in git.  Since revision properties are 
> disabled by default in Subversion you may be able to simply disallow them 
> permanently - but I don't know anything about the Subversion protocol ...

I thought about the properties, but didn't bother to write anything
on that subject as we may just be able to say "look, properties
are not supported in git-svnserver, so don't try to use them".

The git-svnserver is meant to help users migrate to Git, not to offer
up all of SVNs features and warts.  A big part of this project may
just be deciding what features we are willing to try to emulate.

-- 
Shawn.

Re: Git / Subversion Interoperability

[Julian Phillips]

On Thu, 22 Mar 2007, Shawn O. Pearce wrote:

> Julian Phillips <julian@quantumfyre.co.uk> wrote:
>> That would probably be good enough for the majority of
>> one-project-per-repo Subversion projects at least.  Though there is still
>> the issue that Subversion will actually let you create a "tag" simply by
>> committing whatever you currently have in your working copy (including
>> localally modified files ... yeuch).
>
> Heh.  People do weird things.  ;-)

Well, I don't think there would be too many complaints if a git-based 
Subversion server didn't support such methods of creating tags ... ;)

>
>>> That's all I can think of right now.  But I'm sure there are more.
>>
>> Properties are probably the next biggest headache.  Subversion allows you
>> to associate arbitrary keyword value pairs with files (which are
>> versioned) and with revisions (which are not versioned).  You would need
>> to find someway to support this in git.  Since revision properties are
>> disabled by default in Subversion you may be able to simply disallow them
>> permanently - but I don't know anything about the Subversion protocol ...
>
> I thought about the properties, but didn't bother to write anything
> on that subject as we may just be able to say "look, properties
> are not supported in git-svnserver, so don't try to use them".

Indeed.  That's what I was aiming at with the the last sentence (not very 
clearly unfortunately).  It would be nice if the server could tell the 
client that properties are not supported ... but I doubt that it can, 
since they are fundamental part of Subversion.

In particular it's how the client finds out about things like symlinks and 
line ending conversion.  It may be necessary to provide some basic support 
for some of the properties in the svn:... namespace in order for the 
Subversion repo access library not to refuse to talk to the git server.

>
> The git-svnserver is meant to help users migrate to Git, not to offer
> up all of SVNs features and warts.  A big part of this project may
> just be deciding what features we are willing to try to emulate.

Indeed.  It sounds like an interesting idea.  Anything that increases the 
possibilty of my convincing the appropriate people to migrate my work 
project ... ;)

-- 
Julian

  ---
"From there to here, from here to there, funny things are everywhere."
-- Dr. Seuss

Re: Git / Subversion Interoperability

[Karl Hasselström]

On 2007-03-23 01:36:11 +0000, Julian Phillips wrote:

> On Thu, 22 Mar 2007, Shawn O. Pearce wrote:
>
> > I thought about the properties, but didn't bother to write
> > anything on that subject as we may just be able to say "look,
> > properties are not supported in git-svnserver, so don't try to use
> > them".
>
> Indeed. That's what I was aiming at with the the last sentence (not
> very clearly unfortunately). It would be nice if the server could
> tell the client that properties are not supported ... but I doubt
> that it can, since they are fundamental part of Subversion.
>
> In particular it's how the client finds out about things like
> symlinks and line ending conversion. It may be necessary to provide
> some basic support for some of the properties in the svn:...
> namespace in order for the Subversion repo access library not to
> refuse to talk to the git server.

Maybe the pragmatic solution would be to have built-in handling of a
few properties such as svn:executable and svn:ignore that have git
equivalents, and just emulate all other properties with files.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Git / Subversion Interoperability

[Jakub Narebski]

Shawn O. Pearce wrote:

> - Assigning repository-wide revision numbers.  Git doesn't have
> such a concept, but its key to SVN.  These would need to be stored
> in a file so the server can quickly map from revision number to
> Git commit SHA-1.  The reflogs may help here, but currently they
> also expire.  Any reflog that is being used to do this mapping
> cannot be expired, ever.

Another idea is to use lightweight tags for that, especially now
with packed refs support in git.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: Git / Subversion Interoperability

[Bruno Cesar Ribas]

On Fri, Mar 23, 2007 at 11:34:26AM +0100, Karl Hasselström wrote:
> On 2007-03-23 01:36:11 +0000, Julian Phillips wrote:
> 
> > On Thu, 22 Mar 2007, Shawn O. Pearce wrote:
> >
> > > I thought about the properties, but didn't bother to write
> > > anything on that subject as we may just be able to say "look,
> > > properties are not supported in git-svnserver, so don't try to use
> > > them".
> >
> > Indeed. That's what I was aiming at with the the last sentence (not
> > very clearly unfortunately). It would be nice if the server could
> > tell the client that properties are not supported ... but I doubt
> > that it can, since they are fundamental part of Subversion.
> >
> > In particular it's how the client finds out about things like
> > symlinks and line ending conversion. It may be necessary to provide
> > some basic support for some of the properties in the svn:...
> > namespace in order for the Subversion repo access library not to
> > refuse to talk to the git server.
> 
> Maybe the pragmatic solution would be to have built-in handling of a
> few properties such as svn:executable and svn:ignore that have git
> equivalents, and just emulate all other properties with files.

My idea is to write the git-svnserver!!! I think it will be easier.

To begin coding, i plan to write basic functions [updade,commit,checkout,
copy,merge,...] then start to implement most "complex" instructions.

As spearce said before, the idea of this is to migrate from svn to git
without many trouble in transition, so basic can work as the initial thought.

> 
> -- 
> Karl Hasselström, kha@treskal.com
>       www.treskal.com/kalle

-- 
Bruno Ribas - ribas@c3sl.ufpr.br
http://web.inf.ufpr.br/ribas
C3SL: http://www.c3sl.ufpr.br 

Re: Git / Subversion Interoperability

[Karl Hasselström]

On 2007-03-23 12:21:02 -0300, Bruno Cesar Ribas wrote:

> On Fri, Mar 23, 2007 at 11:34:26AM +0100, Karl Hasselström wrote:
>
> > Maybe the pragmatic solution would be to have built-in handling of
> > a few properties such as svn:executable and svn:ignore that have
> > git equivalents, and just emulate all other properties with files.
>
> My idea is to write the git-svnserver!!! I think it will be easier.

Umm ... what I said was intended as a possible way to do the mapping
between svn and git concepts. I was under the impresson that
git-svnserver would need to use such a mapping.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Git / Subversion Interoperability

[Julian Phillips]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2637 bytes --]

On Fri, 23 Mar 2007, Bruno Cesar Ribas wrote:

> On Fri, Mar 23, 2007 at 11:34:26AM +0100, Karl Hasselström wrote:
>> On 2007-03-23 01:36:11 +0000, Julian Phillips wrote:
>>
>>> In particular it's how the client finds out about things like
>>> symlinks and line ending conversion. It may be necessary to provide
>>> some basic support for some of the properties in the svn:...
>>> namespace in order for the Subversion repo access library not to
>>> refuse to talk to the git server.
>>
>> Maybe the pragmatic solution would be to have built-in handling of a
>> few properties such as svn:executable and svn:ignore that have git
>> equivalents, and just emulate all other properties with files.
>
> My idea is to write the git-svnserver!!! I think it will be easier.

Almost certainly - I don't think anyone was proposing otherwise?  The repo 
access library I was referring to is one of the libraries used by the 
client - specifically the one used to talk to a remote server (I think 
it may be called ra or something?  I'm not familiar with Subversion 
internals).

>
> To begin coding, i plan to write basic functions [updade,commit,checkout,
> copy,merge,...] then start to implement most "complex" instructions.

Ok, but ...

I don't think that merge is a basic function in Subversion.  I believe 
that it is all client side using diffs provided by the server into the 
working copy, certainly it is up to the user to commit the result. 
Unfortunately that would make it necesary to do some very nasty work 
server side to try and autodetect merges - git-svnimport (and git-svn?) 
already does this sort of thing.  There are noises about this changing at 
some point though ...

>
> As spearce said before, the idea of this is to migrate from svn to git
> without many trouble in transition, so basic can work as the initial thought.

Which means that you have to at the very least support the basic 
operations of the Subversion supplied svn client.  And my point was that I 
wouldn't be surprised if this meant dealing with properties, since 
subversion uses properties to expose things like the log message, commit 
date and author for revisions and other misc. info for files.  You may not 
even be able to get commit/checkout working without _some_ property 
support.

Having said that, I don't think it would be a _bad_ thing if this work 
went far enough that one could replace a Subversion server without any 
users even noticing ;)

-- 
Julian

  ---
You know you're using the computer too much when:
a party sucks if theres no computers to use.
 	-- electrofreak

Re: Git / Subversion Interoperability

[Bruno Cesar Ribas]

On Fri, Mar 23, 2007 at 06:13:16PM +0000, Julian Phillips wrote:
> On Fri, 23 Mar 2007, Bruno Cesar Ribas wrote:
> 
> >On Fri, Mar 23, 2007 at 11:34:26AM +0100, Karl Hasselström wrote:
> >>On 2007-03-23 01:36:11 +0000, Julian Phillips wrote:
> >>
> >>>In particular it's how the client finds out about things like
> >>>symlinks and line ending conversion. It may be necessary to provide
> >>>some basic support for some of the properties in the svn:...
> >>>namespace in order for the Subversion repo access library not to
> >>>refuse to talk to the git server.
> >>
> >>Maybe the pragmatic solution would be to have built-in handling of a
> >>few properties such as svn:executable and svn:ignore that have git
> >>equivalents, and just emulate all other properties with files.
> >
> >My idea is to write the git-svnserver!!! I think it will be easier.
> 
> Almost certainly - I don't think anyone was proposing otherwise?  The repo 
> access library I was referring to is one of the libraries used by the 
> client - specifically the one used to talk to a remote server (I think 
> it may be called ra or something?  I'm not familiar with Subversion 
> internals).
> 
> >
> >To begin coding, i plan to write basic functions [updade,commit,checkout,
> >copy,merge,...] then start to implement most "complex" instructions.
> 
> Ok, but ...
> 
> I don't think that merge is a basic function in Subversion.  I believe 
> that it is all client side using diffs provided by the server into the 
> working copy, certainly it is up to the user to commit the result. 
> Unfortunately that would make it necesary to do some very nasty work 
> server side to try and autodetect merges - git-svnimport (and git-svn?) 
> already does this sort of thing.  There are noises about this changing at 
> some point though ...
> 
> >
> >As spearce said before, the idea of this is to migrate from svn to git
> >without many trouble in transition, so basic can work as the initial 
> >thought.
> 
> Which means that you have to at the very least support the basic 
> operations of the Subversion supplied svn client.  And my point was that I 
> wouldn't be surprised if this meant dealing with properties, since 
> subversion uses properties to expose things like the log message, commit 
> date and author for revisions and other misc. info for files.  You may not 
> even be able to get commit/checkout working without _some_ property 
> support.
> 
> Having said that, I don't think it would be a _bad_ thing if this work 
> went far enough that one could replace a Subversion server without any 
> users even noticing ;)

I began to think about it... the idea to have a git-svnserver is to move from
svn to git and get my dev team not to worry about the transition at the
start, but it is a good idea to make people get moving to git idea of
devlopment, right?
Or the main idea is that we will have a group devloping under git repo and a
group under svn-gateway for the same project? I don't see a point to have
this! when a repo type is defined everyone must begin to understand that way,
even if it was changed in the middle of the project.

When decided above, we can define our dev of this project. Because as people
make those 'strange' stuff at svn repo that git won't support we will have to
make some workarounds to get git understand those things, and if we have some
people migrated do git and some using svn they will not see some 'svn tags'
on git repo.

That's why i think it should be a transition thing from svn to git.



> 
> -- 
> Julian
> 
>  ---
> You know you're using the computer too much when:
> a party sucks if theres no computers to use.
> 	-- 
> 	electrofreak


-- 
Bruno Ribas - ribas@c3sl.ufpr.br
http://web.inf.ufpr.br/ribas
C3SL: http://www.c3sl.ufpr.br 

Re: Git / Subversion Interoperability

[Christian Wiese]

[-- Attachment #1: Type: text/plain, Size: 1266 bytes --]

Hello,

> > - Branches.  In SVN these are in the repository wide namespace,
> > but in Git they aren't.  I imagine we'd want to just enforce the
> > standard layout that the SVN people recommened:
> >
> >  /trunk/    --> refs/heads/master
> >  /branches/ --> refs/heads/ (minus master)
> >  /tags/     --> refs/tags/
> 
> That would probably be good enough for the majority of 
> one-project-per-repo Subversion projects at least.  Though there is
> still the issue that Subversion will actually let you create a "tag"
> simply by committing whatever you currently have in your working copy
> (including localally modified files ... yeuch).

If the proposed solution just supports "one-project-per-repo"
Subversion projects then you won't attract much projects which are
currently using subversion. I don't know many svn repos of this kind.

What you should also consider, is that a project might have splitted a
project within a repo into smaller subprojects.
I'm currently facing this issue while trying to migrate from svn to git
within my local working environment to gather experience while working
with git to be able to effectively "sell" git to the other people of
our project and provide a smooth migration path for them.

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Git / Subversion Interoperability

[Steven Grimm]

Christian Wiese wrote:
> I'm currently facing this issue while trying to migrate from svn to git
> within my local working environment to gather experience while working
> with git to be able to effectively "sell" git to the other people of
> our project and provide a smooth migration path for them.
>   

Would a git svn server be of interest in such a situation, though? 
That's exactly the situation I'm in, and git-svn is doing an admirable 
job of integrating the two environments without disrupting the folks who 
are happy with svn. It seems like the git svn server would be useful in 
the opposite environment, where you have a git project that people want 
to access with svn tools (e.g. because there's no git integration in an 
IDE or whatever.)

Maybe I'm just misunderstanding what the server in question would do, 
but it seems like it's not likely to be that useful in cases where you 
already have a real svn server running and holding the canonical copy of 
the project.

-Steve

Re: Git / Subversion Interoperability

[David Lang]

On Fri, 23 Mar 2007, Bruno Cesar Ribas wrote:

> On Fri, Mar 23, 2007 at 06:13:16PM +0000, Julian Phillips wrote:
>> On Fri, 23 Mar 2007, Bruno Cesar Ribas wrote:
>>
>> Having said that, I don't think it would be a _bad_ thing if this work
>> went far enough that one could replace a Subversion server without any
>> users even noticing ;)
>
> I began to think about it... the idea to have a git-svnserver is to move from
> svn to git and get my dev team not to worry about the transition at the
> start, but it is a good idea to make people get moving to git idea of
> devlopment, right?
> Or the main idea is that we will have a group devloping under git repo and a
> group under svn-gateway for the same project? I don't see a point to have
> this! when a repo type is defined everyone must begin to understand that way,
> even if it was changed in the middle of the project.
>
> When decided above, we can define our dev of this project. Because as people
> make those 'strange' stuff at svn repo that git won't support we will have to
> make some workarounds to get git understand those things, and if we have some
> people migrated do git and some using svn they will not see some 'svn tags'
> on git repo.
>
> That's why i think it should be a transition thing from svn to git.

if you just want to transition everyone on a project from svn to git then you 
don't need a git-svn server, you just need a translation tool

what a git-svn server gives you is the ability to have some people use svn 
(becouse 'git is too complicated') while others use git.

think about sourceforge as an example. they only want to provide one version 
control tool, they started with cvs and are migrating to svn, but if they could 
put in a git server and git-svn server and support both.

think of it as 'one version control server to rule them all' (and then encourage 
people to do git-hg-server, git-arch-server, git-darc-server, etc)

as far as I can tell (with the exception of recording explicit renames) the git 
functionality is a strict superset of all the other version control tools, so it 
should be possible to emulate them with git. svn is just the biggest 
bang-for-the-buck in terms of number of users.

David Lang

Re: Git / Subversion Interoperability

[Daniel Barkalow]

On Fri, 23 Mar 2007, Bruno Cesar Ribas wrote:

> I began to think about it... the idea to have a git-svnserver is to move from
> svn to git and get my dev team not to worry about the transition at the
> start, but it is a good idea to make people get moving to git idea of
> devlopment, right?
> Or the main idea is that we will have a group devloping under git repo and a
> group under svn-gateway for the same project? I don't see a point to have
> this! when a repo type is defined everyone must begin to understand that way,
> even if it was changed in the middle of the project.

It's been useful in the past (when Wine went from CVS to git) to make it 
unnecessary for all developers to switch which software they're using at 
the same time. With that particular project, it really went from having a 
CVS repository and a git gateway to having a git repository and a CVS 
gateway without users seeing a difference (except that the CVS access was 
suddenly faster).

Then, of course, the git users were able to start making use of the more 
flexible representation of history, since the official repository then 
could represent it.

	-Daniel
*This .sig left intentionally blank*

Re: Git / Subversion Interoperability

[Shawn O. Pearce]

Karl Hasselstr??m <kha@treskal.com> wrote:
> On 2007-03-23 01:36:11 +0000, Julian Phillips wrote:
> > Indeed. That's what I was aiming at with the the last sentence (not
> > very clearly unfortunately). It would be nice if the server could
> > tell the client that properties are not supported ... but I doubt
> > that it can, since they are fundamental part of Subversion.
> >
> > In particular it's how the client finds out about things like
> > symlinks and line ending conversion. It may be necessary to provide
> > some basic support for some of the properties in the svn:...
> > namespace in order for the Subversion repo access library not to
> > refuse to talk to the git server.
> 
> Maybe the pragmatic solution would be to have built-in handling of a
> few properties such as svn:executable and svn:ignore that have git
> equivalents, and just emulate all other properties with files.

That is not a bad idea at all.  We could easily do

  svn:executable -> (mode & 0111);
  svn:ignore -> .gitignore;

I'm sure there are others, but they are the only two Karl mentioned,
and that I know of off the top of my head.  ;-)

Additional random properties could just be written out to .gitsvnprop
in the parent directory.

Here's another SVN feature that we might have to actually emulate:
locks.  Apparently clients can "lock" a file, preventing others
from editing that same file.  This is good for say artwork where
the file format is a binary thing that just cannot be merged.
Some users may actually be using SVN for exactly this reason.
Replacing it for some users may require that feature.

Heck, the F!@&@#*!@&@!*! POS (not SVN) that I have to use at my day
job can *only* work in this mode (exclusive lock file) and people
there still don't know how two different developers can safely edit
the same Java file at the same time without that exclusive lock
system in place.  They are absolutely terrified of auto-merges like
what Git does.  ;-)

Then they same users go off and incorrectly hand-merge stuff anyway.
Stuff that xdiff's 3-way file merge would have done correctly
without any problems.  Argh!

-- 
Shawn.

Re: Git / Subversion Interoperability

[Shawn O. Pearce]

Jakub Narebski <jnareb@gmail.com> wrote:
> Shawn O. Pearce wrote:
> 
> > - Assigning repository-wide revision numbers.  Git doesn't have
> > such a concept, but its key to SVN.  These would need to be stored
> > in a file so the server can quickly map from revision number to
> > Git commit SHA-1.  The reflogs may help here, but currently they
> > also expire.  Any reflog that is being used to do this mapping
> > cannot be expired, ever.
> 
> Another idea is to use lightweight tags for that, especially now
> with packed refs support in git.

Yes, not a bad idea.

Except the packed refs file can have variable sized records.  It has
no faster access path than to just scan it.  Some SVN repositories
have revision counts up into the 5 and 6 digits.  These would take
quite a while to scan, even as packed refs.  ;-)

I'm inclined to lean towards a really simple binary flat file holding
just 20 byte SHA-1s, in "SVN order".  Then its just a simple array
index operation to locate the correct Git commit.  And adding a
new commit is really just an append to the end operation.

Yea, I know, append == bad on some older Linux systems, and is
hard to roll back if you partially appended as it gets into some
possible nasty mess with truncate, but I think that when combined
with a simple lock file we can make it safe enough that its not
really going to be a problem.

-- 
Shawn.

Re: Git / Subversion Interoperability

[Shawn O. Pearce]

Steven Grimm <koreth@midwinter.com> wrote:
> Christian Wiese wrote:
> >I'm currently facing this issue while trying to migrate from svn to git
> >within my local working environment to gather experience while working
> >with git to be able to effectively "sell" git to the other people of
> >our project and provide a smooth migration path for them.
> >  
> 
> Would a git svn server be of interest in such a situation, though? 
> That's exactly the situation I'm in, and git-svn is doing an admirable 
> job of integrating the two environments without disrupting the folks who 
> are happy with svn. It seems like the git svn server would be useful in 
> the opposite environment, where you have a git project that people want 
> to access with svn tools (e.g. because there's no git integration in an 
> IDE or whatever.)
> 
> Maybe I'm just misunderstanding what the server in question would do, 
> but it seems like it's not likely to be that useful in cases where you 
> already have a real svn server running and holding the canonical copy of 
> the project.

Well, it does make it harder to transparently remove the SVN server
and drop in a Git based one.  ;-)

But I think Steven is right; like the git-cvsserver you can't do
everything with git-svnserver that you could have done with SVN or
with Git.  But you can get basic access to the Git repository and
still work with it through SVN based tools.

If that means that we export things according to the suggested
SVN repository layout, so be it.  Its difficult to argue with
us when we've followed the SVN advice.  ;-)

Christian pointed out that some projects may have different "slightly
related" modules stored in the same repository, and have come to
expect this to work in certain ways.  Moving to Git is difficult
for them as git-svnimport and git-svn can have a difficult time
divining the correct layout from the soup that is available in SVN.

But its also hard for Git to export soup back.  That is, even once
we've successfully decoded the soup we don't really have good
submodule support, so we cannot (easily) take two different Git
repositories (one per "slightly related" module) and export them
under a single SVN name/revision-space.

-- 
Shawn.

Re: Git / Subversion Interoperability

[Karl Hasselström]

On 2007-03-24 02:41:30 -0400, Shawn O. Pearce wrote:

> That is not a bad idea at all.  We could easily do
>
>   svn:executable -> (mode & 0111);
>   svn:ignore -> .gitignore;
>
> I'm sure there are others, but they are the only two Karl mentioned,
> and that I know of off the top of my head. ;-)

These are the two that git-svnimport handles. I don't think it knows
about any other subversion properties.

There is a slight problem with directly converting between svn:ignore
and .gitignore, though: the semantics are only almost the same.
Specifically, git's ignore patters are recursive (applies to
subdirectories) while subversion's are not. But it's not a big
problem.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: Git / Subversion Interoperability

[Linus Torvalds]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 986 bytes --]



On Sat, 24 Mar 2007, Karl Hasselström wrote:
>
> There is a slight problem with directly converting between svn:ignore
> and .gitignore, though: the semantics are only almost the same.
> Specifically, git's ignore patters are recursive (applies to
> subdirectories) while subversion's are not. But it's not a big
> problem.

You should be able to use an "absolute" path-name to ignore something in 
.gitignore, in which case it is not used recursively.

So if your .gitignore looks like

	*.o

it means that it recursively ignores all *.o files starting at that level.

HOWEVER, if you write it as

	/*.o

it means that it ignores *.o files only *within* that level (so it's 
"absolute" wrt the particular .gitignore file, not globally).

So you can have both behaviours.

[ I think the exact behaviour is: if there is a '/' anywhere in the name, 
  it's not a recursive match and has to match the file exactly, but 
  somebody like Junio should probably back me up on that ]

		Linus

Re: Git / Subversion Interoperability

[Eric Wong]

Bruno Cesar Ribas <ribas@c3sl.ufpr.br> wrote:
> Hello,
> 
> I'm going to apply for the Git / Subversion Interoperability project.
> 
> I saw that there is no mentor yet assigned for the "job". So i'm sending this
> mail to get some help to start the project by submiting to GSOC and begin to
> work :)

I work on git-svn and I can certainly help you in some ways.  However
I'm uncomfortable committing myself to this because of time constraints;
so I won't be signing up as a mentor.  Sorry.

> My idea on this project is to create:
> 1.  git-svnserver

I'm not familiar with the internals of the SVN protocol, and my work
with git-svn has sufficiently frightened me away from wanting to
investigate further :).

I also suggest you write a HTTP/DAV-based server/plugin/cgi instead of
relying on the SVN protocol.  In my experience svnserve itself is quite
finnicky.

> 2.  write a backend for Subversion

One key difference between git and Subversion is the design emphasis on
data structures versus design emphasis on code/APIs.  Subversion
certainly tries to keep their APIs stable (and for the most part seem
successful), but their publically available storage backends are
definitely lacking.

I'm not sure this part of the project is going to help git very much
unless it speeds up the libification effort.

> And make it easy to work with SSH using those "common" flags in
> authorized_keys like:
> command="svnserve -t -r /home/svn --tunnel-user=bruno" ssh-dss bla bla
> 
> And as an idea i would like to make the same funcionality to git, as it is
> not as easy today to do something like above :)

One reason to work on your project is to help SVN users who have no idea
what UNIX, ssh, and tunneling are.  This is also why I recommend working
on the HTTP/DAV protocol instead.

-- 
Eric Wong

Re: Git / Subversion Interoperability

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> So if your .gitignore looks like
>
> 	*.o
>
> it means that it recursively ignores all *.o files starting at that level.
>
> HOWEVER, if you write it as
>
> 	/*.o
>
> it means that it ignores *.o files only *within* that level (so it's 
> "absolute" wrt the particular .gitignore file, not globally).
>
> So you can have both behaviours.
>
> [ I think the exact behaviour is: if there is a '/' anywhere in the name, 
>   it's not a recursive match and has to match the file exactly, but 
>   somebody like Junio should probably back me up on that ]
> 		Linus

Yes, that is what dir.c::excluded_1() says.  If it does not have
'/' then fnmatch the basename, otherwise fnmatch with
FNM_PATHNAME.  So I think if you write

               /obj/*.o

you can ignore *.o files in obj/ directory (but won't ignore a/obj/b.o).

Re: Git / Subversion Interoperability

[Eric Wong]

"Shawn O. Pearce" <spearce@spearce.org> wrote:
> Jakub Narebski <jnareb@gmail.com> wrote:
> > Shawn O. Pearce wrote:
> > 
> > > - Assigning repository-wide revision numbers.  Git doesn't have
> > > such a concept, but its key to SVN.  These would need to be stored
> > > in a file so the server can quickly map from revision number to
> > > Git commit SHA-1.  The reflogs may help here, but currently they
> > > also expire.  Any reflog that is being used to do this mapping
> > > cannot be expired, ever.
> > 
> > Another idea is to use lightweight tags for that, especially now
> > with packed refs support in git.
> 
> Yes, not a bad idea.
> 
> Except the packed refs file can have variable sized records.  It has
> no faster access path than to just scan it.  Some SVN repositories
> have revision counts up into the 5 and 6 digits.  These would take
> quite a while to scan, even as packed refs.  ;-)
> 
> I'm inclined to lean towards a really simple binary flat file holding
> just 20 byte SHA-1s, in "SVN order".  Then its just a simple array
> index operation to locate the correct Git commit.  And adding a
> new commit is really just an append to the end operation.

The rev_db implementation in git-svn is very similar to what you're
describing.  However, I decided on 41-byte (SHA1 ascii + "\n") records
since it was easier to debug if any problems came up, and a ~50% size
reduction wasn't worth the potential headache (I already had a lot of
those from just dealing with SVN :).

The biggest weakness of fixed records is that tags or sparsely worked on
branches waste a lot of disk space, but it hasn't been a huge problem
for me.

-- 
Eric Wong

Re: Git / Subversion Interoperability

[Sam Vilain]

Shawn O. Pearce wrote:
> But its also hard for Git to export soup back.  That is, even once
> we've successfully decoded the soup we don't really have good
> submodule support, so we cannot (easily) take two different Git
> repositories (one per "slightly related" module) and export them
> under a single SVN name/revision-space.
>   

Sure you can: subprojects

Sam.

Re: Git / Subversion Interoperability

[Sam Vilain]

Shawn O. Pearce wrote:
> That is not a bad idea at all.  We could easily do
>
>   svn:executable -> (mode & 0111);
>   svn:ignore -> .gitignore;
>
> I'm sure there are others, but they are the only two Karl mentioned,
> and that I know of off the top of my head.  ;-)
>
> Additional random properties could just be written out to .gitsvnprop
> in the parent directory.
>   

There are also the important properties "svk:merge" and "svnmerge",
which allow you to notate when a revision has an extra parent.  SVK
and/or svnmerge should then be able to use the extra merge information
without any extra work.  Additionally, people might want to push back
with these properties.

Sam.