git-svn problem with v1.6.5

git-svn problem with v1.6.5

[Pascal Obry]

Hello,

A git-svn mirror is created using Git v1.6.4. When I copy this mirror 
into another machine which is using v1.6.5 I get the following error:

$ git svn rebase
fatal: Invalid revision range 
d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk
rev-list --pretty=raw --no-color --reverse 
d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk --: 
command returned error: 128

Using v1.6.4 instead I have no problem. Any idea to track this problem down?

Thanks.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: git-svn problem with v1.6.5

[Avery Pennarun]

On Tue, Nov 10, 2009 at 5:23 PM, Pascal Obry <pascal@obry.net> wrote:
> $ git svn rebase
> fatal: Invalid revision range
> d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk
> rev-list --pretty=raw --no-color --reverse
> d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk --: command
> returned error: 128

Is d2cf08bb67e4b7da33a250127aab784f1f2f58d3 a valid revision?  (git
log d2cf08bb67e4b7da33a250127aab784f1f2f58d3).  Is
refs/remotes/svn/trunk valid?  (git log refs/remotes/svn/trunk).  It
seems like a strange problem.

> Using v1.6.4 instead I have no problem. Any idea to track this problem down?

You could try using git bisect to figure out which exact commit to
git.git created the problem.

Avery

Re: git-svn problem with v1.6.5

[Sam Vilain]

Pascal Obry wrote:
> Hello,
>
> A git-svn mirror is created using Git v1.6.4. When I copy this mirror 
> into another machine which is using v1.6.5 I get the following error:
>
> $ git svn rebase
> fatal: Invalid revision range 
> d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk
> rev-list --pretty=raw --no-color --reverse 
> d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk --: 
> command returned error: 128
>
> Using v1.6.4 instead I have no problem. Any idea to track this problem down?
>   

Did you copy the entire directory across using rsync/tar/etc or 'git
clone' ?   You'll need to make sure that the 'refs/remotes/*' refs look
exactly the same on the clone as they did on the original import repository.

Sam

Re: git-svn problem with v1.6.5

[Pascal Obry]

Sam,

> Did you copy the entire directory across using rsync/tar/etc or 'git
> clone' ?

I did a rsync of the .git directory so I expect all refs to be there.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: git-svn problem with v1.6.5

[Pascal Obry]

Avery,

 > Is d2cf08bb67e4b7da33a250127aab784f1f2f58d3 a valid revision?  (git
 > log d2cf08bb67e4b7da33a250127aab784f1f2f58d3).

No.

 > Is refs/remotes/svn/trunk valid?  (git log refs/remotes/svn/trunk).

Yes.

 > It seems like a strange problem.

Agreed!

> You could try using git bisect to figure out which exact commit to
> git.git created the problem.

I wanted to avoid that :) Anyway, here is the culprit:


commit 6f5748e14cc5bb0a836b649fb8e2d6a5eb166f1d
Author: Adam Brewster <adambrewster@gmail.com>
Date:   Tue Aug 11 23:14:27 2009 -0400

     svn: allow branches outside of refs/remotes

     It may be convenient for some users to store svn remote tracking
     branches outside of the refs/remotes/ heirarchy.

     To accomplish this feat, this patch includes the entire path to
     the ref in $r->{'refname'} in &read_all_remotes and tries to change
     references to this entry so the new value makes sense.

     [ew: fixed backwards compatibility, long lines]

     Signed-off-by: Adam Brewster <adambrewster@gmail.com>
     Signed-off-by: Eric Wong <normalperson@yhbt.net>

  git-svn.perl                     |   83 
++++++++++++++++++++++----------------
  t/lib-git-svn.sh                 |    2 +-
  t/t9104-git-svn-follow-parent.sh |   10 ++--
  t/t9107-git-svn-migrate.sh       |   14 +++----
  t/t9143-git-svn-gc.sh            |   10 ++--
  t/t9144-git-svn-old-rev_map.sh   |   31 ++++++++++++++
  6 files changed, 96 insertions(+), 54 deletions(-)


But frankly I'm no expert on Perl... I fear that I won't be able to 
debug that!

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: git-svn problem with v1.6.5

[Avery Pennarun]

On Wed, Nov 11, 2009 at 5:28 AM, Pascal Obry <pascal@obry.net> wrote:
>> Avery wrote:
>> Is d2cf08bb67e4b7da33a250127aab784f1f2f58d3 a valid revision?  (git
>> log d2cf08bb67e4b7da33a250127aab784f1f2f58d3).
>
> No.

Well, that's bad news.  Does "git log --all --parents | grep
d2cf08bb67e4b7da33a250127aab784f1f2f58d3" reveal any places that refer
to it?

It sounds a bit like your git-svn thinks something maps on to that
commit id, but a previous 'git gc' or something has thrown it away.
However, that doesn't explain why earlier git versions don't have this
problem.

>> You could try using git bisect to figure out which exact commit to
>> git.git created the problem.
>
> I wanted to avoid that :) Anyway, here is the culprit:
>
> commit 6f5748e14cc5bb0a836b649fb8e2d6a5eb166f1d
> Author: Adam Brewster <adambrewster@gmail.com>
> Date:   Tue Aug 11 23:14:27 2009 -0400
>
>   svn: allow branches outside of refs/remotes
[...]
>
> But frankly I'm no expert on Perl... I fear that I won't be able to debug
> that!

If you retrieve the latest version of git and then git revert the
above commit, does that fix the problem, at least?

Is it possible you have some weird branches outside of the
refs/remotes tree (either in .git itself, or in .git/refs/*) that you
forgot about, and which the new version of git-svn is finding somehow?

Avery

Re: git-svn problem with v1.6.5

[Pascal Obry]

Avery,

> Well, that's bad news.  Does "git log --all --parents | grep
> d2cf08bb67e4b7da33a250127aab784f1f2f58d3" reveal any places that refer
> to it?

No.

> It sounds a bit like your git-svn thinks something maps on to that
> commit id, but a previous 'git gc' or something has thrown it away.
> However, that doesn't explain why earlier git versions don't have this
> problem.

Maybe, but that would be a quite annoying bug!

> If you retrieve the latest version of git and then git revert the
> above commit, does that fix the problem, at least?

I cannot, this does not revert cleanly and I don't know how to properly 
resolve this conflict. I'm no Perl expert!

But reverting using the version just before fixes this problem.

> Is it possible you have some weird branches outside of the
> refs/remotes tree (either in .git itself, or in .git/refs/*) that you
> forgot about, and which the new version of git-svn is finding somehow?

I do not see something under .git/refs/* (only empty directories).

The project has been imported using something like this:

  $ git svn clone --prefix=svn/ svn+ssh://server/path \
	--revision=580:HEAD \
	--trunk=trunk/project \
	--tags=tags/project \
	--branches=branches/project \
	--branches="branches/global/*/project" project

Nothing fancy. The git-svn mirrors is updated every night using Git 
1.6.4 without problem.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: git-svn problem with v1.6.5

[Eric Wong]

Pascal Obry <pascal@obry.net> wrote:
> Avery,
>
>> Well, that's bad news.  Does "git log --all --parents | grep
>> d2cf08bb67e4b7da33a250127aab784f1f2f58d3" reveal any places that refer
>> to it?
>
> No.

Weird...

>> It sounds a bit like your git-svn thinks something maps on to that
>> commit id, but a previous 'git gc' or something has thrown it away.
>> However, that doesn't explain why earlier git versions don't have this
>> problem.
>
> Maybe, but that would be a quite annoying bug!

>> If you retrieve the latest version of git and then git revert the
>> above commit, does that fix the problem, at least?
>
> I cannot, this does not revert cleanly and I don't know how to properly  
> resolve this conflict. I'm no Perl expert!
>
> But reverting using the version just before fixes this problem.

That should rule out filesystem corruption...

>> Is it possible you have some weird branches outside of the
>> refs/remotes tree (either in .git itself, or in .git/refs/*) that you
>> forgot about, and which the new version of git-svn is finding somehow?
>
> I do not see something under .git/refs/* (only empty directories).
>
> The project has been imported using something like this:
>
>  $ git svn clone --prefix=svn/ svn+ssh://server/path \
> 	--revision=580:HEAD \
> 	--trunk=trunk/project \
> 	--tags=tags/project \
> 	--branches=branches/project \
> 	--branches="branches/global/*/project" project

It looks like you didn't use any of the weird[1] options that make it
unsafe to clobber the rev_maps.  Can you try removing all the .rev_map.*
files in $GIT_DIR/svn and running "git svn fetch" to rebuild them?

[1] --no-metadata, --use-svm-props
-- 
Eric Wong

Re: git-svn problem with v1.6.5

[Eric Wong]

Eric Wong <normalperson@yhbt.net> wrote:
> Pascal Obry <pascal@obry.net> wrote:
> > I do not see something under .git/refs/* (only empty directories).
> >
> > The project has been imported using something like this:
> >
> >  $ git svn clone --prefix=svn/ svn+ssh://server/path \
> > 	--revision=580:HEAD \
> > 	--trunk=trunk/project \
> > 	--tags=tags/project \
> > 	--branches=branches/project \
> > 	--branches="branches/global/*/project" project
> 
> It looks like you didn't use any of the weird[1] options that make it
> unsafe to clobber the rev_maps.  Can you try removing all the .rev_map.*
> files in $GIT_DIR/svn and running "git svn fetch" to rebuild them?
> 
> [1] --no-metadata, --use-svm-props

Also, any chance you have multiple refs with "trunk" in the basename?

 git rev-parse --symbolic --all | grep '/trunk'

It could be a backwards compatibility issue with git svn looking
in multiple places for trunk.

With Adam's commit, it'll try $GIT_DIR/svn/refs/remotes/trunk/* first
Then it'll try $GIT_DIR/svn/trunk/* as a fallback.

-- 
Eric Wong

Re: git-svn problem with v1.6.5

[Pascal Obry]

Eric,

> Also, any chance you have multiple refs with "trunk" in the basename?
>
>   git rev-parse --symbolic --all | grep '/trunk'

This reports:

$ git rev-parse --symbolic --all | grep '/trunk'
refs/remotes/svn/trunk

> It could be a backwards compatibility issue with git svn looking
> in multiple places for trunk.

But I have multiple trunk:

$ ls .git/svn/svn/trunk/
./  ../  .rev_map.936e1b1b-40f2-da11-902a-00137254ae57  unhandled.log

$ ls  .git/svn/refs/remotes/svn/trunk/
./  ../  .rev_map.936e1b1b-40f2-da11-902a-00137254ae57  unhandled.log

Removing all the .rev_map* fix the problem.

Removing only the one in .git/svn/refs/remotes/svn/trunk/ fix the 
problem too.

> With Adam's commit, it'll try $GIT_DIR/svn/refs/remotes/trunk/* first
> Then it'll try $GIT_DIR/svn/trunk/* as a fallback.

Does this means that it was looking first in $GIT_DIR/svn/trunk/* 
before? And the confusion come because now it is looking in 
$GIT_DIR/svn/refs/remotes/trunk/* first?

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: git-svn problem with v1.6.5

[Eric Wong]

Pascal Obry <pascal@obry.net> wrote:
> Eric,
>
>> Also, any chance you have multiple refs with "trunk" in the basename?
>>
>>   git rev-parse --symbolic --all | grep '/trunk'
>
> This reports:
>
> $ git rev-parse --symbolic --all | grep '/trunk'
> refs/remotes/svn/trunk

Pascal,

OK, this is normal.

>> It could be a backwards compatibility issue with git svn looking
>> in multiple places for trunk.
>
> But I have multiple trunk:
>
> $ ls .git/svn/svn/trunk/
> ./  ../  .rev_map.936e1b1b-40f2-da11-902a-00137254ae57  unhandled.log
>
> $ ls  .git/svn/refs/remotes/svn/trunk/
> ./  ../  .rev_map.936e1b1b-40f2-da11-902a-00137254ae57  unhandled.log
>
> Removing all the .rev_map* fix the problem.
>
> Removing only the one in .git/svn/refs/remotes/svn/trunk/ fix the  
> problem too.
>
>> With Adam's commit, it'll try $GIT_DIR/svn/refs/remotes/trunk/* first
>> Then it'll try $GIT_DIR/svn/trunk/* as a fallback.
>
> Does this means that it was looking first in $GIT_DIR/svn/trunk/*  
> before? And the confusion come because now it is looking in  
> $GIT_DIR/svn/refs/remotes/trunk/* first?

Yes, but somehow $GIT_DIR/svn/refs/remotes/trunk should not have
been created since $GIT_DIR/svn/svn/trunk already existed.  Both
of those directories existing at the same time should not

Did you try an early version of Adam's patch before it made it into
git.git by any chance?

Or, did you by any chance start a fresh import with a v1.6.5 and then
rsync $GIT_DIR to one created with 1.6.4 and not use --delete with
rsync?

-- 
Eric Wong

Re: git-svn problem with v1.6.5

[Pascal Obry]

Eric,

> Did you try an early version of Adam's patch before it made it into
> git.git by any chance?

No.

> Or, did you by any chance start a fresh import with a v1.6.5 and then
> rsync $GIT_DIR to one created with 1.6.4 and not use --delete with
> rsync?

No. The import has been done with 1.6.4. A filter-branch has been 
applied to fix author names. But AFAIK (will check) it has been done 
with 1.6.4 too.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: git-svn problem with v1.6.5

[Eric Wong]

Pascal Obry <pascal@obry.net> wrote:
> Eric,
>
>> Did you try an early version of Adam's patch before it made it into
>> git.git by any chance?
>
> No.
>
>> Or, did you by any chance start a fresh import with a v1.6.5 and then
>> rsync $GIT_DIR to one created with 1.6.4 and not use --delete with
>> rsync?
>
> No. The import has been done with 1.6.4. A filter-branch has been  
> applied to fix author names. But AFAIK (will check) it has been done  
> with 1.6.4 too.

Oh...  Any chance that lost commit you couldn't find was clobbered
by filter-branch?  That would explain a lot...

-- 
Eric Wong

Re: git-svn problem with v1.6.5

[Pascal Obry]

Eric,

> Oh...  Any chance that lost commit you couldn't find was clobbered
> by filter-branch?  That would explain a lot...

I still have the AWS repo as it was from initial import. And I can see
the commit
on this one:

$ git log --all --parents | grep d2cf08bb67e4b7da33a250127aab784f1f2f58d3
commit bd95b2b254286e6f04015b7f58266d66abc134a7
d2cf08bb67e4b7da33a250127aab784f1f2f58d3
commit d2cf08bb67e4b7da33a250127aab784f1f2f58d3
406eaf7e51229f60048ea80cdb6904e33ef59a81

So that's definitely a problem with the filter-branch. Thanks a lot
for your precious help!

Will review the procedure for filtering branches :(

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|              http://www.obry.net
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: git-svn problem with v1.6.5

[Pascal Obry]

Eric,

> Will review the procedure for filtering branches :(

A quick review does not show something obviously wrong on my side.

The commit on the rewritten repo has a different sha-1 which is
expected. So to me it seems that after a filter-branch on a git-svn
repo it is mandatory to recreate all .rev_map*? Is that right? Is so,
this is the missing piece in the procedure on my side.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|              http://www.obry.net
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: git-svn problem with v1.6.5

[Eric Wong]

Pascal Obry <pascal@obry.net> wrote:
> Eric,
> 
> > Will review the procedure for filtering branches :(
> 
> A quick review does not show something obviously wrong on my side.
> 
> The commit on the rewritten repo has a different sha-1 which is
> expected. So to me it seems that after a filter-branch on a git-svn
> repo it is mandatory to recreate all .rev_map*? Is that right? Is so,
> this is the missing piece in the procedure on my side.

Yes, you need to recreate all the .rev_map* files if you use
filter-branch.  Just remove them and they should be automatically
recreated for your case (not using --no-metadata or --use-svm-props)

-- 
Eric Wong

Re: git-svn problem with v1.6.5

[Pascal Obry]

Eric,

> Yes, you need to recreate all the .rev_map* files if you use
> filter-branch.  Just remove them and they should be automatically
> recreated for your case (not using --no-metadata or --use-svm-props)

Thanks for the confirmation, I'll do that.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|              http://www.obry.net
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595