Change in cvsps maintainership, abd a --fast-export option

Change in cvsps maintainership, abd a --fast-export option

[Eric S. Raymond]

Earlier today David Mansfield handed off to me the cvsps project. This
is the code used as an engine for reading CVS repositories by
git-cvsimport.

His reason (aside from general overwork and no longer having a strong
interest on the code) is that I have added a --fast-export option to
cvsps-3.0 that emits a git fast-import stream representing the CVS
history.
 
I did this so that reposurgeon could use cvsps as a front end.  But I
expect it will be of some interest to whoever is maintaining
git-cvsimport. That code can now become much, *much* smaller and
simpler.

The new --fast-export mode solves at least one bug mentioned on the
git-cvsimport man page; multiple tags pointing at the same CVS changeset
will be passed through correctly.

Possibly it fixes some other problems described there as well.  
I don't understand all the bug warnings on that page and would like to
discuss them with the author, whoever that is.  Possibly cvsps can be
further enhanced to address these problems; I'm willing to work on that.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

To stay young requires the unceasing cultivation of the ability to
unlearn old falsehoods.
	-- Lazarus Long 

Re: Change in cvsps maintainership, abd a --fast-export option

[Michael Haggerty]

On 12/20/2012 10:56 PM, Eric S. Raymond wrote:
> Earlier today David Mansfield handed off to me the cvsps project. This
> is the code used as an engine for reading CVS repositories by
> git-cvsimport.
> [...] I have added a --fast-export option to
> cvsps-3.0 that emits a git fast-import stream representing the CVS
> history.
> [...]
> Possibly it fixes some other problems described there as well.  
> I don't understand all the bug warnings on that page and would like to
> discuss them with the author, whoever that is.  Possibly cvsps can be
> further enhanced to address these problems; I'm willing to work on that.

In 2009 I added tests demonstrating some of the erroneous behavior of
git-cvsimport.  The failing tests in t9601-t9603 are concrete examples
of the problems mentioned in the manpage.

If you haven't yet seen it, there is a writeup of the algorithm used by
cvs2git to infer the history of a CVS repository [1].  If your goal is
to make cvsps more robust, you might want to consider the ideas
described there.

Michael

[1] File doc/design-notes.txt in the cvs2svn source tree, also visible here:


http://cvs2svn.tigris.org/source/browse/*checkout*/cvs2svn/trunk/doc/design-notes.txt

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: Change in cvsps maintainership, abd a --fast-export option

[Eric S. Raymond]

Michael Haggerty <mhagger@alum.mit.edu>:
> In 2009 I added tests demonstrating some of the erroneous behavior of
> git-cvsimport.  The failing tests in t9601-t9603 are concrete examples
> of the problems mentioned in the manpage.

Thanks, that will be extremely useful. One of the things I'm putting effort
into is building a good test suite for the tool; I may well be able to adapt
your tests directly.
>
> If you haven't yet seen it, there is a writeup of the algorithm used by
> cvs2git to infer the history of a CVS repository [1].  If your goal is
> to make cvsps more robust, you might want to consider the ideas
> described there.

I shall do so.  Their design ideas may well be interesting, even though I
don't trust their code.  I've seem cvs2svn drop far too many weird artifacts 
and just plain broken commits in the back history of Subversion repositories.
I don't know if this is due to design problems, implementation bugs, or both.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Change in cvsps maintainership, abd a --fast-export option

[Michael Haggerty]

On 12/21/2012 11:44 AM, Eric S. Raymond wrote:
> Michael Haggerty <mhagger@alum.mit.edu>:
>> If you haven't yet seen it, there is a writeup of the algorithm used by
>> cvs2git to infer the history of a CVS repository [1].  If your goal is
>> to make cvsps more robust, you might want to consider the ideas
>> described there.
> 
> I shall do so.  Their design ideas may well be interesting, even though I
> don't trust their code.  I've seem cvs2svn drop far too many weird artifacts 
> and just plain broken commits in the back history of Subversion repositories.
> I don't know if this is due to design problems, implementation bugs, or both.

If you have seen any problems with cvs2svn/cvs2git, please file bug
reports.  In the past years there have been very few valid bugs
reported.  We very often find that artifacts that users thought were
bugs are in fact intentional workarounds required to make the contents
of branches and tags in the target VCS agree with those in the CVS
repository.

Perhaps your experience is with an older version of cvs2svn?  If not,
please be specific rather than just making complaints that are too vague
to be rebutted or fixed (whichever is appropriate).  I put a *lot* of
effort into getting cvs2svn to run correctly, and I take bug reports
very seriously.

Michael
(the cvs2svn maintainer)

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: Change in cvsps maintainership, abd a --fast-export option

[Eric S. Raymond]

Michael Haggerty <mhagger@alum.mit.edu>:
> Perhaps your experience is with an older version of cvs2svn? 

Well, it has been at least four years since I ran it on anything.
Maybe that counts as old. 

I'm willing to believe it's working better now, but I've had to deal
with geological strata of nastiness older versions produced in various
Subversion repositories (cleaning up that crud was a major motivation
for reposurgeon) and I'm fairly sure I haven't seen the last such
fossils.  Every sufficiently old Subversion repository seems to have
few.

>                                                             If not,
> please be specific rather than just making complaints that are too vague
> to be rebutted or fixed (whichever is appropriate).  I put a *lot* of
> effort into getting cvs2svn to run correctly, and I take bug reports
> very seriously.

I can't be specific now, but that may change shortly.  I'm putting
together a test suite for cvsps with the specific intention of
capturing as many odd corner cases as I can.  (I just finished writing
a small Python framework for expressing interleaved CVS command
sequences on multiple checkouts in a way that can be easily run.)

It wouldn't be difficult for me to test whether these break cvs2svn. 
You've established that someone over there is paying attention, so
I'll do that, I guess.

I'm willing to share my test suite as well.  Do you have your own zoo
of odd cases I could test on?
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Change in cvsps maintainership, abd a --fast-export option

[Michael Haggerty]

On 12/21/2012 10:43 PM, Eric S. Raymond wrote:
> Michael Haggerty <mhagger@alum.mit.edu>:
>> Perhaps your experience is with an older version of cvs2svn? 
> 
> Well, it has been at least four years since I ran it on anything.
> Maybe that counts as old. 

cvs2svn version 2.0 (Aug 2007) totally changed how cvs2svn deduces
changesets.  Any results from before that are utterly irrelevant to
modern times.

The changes between version 2.0 and version 2.4 (Sep 2012) have done a
lot more to improve the quality of the conversion.  The state of the art
is far beyond what it was four years ago.

>>                                                             If not,
>> please be specific rather than just making complaints that are too vague
>> to be rebutted or fixed (whichever is appropriate).  I put a *lot* of
>> effort into getting cvs2svn to run correctly, and I take bug reports
>> very seriously.
> 
> I can't be specific now, but that may change shortly.  I'm putting
> together a test suite for cvsps with the specific intention of
> capturing as many odd corner cases as I can.  (I just finished writing
> a small Python framework for expressing interleaved CVS command
> sequences on multiple checkouts in a way that can be easily run.)
> 
> It wouldn't be difficult for me to test whether these break cvs2svn. 
> You've established that someone over there is paying attention, so
> I'll do that, I guess.

I look forward to your results (whether positive or negative).

> I'm willing to share my test suite as well.  Do you have your own zoo
> of odd cases I could test on?

Yes, we have a rather extensive test suite that includes lots of CVS
perversions.  See run-tests.py and the CVS repositories under test-data.
 Unfortunately, the tests mostly cover only 2svn conversions, but since
most of the tricky work is in the common code that infers the CVS
history, cvs2git benefits from the tests too.

This discussion is getting pretty cvs2svn-specific, so we should
probably move it over to the cvs2svn-dev mailing list.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: Re: Change in cvsps maintainership, abd a --fast-export option

[Heiko Voigt]

Hi,

On Fri, Dec 21, 2012 at 05:44:37AM -0500, Eric S. Raymond wrote:
> Michael Haggerty <mhagger@alum.mit.edu>:
> > In 2009 I added tests demonstrating some of the erroneous behavior of
> > git-cvsimport.  The failing tests in t9601-t9603 are concrete examples
> > of the problems mentioned in the manpage.
> 
> Thanks, that will be extremely useful. One of the things I'm putting effort
> into is building a good test suite for the tool; I may well be able to adapt
> your tests directly.

Back then when I was converting some repositories to git and I also
wrote a quick testsuite for cvsps in an attempt to fix the bugs but gave
up. That was the point when I wrote about cvsimports limitations in the
documentation.

My commits can be found here:

	http://repo.or.cz/w/cvsps-hv.git

I just quickly checked and it seems that it does not run cleanly on a
modern Linux anymore. If it is of interest to you I can try to get it
running again.

Cheers Heiko

Re: Re: Change in cvsps maintainership, abd a --fast-export option

[Eric S. Raymond]

Heiko Voigt <hvoigt@hvoigt.net>:
> Back then when I was converting some repositories to git and I also
> wrote a quick testsuite for cvsps in an attempt to fix the bugs but gave
> up. That was the point when I wrote about cvsimports limitations in the
> documentation.
> 
> My commits can be found here:
> 
> 	http://repo.or.cz/w/cvsps-hv.git
> 
> I just quickly checked and it seems that it does not run cleanly on a
> modern Linux anymore. If it is of interest to you I can try to get it
> running again.

That would be helpful.  Please give it some effort.

I'm going to work on merging the cvsps patches Yann Dirson has been 
accumulating.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Re: Re: Change in cvsps maintainership, abd a --fast-export option

[Heiko Voigt]

Hi,

On Sat, Dec 22, 2012 at 01:21:18AM -0500, Eric S. Raymond wrote:
> Heiko Voigt <hvoigt@hvoigt.net>:
> > Back then when I was converting some repositories to git and I also
> > wrote a quick testsuite for cvsps in an attempt to fix the bugs but gave
> > up. That was the point when I wrote about cvsimports limitations in the
> > documentation.
> > 
> > My commits can be found here:
> > 
> > 	http://repo.or.cz/w/cvsps-hv.git
> > 
> > I just quickly checked and it seems that it does not run cleanly on a
> > modern Linux anymore. If it is of interest to you I can try to get it
> > running again.
> 
> That would be helpful.  Please give it some effort.

Here you go. I have pushed my changes on the master branch there.

You should now be able to run my tests with

	make test

from the root directory of the repository. The expected and actual
output can be found in the t[0-9]{4}... directories underneath t/.

Cheers Heiko

Re: Re: Change in cvsps maintainership, abd a --fast-export option

[Eric S. Raymond]

Heiko Voigt <hvoigt@hvoigt.net>:
> My commits can be found here:
> 
> 	http://repo.or.cz/w/cvsps-hv.git
> 
> I just quickly checked and it seems that it does not run cleanly on a
> modern Linux anymore. If it is of interest to you I can try to get it
> running again.

esr@snark:~/WWW/cvsps/fixrepos$ git clone http://repo.or.cz/w/cvsps-hv.git
Cloning into 'cvsps-hv'...
fatal: http://repo.or.cz/w/cvsps-hv.git/info/refs not valid: is this a git repository?

Doesn't seem to be a good day for cloning - I can't get Yann's repo either,
something about HEAD pointing to an invalid reference.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Re: Change in cvsps maintainership, abd a --fast-export option

[Antoine Pelisse]

> esr@snark:~/WWW/cvsps/fixrepos$ git clone http://repo.or.cz/w/cvsps-hv.git
> Cloning into 'cvsps-hv'...
> fatal: http://repo.or.cz/w/cvsps-hv.git/info/refs not valid: is this a git repository?

I guess 'w' means write, and you don't have write access. You should use

    http://repo.or.cz/r/cvsps-hv.git

instead. It works for me.

Re: Re: Re: Change in cvsps maintainership, abd a --fast-export option

[Eric S. Raymond]

Heiko Voigt <hvoigt@hvoigt.net>:
> Hi,
> 
> On Sat, Dec 22, 2012 at 01:21:18AM -0500, Eric S. Raymond wrote:
> > Heiko Voigt <hvoigt@hvoigt.net>:
> > > Back then when I was converting some repositories to git and I also
> > > wrote a quick testsuite for cvsps in an attempt to fix the bugs but gave
> > > up. That was the point when I wrote about cvsimports limitations in the
> > > documentation.
> > > 
> > > My commits can be found here:
> > > 
> > > 	http://repo.or.cz/w/cvsps-hv.git
> > > 
> > > I just quickly checked and it seems that it does not run cleanly on a
> > > modern Linux anymore. If it is of interest to you I can try to get it
> > > running again.
> > 
> > That would be helpful.  Please give it some effort.
> 
> Here you go. I have pushed my changes on the master branch there.
> 
> You should now be able to run my tests with
> 
> 	make test
> 
> from the root directory of the repository. The expected and actual
> output can be found in the t[0-9]{4}... directories underneath t/.
> 
> Cheers Heiko

sr@snark:~/WWW/cvsps/fixrepos$ git clone http://repo.or.cz/w/cvsps-hv.git
Cloning into 'cvsps-hv'...
fatal: http://repo.or.cz/w/cvsps-hv.git/info/refs not valid: is this a git repository?

-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Re: Change in cvsps maintainership, abd a --fast-export option

[Eric S. Raymond]

Antoine Pelisse <apelisse@gmail.com>:
> > esr@snark:~/WWW/cvsps/fixrepos$ git clone http://repo.or.cz/w/cvsps-hv.git
> > Cloning into 'cvsps-hv'...
> > fatal: http://repo.or.cz/w/cvsps-hv.git/info/refs not valid: is this a git repository?
> 
> I guess 'w' means write, and you don't have write access. You should use
> 
>     http://repo.or.cz/r/cvsps-hv.git
> 
> instead. It works for me.

OK, that got it.  Looking at the tests now.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Change in cvsps maintainership, abd a --fast-export option

[Andreas Schwab]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> esr@snark:~/WWW/cvsps/fixrepos$ git clone http://repo.or.cz/w/cvsps-hv.git

>From <http://repo.or.cz/w/cvsps-hv.git>:

URL  git://repo.or.cz/cvsps-hv.git
     http://repo.or.cz/r/cvsps-hv.git

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Change in cvsps maintainership, abd a --fast-export option

[Andreas Schwab]

Antoine Pelisse <apelisse@gmail.com> writes:

> I guess 'w' means write

No, 'w' mean gitweb.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Re: Re: Re: Change in cvsps maintainership, abd a --fast-export option

[Heiko Voigt]

On Sat, Dec 22, 2012 at 09:15:19AM -0500, Eric S. Raymond wrote:
> sr@snark:~/WWW/cvsps/fixrepos$ git clone http://repo.or.cz/w/cvsps-hv.git
> Cloning into 'cvsps-hv'...
> fatal: http://repo.or.cz/w/cvsps-hv.git/info/refs not valid: is this a git repository?

That link refers to the webpage of the repository. The clone url is found on
that page. Use this address for cloning:

	git://repo.or.cz/cvsps-hv.git

Cheers Heiko