All of lore.kernel.org
 help / color / mirror / Atom feed
* [RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper "import" command
@ 2012-01-22  5:46 Jonathan Nieder
  2012-01-22 23:35 ` Jelmer Vernooij
  2012-02-03  9:43 ` Gabriel Filion
  0 siblings, 2 replies; 4+ messages in thread
From: Jonathan Nieder @ 2012-01-22  5:46 UTC (permalink / raw)
  To: Gabriel Filion
  Cc: git, Simon Poirier, Sverre Rabbelier, Jeff King, David Barr,
	Dmitry Ivankov

Git 1.7.7 (commit 9504bc9d, "transport-helper: change import
semantics", 2011-07-16) incompatibly changed the interface of the
"import" capability.

Before, git would always send a single import command, which the
remote helper would respond to with a fast-import stream, terminated
by end of file, meaning there was no way to fetch multiple refs in one
connection.  Nowadays, git instead sends a sequence of import lines:

	import refs/heads/foo
	import refs/heads/bar

terminated by a blank line.  The helper is to respond with a
fast-import stream terminated by the "done" command and process
further commands until another blank line indicates the end of the
command stream.
---
Hi Simon and Gabriel,

Here's a rough patch against git://github.com/lelutin/git-remote-bzr.git
master.

Without this patch, whenever I try to use "git clone bzr::<something>",
after doing all the work it removes the resulting repo and exits with
status 141 (SIGPIPE).  Maybe the transport-helper should mask SIGPIPE
when writing the final newline to avoid that.

I'd have prefered to write a patch for remote-bzr that works with
older versions of git fast-import, too, but it wasn't obvious how.
Hints welcome.

BTW, would you mind if I sent a patch to include git-remote-bzr in
git.git under contrib/?

Thanks for git remote-bzr!  I'd be happy for any thoughts you have.

Ciao,
Jonathan

[1] http://thread.gmane.org/gmane.comp.version-control.git/176002/focus=176606

 README.rst     |    2 +-
 git-remote-bzr |   33 ++++++++++++++++++++++++++++++---
 2 files changed, 31 insertions(+), 4 deletions(-)

diff --git a/README.rst b/README.rst
index 3eb3e476..f4dbbeb2 100644
--- a/README.rst
+++ b/README.rst
@@ -34,7 +34,7 @@ Relevant bug reports
 Requirements
 ------------
 
-- git 1.6.6 or later
+- git 1.7.7 or later
 - python 2.5 +
 - bzr 2.x
 - bzr-fastimport
diff --git a/git-remote-bzr b/git-remote-bzr
index 1e3a05f9..501fffe3 100755
--- a/git-remote-bzr
+++ b/git-remote-bzr
@@ -49,7 +49,7 @@ def do_list(repo, args):
     print  # end list
 
 
-def do_import(repo, args):
+def import_one_ref(repo, args):
     """Import a fast-import stream that is exported from Bazaar."""
     if len(args) != 1:
         die("Import needs exactly one ref")
@@ -65,6 +65,23 @@ def do_import(repo, args):
     if bzrp.wait():
         die("'bzr fast-export' returned unexpectedly with code %d",
             bzrp.returncode)
+    print "done"
+
+
+def do_import(repo,args):
+    import_one_ref(repo, args)
+
+    cmdline = True
+    while cmdline:
+        cmdline = next_command()
+        if not cmdline:
+            # Return to main processing loop
+            return True
+        cmd = cmdline.pop(0)
+        if cmd != "import":
+            warn("Unexpected command %s during import" % cmd)
+            return False
+        import_one_ref(repo, cmdline)
 
 
 def do_push(repo, args):
@@ -123,8 +140,8 @@ def sanitize(value):
     return value
 
 
-def read_one_line(repo):
-    """Read and process one command."""
+def next_command():
+    """Read one command."""
     line = sys.stdin.readline()
 
     cmdline = line
@@ -138,6 +155,16 @@ def read_one_line(repo):
         # Blank line means we're about to quit
         return False
 
+    return cmdline
+
+
+def read_one_line(repo):
+    """Read and process one command."""
+    cmdline = next_command()
+
+    if not cmdline:
+        return False
+
     cmd = cmdline.pop(0)
     debug("Got command '%s' with args '%s'", cmd, ' '.join(cmdline))
 
-- 
1.7.9.rc2

^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper "import" command
  2012-01-22  5:46 [RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper "import" command Jonathan Nieder
@ 2012-01-22 23:35 ` Jelmer Vernooij
  2012-01-23  0:12   ` Jonathan Nieder
  2012-02-03  9:43 ` Gabriel Filion
  1 sibling, 1 reply; 4+ messages in thread
From: Jelmer Vernooij @ 2012-01-22 23:35 UTC (permalink / raw)
  To: Jonathan Nieder
  Cc: Gabriel Filion, git, Simon Poirier, Sverre Rabbelier, Jeff King,
	David Barr, Dmitry Ivankov

On 01/22/2012 06:46 AM, Jonathan Nieder wrote:
> Git 1.7.7 (commit 9504bc9d, "transport-helper: change import
> semantics", 2011-07-16) incompatibly changed the interface of the
> "import" capability.
>
> Before, git would always send a single import command, which the
> remote helper would respond to with a fast-import stream, terminated
> by end of file, meaning there was no way to fetch multiple refs in one
> connection.  Nowadays, git instead sends a sequence of import lines:
>
> 	import refs/heads/foo
> 	import refs/heads/bar
>
> terminated by a blank line.  The helper is to respond with a
> fast-import stream terminated by the "done" command and process
> further commands until another blank line indicates the end of the
> command stream.
> ---
> Hi Simon and Gabriel,
>
> Here's a rough patch against git://github.com/lelutin/git-remote-bzr.git
> master.
>
> Without this patch, whenever I try to use "git clone bzr::<something>",
> after doing all the work it removes the resulting repo and exits with
> status 141 (SIGPIPE).  Maybe the transport-helper should mask SIGPIPE
> when writing the final newline to avoid that.
>
> I'd have prefered to write a patch for remote-bzr that works with
> older versions of git fast-import, too, but it wasn't obvious how.
> Hints welcome.
>
> BTW, would you mind if I sent a patch to include git-remote-bzr in
> git.git under contrib/?
Please note that the bzr-git package, which provides git integration for 
bzr and vice versa, also includes a 'git-remote-bzr' command. Apart from 
the 'import' command, it includes experimental implementations of 
'fetch' and push as well.

It would be nice to consolidate the efforts, or at the very least 
prevent name clashes.

Cheers,

Jelmer

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper "import" command
  2012-01-22 23:35 ` Jelmer Vernooij
@ 2012-01-23  0:12   ` Jonathan Nieder
  0 siblings, 0 replies; 4+ messages in thread
From: Jonathan Nieder @ 2012-01-23  0:12 UTC (permalink / raw)
  To: Jelmer Vernooij
  Cc: Gabriel Filion, git, Sverre Rabbelier, Jeff King, David Barr,
	Dmitry Ivankov

Jelmer Vernooij wrote:
> On 01/22/2012 06:46 AM, Jonathan Nieder wrote:

>> BTW, would you mind if I sent a patch to include git-remote-bzr in
>> git.git under contrib/?
>
> Please note that the bzr-git package, which provides git integration
> for bzr and vice versa, also includes a 'git-remote-bzr' command.

That's good to hear.  Then there should be no need for git.git to have
its own helper.

Unfortunately, when I try to clone any repo, at the last step I get

 Traceback (most recent call last):map 6/7
  File "/usr/lib/git-core/git-remote-bzr", line 220, in <module>
    commands[argv[0]](argv, shortname, remote_dir)
  File "/usr/lib/git-core/git-remote-bzr", line 89, in cmd_list
    for ref, git_sha1 in refs.as_dict().iteritems():
  File "/usr/lib/python2.7/dist-packages/dulwich/repo.py", line 196, in as_dict
    ret[key] = self[("%s/%s" % (base, key)).strip("/")]
  File "/usr/lib/python2.7/dist-packages/dulwich/repo.py", line 267, in __getitem__
    _, sha = self._follow(name)
  File "/usr/lib/python2.7/dist-packages/dulwich/repo.py", line 249, in _follow
    contents = self.read_ref(refname)
  File "/usr/lib/python2.7/dist-packages/dulwich/repo.py", line 225, in read_ref
    contents = self.read_loose_ref(refname)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/git/refs.py", line 129, in read_loose_ref
    tag_name = ref_to_tag_name(ref)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/git/refs.py", line 89, in ref_to_tag_name
    raise ValueError("unable to map ref %s back to tag name" % ref)
 ValueError: unable to map ref refs/heads back to tag name

Will file a bug.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper "import" command
  2012-01-22  5:46 [RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper "import" command Jonathan Nieder
  2012-01-22 23:35 ` Jelmer Vernooij
@ 2012-02-03  9:43 ` Gabriel Filion
  1 sibling, 0 replies; 4+ messages in thread
From: Gabriel Filion @ 2012-02-03  9:43 UTC (permalink / raw)
  To: Jonathan Nieder
  Cc: git, Simon Poirier, Sverre Rabbelier, Jeff King, David Barr,
	Dmitry Ivankov, Jelmer Vernooij

Hello,

On 12-01-22 12:46 AM, Jonathan Nieder wrote:
> Hi Simon and Gabriel,
> 
> Here's a rough patch against git://github.com/lelutin/git-remote-bzr.git
> master.

great! thanks for your help.

I must admit that this project never got to completion and is now
getting quite the low priority for my part. I'm no longer working for
the company that was using mainly Bazaar as their VCS, and I've been a
happy git-only user for some time now.
So, I don't have the same incentive to complete the project as I had before.

But I'd be happy to see this get to a point where it's working fine.

> Without this patch, whenever I try to use "git clone bzr::<something>",
> after doing all the work it removes the resulting repo and exits with
> status 141 (SIGPIPE).  Maybe the transport-helper should mask SIGPIPE
> when writing the final newline to avoid that.
> 
> I'd have prefered to write a patch for remote-bzr that works with
> older versions of git fast-import, too, but it wasn't obvious how.
> Hints welcome.

hmm.. I can wait some time to see if some ideas come out around this,
and commit your patch as-is if there are no comments/reworks.

> BTW, would you mind if I sent a patch to include git-remote-bzr in
> git.git under contrib/?

absolutely not, that'd be great actually :)

I didn't do that up to now, though, since I bumped into so much bugs
that I couldn't work out -- some very bad performance issues, and
problems with handling mark files with bzr-fastimport.

> Thanks for git remote-bzr!  I'd be happy for any thoughts you have.

The idea behind git-remote-bzr was to be able to interact with Bazaar
from within your git repository, i.e. to expose remote branches that you
can pull from and push to using the default git commands, without having
to learn to use yet another tool since the remote-helper would be
interfacing with the tool for you.

I have dived for a short period of time into bzrlib, the python library
behind Bazaar, to see how much work it would take to put together a
simplified fast-import client.. and .... wechrk!
It was a huge maze of version-dependant code (the API undergoes big
changes frequently, and backwards compatibility is maintained pretty
far) mixed with a 40-foot-deep class hierarchy. So I quickly gave up on
that idea..

IIRC, at the time I started work on this, Simon and I used
bzr-fastimport[1] because we were able to get farther with this tool. I
haven't used bzr-git[2] that much, though, so I can't comment too
extensively on it. But I would guess that it is maintained more
frequently than bzr-fastimport is, so it could be a better choice for
the backend fast-import client.

I would guess that Jelmer would be happy to help out with interfacing
with bzr-git.

-- 
Gabriel Filion

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2012-02-03  9:43 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-01-22  5:46 [RFC/PATCH git-remote-bzr] Adapt to new semantics of remote-helper "import" command Jonathan Nieder
2012-01-22 23:35 ` Jelmer Vernooij
2012-01-23  0:12   ` Jonathan Nieder
2012-02-03  9:43 ` Gabriel Filion

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.