[PATCH 00/13] remote-hg: general updates

[PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

Hi,

Here is the next round of patches for remote-hg, some which have been
contributed through github.

Fortunately it seems to be working for the most part, but there are some
considerable issues while pushing branches and tags.

Dusty Phillips (1):
  remote-hg: add missing config variable in doc

Felipe Contreras (11):
  remote-hg: trivial cleanups
  remote-hg: properly report errors on bookmark pushes
  remote-hg: make sure fake bookmarks are updated
  remote-hg: trivial test cleanups
  remote-hg: redirect buggy mercurial output
  remote-hg: split bookmark handling
  remote-hg: refactor export
  remote-hg: update remote bookmarks
  remote-hg: force remote push
  remote-hg: don't update bookmarks unnecessarily
  remote-hg: update tags globally

Peter van Zetten (1):
  remote-hg: fix for files with spaces

 contrib/remote-helpers/git-remote-hg     | 73 ++++++++++++++++++++++++--------
 contrib/remote-helpers/test-hg-bidi.sh   |  6 +--
 contrib/remote-helpers/test-hg-hg-git.sh |  4 +-
 3 files changed, 61 insertions(+), 22 deletions(-)

-- 
1.8.2

[PATCH 01/13] remote-hg: trivial cleanups

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 328c2dc..d0dfb1e 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -531,7 +531,6 @@ def parse_blob(parser):
     data = parser.get_data()
     blob_marks[mark] = data
     parser.next()
-    return
 
 def get_merge_files(repo, p1, p2, files):
     for e in repo[p1].files():
@@ -542,7 +541,7 @@ def get_merge_files(repo, p1, p2, files):
             files[e] = f
 
 def parse_commit(parser):
-    global marks, blob_marks, bmarks, parsed_refs
+    global marks, blob_marks, parsed_refs
     global mode
 
     from_mark = merge_mark = None
@@ -647,10 +646,11 @@ def parse_commit(parser):
     rev = repo[node].rev()
 
     parsed_refs[ref] = node
-
     marks.new_mark(rev, commit_mark)
 
 def parse_reset(parser):
+    global parsed_refs
+
     ref = parser[1]
     parser.next()
     # ugh
@@ -715,11 +715,11 @@ def do_export(parser):
             continue
         print "ok %s" % ref
 
-    print
-
     if peer:
         parser.repo.push(peer, force=False)
 
+    print
+
 def fix_path(alias, repo, orig_url):
     repo_url = util.url(repo.url())
     url = util.url(orig_url)
-- 
1.8.2

[PATCH 02/13] remote-hg: add missing config variable in doc

[Felipe Contreras]

From: Dusty Phillips <dusty@linux.ca>

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index d0dfb1e..844ec50 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -23,6 +23,10 @@ import urllib
 # If you want to switch to hg-git compatibility mode:
 # git config --global remote-hg.hg-git-compat true
 #
+# If you are not in hg-git-compat mode and want to disable the tracking of
+# named branches:
+# git config --global remote-hg.track-branches false
+#
 # git:
 # Sensible defaults for git.
 # hg bookmarks are exported as git branches, hg branches are prefixed
-- 
1.8.2

[PATCH 03/13] remote-hg: properly report errors on bookmark pushes

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 1 +
 1 file changed, 1 insertion(+)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 844ec50..19eb4db 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -710,6 +710,7 @@ def do_export(parser):
             else:
                 old = ''
             if not bookmarks.pushbookmark(parser.repo, bmark, old, node):
+                print "error %s" % ref
                 continue
         elif ref.startswith('refs/tags/'):
             tag = ref[len('refs/tags/'):]
-- 
1.8.2

[PATCH 04/13] remote-hg: fix for files with spaces

[Felipe Contreras]

From: Peter van Zetten <peter.van.zetten@cgi.com>

Set the maximum number of splits to make when dividing the diff stat
lines based on space characters.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 19eb4db..c6a1a47 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -578,7 +578,7 @@ def parse_commit(parser):
             mark = int(mark_ref[1:])
             f = { 'mode' : hgmode(m), 'data' : blob_marks[mark] }
         elif parser.check('D'):
-            t, path = line.split(' ')
+            t, path = line.split(' ', 1)
             f = { 'deleted' : True }
         else:
             die('Unknown file command: %s' % line)
@@ -625,7 +625,7 @@ def parse_commit(parser):
         i = data.find('\n--HG--\n')
         if i >= 0:
             tmp = data[i + len('\n--HG--\n'):].strip()
-            for k, v in [e.split(' : ') for e in tmp.split('\n')]:
+            for k, v in [e.split(' : ', 1) for e in tmp.split('\n')]:
                 if k == 'rename':
                     old, new = v.split(' => ', 1)
                     files[new]['rename'] = old
-- 
1.8.2

[PATCH 05/13] remote-hg: make sure fake bookmarks are updated

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg     | 7 +++++++
 contrib/remote-helpers/test-hg-bidi.sh   | 1 +
 contrib/remote-helpers/test-hg-hg-git.sh | 1 +
 3 files changed, 9 insertions(+)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index c6a1a47..b200e60 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -709,9 +709,16 @@ def do_export(parser):
                 old = bmarks[bmark].hex()
             else:
                 old = ''
+
+            if bmark == 'master' and 'master' not in parser.repo._bookmarks:
+                # fake bookmark
+                print "ok %s" % ref
+                continue
+
             if not bookmarks.pushbookmark(parser.repo, bmark, old, node):
                 print "error %s" % ref
                 continue
+
         elif ref.startswith('refs/tags/'):
             tag = ref[len('refs/tags/'):]
             parser.repo.tag([tag], node, None, True, None, {})
diff --git a/contrib/remote-helpers/test-hg-bidi.sh b/contrib/remote-helpers/test-hg-bidi.sh
index 1d61982..fe38e49 100755
--- a/contrib/remote-helpers/test-hg-bidi.sh
+++ b/contrib/remote-helpers/test-hg-bidi.sh
@@ -30,6 +30,7 @@ git_clone () {
 hg_clone () {
 	(
 	hg init $2 &&
+	hg -R $2 bookmark -i master &&
 	cd $1 &&
 	git push -q "hg::$PWD/../$2" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*'
 	) &&
diff --git a/contrib/remote-helpers/test-hg-hg-git.sh b/contrib/remote-helpers/test-hg-hg-git.sh
index 3f253b7..e116cb0 100755
--- a/contrib/remote-helpers/test-hg-hg-git.sh
+++ b/contrib/remote-helpers/test-hg-hg-git.sh
@@ -35,6 +35,7 @@ git_clone_git () {
 hg_clone_git () {
 	(
 	hg init $2 &&
+	hg -R $2 bookmark -i master &&
 	cd $1 &&
 	git push -q "hg::$PWD/../$2" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*'
 	) &&
-- 
1.8.2

[PATCH 06/13] remote-hg: trivial test cleanups

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/test-hg-bidi.sh   | 5 ++---
 contrib/remote-helpers/test-hg-hg-git.sh | 3 +--
 2 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/contrib/remote-helpers/test-hg-bidi.sh b/contrib/remote-helpers/test-hg-bidi.sh
index fe38e49..a3c88f6 100755
--- a/contrib/remote-helpers/test-hg-bidi.sh
+++ b/contrib/remote-helpers/test-hg-bidi.sh
@@ -22,7 +22,6 @@ fi
 
 # clone to a git repo
 git_clone () {
-	hg -R $1 bookmark -f -r tip master &&
 	git clone -q "hg::$PWD/$1" $2
 }
 
@@ -201,8 +200,8 @@ test_expect_success 'hg branch' '
 	hg_push hgrepo gitrepo &&
 	hg_clone gitrepo hgrepo2 &&
 
-	: TODO, avoid "master" bookmark &&
-	(cd hgrepo2 && hg checkout gamma) &&
+	: Back to the common revision &&
+	(cd hgrepo && hg checkout default) &&
 
 	hg_log hgrepo > expected &&
 	hg_log hgrepo2 > actual &&
diff --git a/contrib/remote-helpers/test-hg-hg-git.sh b/contrib/remote-helpers/test-hg-hg-git.sh
index e116cb0..73ae18d 100755
--- a/contrib/remote-helpers/test-hg-hg-git.sh
+++ b/contrib/remote-helpers/test-hg-hg-git.sh
@@ -27,7 +27,6 @@ fi
 
 # clone to a git repo with git
 git_clone_git () {
-	hg -R $1 bookmark -f -r tip master &&
 	git clone -q "hg::$PWD/$1" $2
 }
 
@@ -48,7 +47,7 @@ git_clone_hg () {
 	(
 	git init -q $2 &&
 	cd $1 &&
-	hg bookmark -f -r tip master &&
+	hg bookmark -i -f -r tip master &&
 	hg -q push -r master ../$2 || true
 	)
 }
-- 
1.8.2

[PATCH 07/13] remote-hg: redirect buggy mercurial output

[Felipe Contreras]

We can't use stdout for that in remote helpers.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 1 +
 1 file changed, 1 insertion(+)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index b200e60..874ccd4 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -271,6 +271,7 @@ def get_repo(url, alias):
 
     myui = ui.ui()
     myui.setconfig('ui', 'interactive', 'off')
+    myui.fout = sys.stderr
 
     if hg.islocal(url):
         repo = hg.repository(myui, url)
-- 
1.8.2

[PATCH 08/13] remote-hg: split bookmark handling

[Felipe Contreras]

Will be useful for remote bookmarks.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 38 +++++++++++++++++++++++-------------
 1 file changed, 24 insertions(+), 14 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 874ccd4..73d79cb 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -685,6 +685,8 @@ def parse_tag(parser):
 def do_export(parser):
     global parsed_refs, bmarks, peer
 
+    p_bmarks = []
+
     parser.next()
 
     for line in parser.each_block('done'):
@@ -706,20 +708,9 @@ def do_export(parser):
             pass
         elif ref.startswith('refs/heads/'):
             bmark = ref[len('refs/heads/'):]
-            if bmark in bmarks:
-                old = bmarks[bmark].hex()
-            else:
-                old = ''
-
-            if bmark == 'master' and 'master' not in parser.repo._bookmarks:
-                # fake bookmark
-                print "ok %s" % ref
-                continue
-
-            if not bookmarks.pushbookmark(parser.repo, bmark, old, node):
-                print "error %s" % ref
-                continue
-
+            p_bmarks.append((bmark, node))
+            # handle below
+            continue
         elif ref.startswith('refs/tags/'):
             tag = ref[len('refs/tags/'):]
             parser.repo.tag([tag], node, None, True, None, {})
@@ -731,6 +722,25 @@ def do_export(parser):
     if peer:
         parser.repo.push(peer, force=False)
 
+    # handle bookmarks
+    for bmark, node in p_bmarks:
+
+        if bmark in bmarks:
+            old = bmarks[bmark].hex()
+        else:
+            old = ''
+
+        if bmark == 'master' and 'master' not in parser.repo._bookmarks:
+            # fake bookmark
+            print "ok %s" % ref
+            continue
+
+        if not bookmarks.pushbookmark(parser.repo, bmark, old, node):
+            print "error %s" % ref
+            continue
+
+        print "ok %s" % ref
+
     print
 
 def fix_path(alias, repo, orig_url):
-- 
1.8.2

[PATCH 09/13] remote-hg: refactor export

[Felipe Contreras]

No functional changes.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 73d79cb..11162a2 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -9,7 +9,7 @@
 # Then you can clone with:
 # git clone hg::/path/to/mercurial/repo/
 
-from mercurial import hg, ui, bookmarks, context, util, encoding
+from mercurial import hg, ui, bookmarks, context, util, encoding, node
 
 import re
 import sys
@@ -60,6 +60,9 @@ def hgmode(mode):
     m = { '100755': 'x', '120000': 'l' }
     return m.get(mode, '')
 
+def hghex(node):
+    return hg.node.hex(node)
+
 def get_config(config):
     cmd = ['git', 'config', '--get', config]
     process = subprocess.Popen(cmd, stdout=subprocess.PIPE)
@@ -705,25 +708,25 @@ def do_export(parser):
 
     for ref, node in parsed_refs.iteritems():
         if ref.startswith('refs/heads/branches'):
-            pass
+            print "ok %s" % ref
         elif ref.startswith('refs/heads/'):
             bmark = ref[len('refs/heads/'):]
             p_bmarks.append((bmark, node))
-            # handle below
             continue
         elif ref.startswith('refs/tags/'):
             tag = ref[len('refs/tags/'):]
             parser.repo.tag([tag], node, None, True, None, {})
+            print "ok %s" % ref
         else:
             # transport-helper/fast-export bugs
             continue
-        print "ok %s" % ref
 
     if peer:
         parser.repo.push(peer, force=False)
 
     # handle bookmarks
     for bmark, node in p_bmarks:
+        new = hghex(node)
 
         if bmark in bmarks:
             old = bmarks[bmark].hex()
@@ -732,10 +735,11 @@ def do_export(parser):
 
         if bmark == 'master' and 'master' not in parser.repo._bookmarks:
             # fake bookmark
-            print "ok %s" % ref
-            continue
-
-        if not bookmarks.pushbookmark(parser.repo, bmark, old, node):
+            pass
+        elif bookmarks.pushbookmark(parser.repo, bmark, old, new):
+            # updated locally
+            pass
+        else:
             print "error %s" % ref
             continue
 
-- 
1.8.2

[PATCH 10/13] remote-hg: update remote bookmarks

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 11162a2..160f486 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -743,6 +743,11 @@ def do_export(parser):
             print "error %s" % ref
             continue
 
+        if peer:
+            if not peer.pushkey('bookmarks', bmark, old, new):
+                print "error %s" % ref
+                continue
+
         print "ok %s" % ref
 
     print
-- 
1.8.2

[PATCH 11/13] remote-hg: force remote push

[Felipe Contreras]

Ideally we shouldn't do this, as it's not recommended in mercurial
documentation, but there's no other way to push multiple bookmarks (on
the same branch), which would be the behavior most similar to git.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 160f486..a1b7e44 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -722,7 +722,7 @@ def do_export(parser):
             continue
 
     if peer:
-        parser.repo.push(peer, force=False)
+        parser.repo.push(peer, force=True)
 
     # handle bookmarks
     for bmark, node in p_bmarks:
-- 
1.8.2

[PATCH 12/13] remote-hg: don't update bookmarks unnecessarily

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index a1b7e44..3130b23 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -733,6 +733,9 @@ def do_export(parser):
         else:
             old = ''
 
+        if old == new:
+            continue
+
         if bmark == 'master' and 'master' not in parser.repo._bookmarks:
             # fake bookmark
             pass
-- 
1.8.2

[PATCH 13/13] remote-hg: update tags globally

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-helpers/git-remote-hg | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
index 3130b23..c9d7636 100755
--- a/contrib/remote-helpers/git-remote-hg
+++ b/contrib/remote-helpers/git-remote-hg
@@ -715,7 +715,11 @@ def do_export(parser):
             continue
         elif ref.startswith('refs/tags/'):
             tag = ref[len('refs/tags/'):]
-            parser.repo.tag([tag], node, None, True, None, {})
+            if mode == 'git':
+                msg = 'Added tag %s for changeset %s' % (tag, hghex(node[:6]));
+                parser.repo.tag([tag], node, msg, False, None, {})
+            else:
+                parser.repo.tag([tag], node, None, True, None, {})
             print "ok %s" % ref
         else:
             # transport-helper/fast-export bugs
-- 
1.8.2

Re: [PATCH 00/13] remote-hg: general updates

[Junio C Hamano]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> Here is the next round of patches for remote-hg, some which have been
> contributed through github.

Thanks.

> Fortunately it seems to be working for the most part, but there are some
> considerable issues while pushing branches and tags.

Do you have a plan in mind what to do about "some considerable
issues"?

I could push these out to 'master' and let the interested parties
sort it out---having early access to the code everybody bases his
effort on would help.

I could queue these on 'pu' and do the same, and wait until you say
"now it is ready, let's go to 'next'" (and same for 'master').

Or are they meant as "There are issues, but here is a snapshot of
what I have at this moment.  Hopefully others can help me update it
by trying it out and discussing, which may lead me to post a reroll
for application"?

I'll queue them on 'pu' in the meantime.

>
> Dusty Phillips (1):
>   remote-hg: add missing config variable in doc
>
> Felipe Contreras (11):
>   remote-hg: trivial cleanups
>   remote-hg: properly report errors on bookmark pushes
>   remote-hg: make sure fake bookmarks are updated
>   remote-hg: trivial test cleanups
>   remote-hg: redirect buggy mercurial output
>   remote-hg: split bookmark handling
>   remote-hg: refactor export
>   remote-hg: update remote bookmarks
>   remote-hg: force remote push
>   remote-hg: don't update bookmarks unnecessarily
>   remote-hg: update tags globally
>
> Peter van Zetten (1):
>   remote-hg: fix for files with spaces
>
>  contrib/remote-helpers/git-remote-hg     | 73 ++++++++++++++++++++++++--------
>  contrib/remote-helpers/test-hg-bidi.sh   |  6 +--
>  contrib/remote-helpers/test-hg-hg-git.sh |  4 +-
>  3 files changed, 61 insertions(+), 22 deletions(-)

Re: [PATCH 07/13] remote-hg: redirect buggy mercurial output

[Junio C Hamano]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> We can't use stdout for that in remote helpers.
>
> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
> ---

You may want to clarify "buggy output" a bit.  Will mercurial
forever be broken?  Some versions of Hg emit [[[it is unclear for
Junio to tell what it is to fill this blank]]] to its output that
we want to ignore?

>  contrib/remote-helpers/git-remote-hg | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/contrib/remote-helpers/git-remote-hg b/contrib/remote-helpers/git-remote-hg
> index b200e60..874ccd4 100755
> --- a/contrib/remote-helpers/git-remote-hg
> +++ b/contrib/remote-helpers/git-remote-hg
> @@ -271,6 +271,7 @@ def get_repo(url, alias):
>  
>      myui = ui.ui()
>      myui.setconfig('ui', 'interactive', 'off')
> +    myui.fout = sys.stderr
>  
>      if hg.islocal(url):
>          repo = hg.repository(myui, url)

Re: [PATCH 00/13] remote-hg: general updates

[John Keeping]

On Tue, Apr 02, 2013 at 01:02:49PM -0600, Felipe Contreras wrote:
> Here is the next round of patches for remote-hg, some which have been
> contributed through github.
> 
> Fortunately it seems to be working for the most part, but there are some
> considerable issues while pushing branches and tags.

How does this compare to the current state of gitifyhg[1]?  That's built
on top of this git-remote-hg script but seems to have been more actively
developed recently.

[1] https://github.com/buchuki/gitifyhg

> Dusty Phillips (1):
>   remote-hg: add missing config variable in doc
> 
> Felipe Contreras (11):
>   remote-hg: trivial cleanups
>   remote-hg: properly report errors on bookmark pushes
>   remote-hg: make sure fake bookmarks are updated
>   remote-hg: trivial test cleanups
>   remote-hg: redirect buggy mercurial output
>   remote-hg: split bookmark handling
>   remote-hg: refactor export
>   remote-hg: update remote bookmarks
>   remote-hg: force remote push
>   remote-hg: don't update bookmarks unnecessarily
>   remote-hg: update tags globally
> 
> Peter van Zetten (1):
>   remote-hg: fix for files with spaces
> 
>  contrib/remote-helpers/git-remote-hg     | 73 ++++++++++++++++++++++++--------
>  contrib/remote-helpers/test-hg-bidi.sh   |  6 +--
>  contrib/remote-helpers/test-hg-hg-git.sh |  4 +-
>  3 files changed, 61 insertions(+), 22 deletions(-)
> 
> -- 
> 1.8.2

Re: [PATCH 07/13] remote-hg: redirect buggy mercurial output

[Felipe Contreras]

On Tue, Apr 2, 2013 at 1:58 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Felipe Contreras <felipe.contreras@gmail.com> writes:
>
>> We can't use stdout for that in remote helpers.
>>
>> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
>> ---
>
> You may want to clarify "buggy output" a bit.  Will mercurial
> forever be broken?  Some versions of Hg emit [[[it is unclear for
> Junio to tell what it is to fill this blank]]] to its output that
> we want to ignore?

The problem is that mercurial's code is kind of hardcoded to run under
mercurial's UI, so it throws messages around willynillingly, like:

searching for changes
no changes found

And they can't be turned off. Theoretically we could override
mercurial's UI class, but I think that has the potential to create
more problems, it's not worth at this point in time.

Cheers.

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Tue, Apr 2, 2013 at 1:55 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Felipe Contreras <felipe.contreras@gmail.com> writes:
>
>> Here is the next round of patches for remote-hg, some which have been
>> contributed through github.
>
> Thanks.
>
>> Fortunately it seems to be working for the most part, but there are some
>> considerable issues while pushing branches and tags.
>
> Do you have a plan in mind what to do about "some considerable
> issues"?

Yes, they should be fixed now with this series :) I'm still waiting
for the people that reported those issues to confirm, but in my tests
they do work.

> I could push these out to 'master' and let the interested parties
> sort it out---having early access to the code everybody bases his
> effort on would help.
>
> I could queue these on 'pu' and do the same, and wait until you say
> "now it is ready, let's go to 'next'" (and same for 'master').

That might help. However, please drop the patch "don't update
bookmarks unnecessarily", I did not intend to push that one, and I
would like the rest of the patches to be tested before pushing that
one out.

> Or are they meant as "There are issues, but here is a snapshot of
> what I have at this moment.  Hopefully others can help me update it
> by trying it out and discussing, which may lead me to post a reroll
> for application"?

Nah, I think these patches fix the issues, the only question is
whether or not they break something else.

> I'll queue them on 'pu' in the meantime.

Thanks.

-- 
Felipe Contreras

Re: [PATCH 07/13] remote-hg: redirect buggy mercurial output

[Junio C Hamano]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> On Tue, Apr 2, 2013 at 1:58 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Felipe Contreras <felipe.contreras@gmail.com> writes:
>>
>>> We can't use stdout for that in remote helpers.
>>>
>>> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
>>> ---
>>
>> You may want to clarify "buggy output" a bit.  Will mercurial
>> forever be broken?  Some versions of Hg emit [[[it is unclear for
>> Junio to tell what it is to fill this blank]]] to its output that
>> we want to ignore?
>
> The problem is that mercurial's code is kind of hardcoded to run under
> mercurial's UI, so it throws messages around willynillingly, like:
>
> searching for changes
> no changes found
>
> And they can't be turned off. Theoretically we could override
> mercurial's UI class, but I think that has the potential to create
> more problems, it's not worth at this point in time.

Oh, I totally agree with you _after_ reading that explanation.

You just shouldn't let me waste your time to explain that to me in
this exchange, and you could have done so by writing a clearer log
message.  That's all.

>
> Cheers.

Re: [PATCH 00/13] remote-hg: general updates

[Junio C Hamano]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> On Tue, Apr 2, 2013 at 1:55 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
>>> Fortunately it seems to be working for the most part, but there are some
>>> considerable issues while pushing branches and tags.
>>
>> Do you have a plan in mind what to do about "some considerable
>> issues"?
>
> Yes, they should be fixed now with this series :) I'm still waiting
> for the people that reported those issues to confirm, but in my tests
> they do work.

Ah, I just misread the original to mean "This fixes pushes but still
has (or even adds new) considerable issues that need to be worked
out", hence my question.

>> I could queue these on 'pu' and do the same, and wait until you say
>> "now it is ready, let's go to 'next'" (and same for 'master').
>
> That might help. However, please drop the patch "don't update
> bookmarks unnecessarily", I did not intend to push that one, and I
> would like the rest of the patches to be tested before pushing that
> one out.

OK, then let's do that.

Thanks.

Re: [PATCH 00/13] remote-hg: general updates

[Max Horn]

On 02.04.2013, at 22:09, John Keeping wrote:

> On Tue, Apr 02, 2013 at 01:02:49PM -0600, Felipe Contreras wrote:
>> Here is the next round of patches for remote-hg, some which have been
>> contributed through github.
>> 
>> Fortunately it seems to be working for the most part, but there are some
>> considerable issues while pushing branches and tags.
> 
> How does this compare to the current state of gitifyhg[1]?  That's built
> on top of this git-remote-hg script but seems to have been more actively
> developed recently.

Several bugs that were fixed in gitifyhg some time ago are now fixed in this remote-hg, too.

I'll try to list some of remaining differences, mostly (in my biased opinion) improvements on the gitifyhg side. Note that some of these might be outdated with felipe's recent changes, i.e. I have not yet had time to review and/or test them all. So please bear that in mind.

* added many new test cases, sadly still including some xfails. Several of these (both passing and xfailing) also apply to remote-hg (i.e. the issue is also present in contrib's remote-hg)

* improved handling of hg user names (remote-hg is not able to deal with some pathological cases, failing to import commits). Sadly, mercurial allows arbitrary strings as usernames, git doesn't...

* failed pushes to hg are cleanly rolled back (using mq.strip() from the mq extension), instead of resulting in inconsistent internal state. This is quite important in real life, and has bitten me several times with remote-hg (and was the initial reason why I switched to gitifyhg). A typical way to reproduce this is to push to a remote repository that has commits not yet in my local clone.

* git notes are used to associate to each git commit the sha1 of the corresponding hg commit, to help users figure out that mapping

* internally, the marks are using the hg sha1s instead of the hg rev ids. The latter are not necessarily invariant, and using the sha1s makes it much easier to recover from semi-broken states.

* Better handling of various hg errors, see e.g. [2]. More work is still needed there with both tools, though [3].

* Support for creating hg tags from git (i.e. pushing light git tags to heavy hg tags)

* The gitifyhg test suite is run after each push on Travis CI against several git / mercurial combinations [4].
In particular, unlike all other remote-hg implementations I know, we explicitly promise (and test) compatibility with a specific range of Mercurial versions (not just the one the dev happens to have installed right now). This has been a frequent issue for me with the msysgit remote-hg

* Renaming a gitifyhg remote just works [5]. Doing that with remote-hg triggers a re-clone of the remote repository (if it works at all, I don't remember). 


Sadly, while working on gitifyhg, we discovered various more design problems (from our perspective, at least) in Mercurial, e.g. the fact that commits are not necessarily normalized, in the sense that "equivalent" commits (same author, time, changed files / code) can have different hashs, with some nasty implications for import. This is potentially problematic because without extra care, these would be mapped to the same commit on the git side.

Unfortunately, we also stumbled into various problems with the git remote-helper system. We are currently using the fast-import remote-helper type, but are encountering more and more of its limitations. This affects remote-hg and gitifyhg equally, and probably other remote helpers. E.g. "git push --dry-run" seems to be impossible to support with such a remote-helper (but then I might be mistaken).

Thing is, for several of these I don't feel quite competent enough to come up with patches that I could submit here. And in my experience just reporting a perceived problem with the remote-helper API is not going to trigger a response here [6]. I guess that's why we stopped reporting them here for now, but if there is interest I could try to compile an overview.


[1] https://github.com/buchuki/gitifyhg
[2] https://github.com/buchuki/gitifyhg/commit/74b71f4
[3] https://github.com/buchuki/gitifyhg/issues/66
[4] https://travis-ci.org/buchuki/gitifyhg/builds
[5] https://github.com/buchuki/gitifyhg/commit/68ce89bb32
[6] http://thread.gmane.org/gmane.comp.version-control.git/214802

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Tue, Apr 2, 2013 at 4:23 PM, Max Horn <max@quendi.de> wrote:
>
> On 02.04.2013, at 22:09, John Keeping wrote:
>
>> On Tue, Apr 02, 2013 at 01:02:49PM -0600, Felipe Contreras wrote:
>>> Here is the next round of patches for remote-hg, some which have been
>>> contributed through github.
>>>
>>> Fortunately it seems to be working for the most part, but there are some
>>> considerable issues while pushing branches and tags.
>>
>> How does this compare to the current state of gitifyhg[1]?  That's built
>> on top of this git-remote-hg script but seems to have been more actively
>> developed recently.

I only learned about it recently, I've looked at the history and to me
it seems rather chaotic, and a lot of the code was simply copied from
git-remote-hg without comment.

> * added many new test cases, sadly still including some xfails. Several of these (both passing and xfailing) also apply to remote-hg (i.e. the issue is also present in contrib's remote-hg)

I ran these test-cases with remote-hg, and the same test-cases pass. I
only had to do minor modifications, most of the failures came from
subtle differences such as different strategies to sanitize authors,
and which branch to pick for HEAD.

> * improved handling of hg user names (remote-hg is not able to deal with some pathological cases, failing to import commits). Sadly, mercurial allows arbitrary strings as usernames, git doesn't...

I wouldn't call it improved. In some cases the remote-hg result is
better, in others gitifyhg is, but there's only a single case where
the author name becomes a significant problem. It's a trivial fix.

> * failed pushes to hg are cleanly rolled back (using mq.strip() from the mq extension), instead of resulting in inconsistent internal state. This is quite important in real life, and has bitten me several times with remote-hg (and was the initial reason why I switched to gitifyhg). A typical way to reproduce this is to push to a remote repository that has commits not yet in my local clone.

This is not an issue in remote-hg any more since now we force the
push. It's not nice, but there's no other way to push multiple
bookmarks (aka git branches) to the same branch (aka commit label).

I doubt these inconsistent states can happen any more, but if they do,
the plan in remote-hg is to simply ignore those revisions, and only
push the ones that have git refs. I have the code for that, but I'll
not be pushing it to git.git for the time being.

> * git notes are used to associate to each git commit the sha1 of the corresponding hg commit, to help users figure out that mapping

This is a minor feature. I've had the code for this for quite some
time, but for the moment I think there are higher priorities.

> * internally, the marks are using the hg sha1s instead of the hg rev ids. The latter are not necessarily invariant, and using the sha1s makes it much easier to recover from semi-broken states.

I doubt this makes any difference (except for more wasted space).

> * Better handling of various hg errors, see e.g. [2]. More work is still needed there with both tools, though [3].

This is literally a three lines fix, and it simply makes one error
nicer. Hardly worth mentioning.

> * Support for creating hg tags from git (i.e. pushing light git tags to heavy hg tags)

remote-hg has the same.

> * The gitifyhg test suite is run after each push on Travis CI against several git / mercurial combinations [4].
> In particular, unlike all other remote-hg implementations I know, we explicitly promise (and test) compatibility with a specific range of Mercurial versions (not just the one the dev happens to have installed right now). This has been a frequent issue for me with the msysgit remote-hg

I've personally checked against multiple versions of Mercurial. It's
possible that some error might slip by, but it would get quickly
noticed.

> * Renaming a gitifyhg remote just works [5]. Doing that with remote-hg triggers a re-clone of the remote repository (if it works at all, I don't remember).

Yeah, now you can change the alias of the remote, but you can't change
the remote url. This is not really an advantage, simply an almost
imperceptible different choice.

I still don't see any good reason why a user might prefer gitifyhg,
even more importantly, why gitifyhg developers don't contribute to
remote-hg.

Also, unlike remote-hg, which basically passes all the tests of
gitifyhg, gitifyhg barely passes any tests of remote-hg (three).

Cheers.

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Tue, Apr 2, 2013 at 7:31 PM, Felipe Contreras
<felipe.contreras@gmail.com> wrote:

>> * added many new test cases, sadly still including some xfails. Several of these (both passing and xfailing) also apply to remote-hg (i.e. the issue is also present in contrib's remote-hg)
>
> I ran these test-cases with remote-hg, and the same test-cases pass. I
> only had to do minor modifications, most of the failures came from
> subtle differences such as different strategies to sanitize authors,
> and which branch to pick for HEAD.

After doing some modifications in remote-hg, here are the test cases
of gitifyhg v0.8 without modifications:

========================================================= test session
starts ==========================================================
platform linux2 -- Python 2.7.3 -- pytest-2.3.4
collected 80 items

test/test_author.py ........F
test/test_clone.py ......xx.........x...x..
test/test_notes.py ..FxF
test/test_pull.py ....x..xx..
test/test_push.py ..............x........FF...
test/test_special_cases.py ...

===============================================================
FAILURES ===============================================================
/home/felipec/tmp/gitifyhg/test/helpers.py:118: assert 'totally
<bad...e used in hg>' == 'totally <unknown>'
/usr/lib/python2.7/site-packages/sh.py:309: ErrorReturnCode_128:
/home/felipec/tmp/gitifyhg/test/test_notes.py:107: assert not 'error'
in 'searching for changes\nno changes found\nFrom
hg::file:///tmp/pytest-91/test_simple_push_updates_notes_after_contentf...rror:
refs/notes/hg does not point to a valid object!\nerror:
refs/notes/hg-origin does not point to a valid object!\n'
/home/felipec/tmp/gitifyhg/test/helpers.py:108: assert [u'Added tag
...780a9c', u'a'] == ['I tagged thi...nd user', 'a']
/home/felipec/tmp/gitifyhg/test/test_push.py:410: assert 'default' ==
'branch_one'
=========================================== 5 failed, 66 passed, 9
xfailed in 75.23 seconds ============================================

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Antoine Pelisse]

>> * internally, the marks are using the hg sha1s instead of the hg rev ids. The latter are not necessarily invariant, and using the sha1s makes it much easier to recover from semi-broken states.
>
> I doubt this makes any difference (except for more wasted space).

I think this is definitely wrong. If you happen to strip a changeset
from the mercurial repository, and redo a completely different commit
on top of it, the new commit will never be seen on git end (because it
will have the same rev id and will thus be identified as identical
from git point of view).

Re: [PATCH 00/13] remote-hg: general updates

[Max Horn]

On 03.04.2013, at 03:31, Felipe Contreras wrote:

> On Tue, Apr 2, 2013 at 4:23 PM, Max Horn <max@quendi.de> wrote:
>> 
>> On 02.04.2013, at 22:09, John Keeping wrote:
>> 
>>> On Tue, Apr 02, 2013 at 01:02:49PM -0600, Felipe Contreras wrote:
>>>> Here is the next round of patches for remote-hg, some which have been
>>>> contributed through github.
>>>> 
>>>> Fortunately it seems to be working for the most part, but there are some
>>>> considerable issues while pushing branches and tags.
>>> 
>>> How does this compare to the current state of gitifyhg[1]?  That's built
>>> on top of this git-remote-hg script but seems to have been more actively
>>> developed recently.
> 
> I only learned about it recently, I've looked at the history and to me
> it seems rather chaotic, and a lot of the code was simply copied from
> git-remote-hg without comment.

gitifyhg was scrapped and completely restarted from scratch at some point. Based largely on your git-remote-hg code. A bit more on its history can be read here:
  http://archlinux.me/dusty/2013/01/06/gitifyhg-rewritten/


> 
>> * added many new test cases, sadly still including some xfails. Several of these (both passing and xfailing) also apply to remote-hg (i.e. the issue is also present in contrib's remote-hg)
> 
> I ran these test-cases with remote-hg, and the same test-cases pass. I
> only had to do minor modifications, most of the failures came from
> subtle differences such as different strategies to sanitize authors,
> and which branch to pick for HEAD.

Yeah, that's because what I wrote was based on the situation before your recent patch series. Glad to see that git/contrib's remote-hg is improving!


> 
>> * improved handling of hg user names (remote-hg is not able to deal with some pathological cases, failing to import commits). Sadly, mercurial allows arbitrary strings as usernames, git doesn't...
> 
> I wouldn't call it improved. In some cases the remote-hg result is
> better, in others gitifyhg is,

I'd love to learn about cases where remote-hg's result is better in your opinion, so that I can see if the mapping in gitifyhg could be improved for those cases.

That said, this part is really quite subjective I guess. In the end, since Mercurial names can be *anything*, one can never get a "perfect" mapping. Luckily, for most real repositories out there, user names will be quite sane and remote-hg and gitifyhg will produce identical results. (Although some hg repos out there have some really weird stuff going on. Yuck.)

> but there's only a single case where
> the author name becomes a significant problem. It's a trivial fix.

Excellent, looking forward to it then.

> 
>> * failed pushes to hg are cleanly rolled back (using mq.strip() from the mq extension), instead of resulting in inconsistent internal state. This is quite important in real life, and has bitten me several times with remote-hg (and was the initial reason why I switched to gitifyhg). A typical way to reproduce this is to push to a remote repository that has commits not yet in my local clone.
> 
> This is not an issue in remote-hg any more since now we force the
> push. It's not nice, but there's no other way to push multiple
> bookmarks (aka git branches) to the same branch (aka commit label).

Uhh, what? The push is forced? That sounds to me like a much, much bigger issue with remote-hg than anything else ever was, from my point of view. That seems tobe akin to making "--force" default to "git push", and I don't think anybody here would consider that a good idea.

> I doubt these inconsistent states can happen any more, but if they do,

Seriously? This is triggered quite frequently in real life. And it will very likely cause somebody to mess up a hg repository they work on. As long is this in, using remote-hg is a total no-go for me. Just consider the following scenario:
* user A clones a hg repository into a git repository
* user A commits some commits in the git clone
* meanwhile, user B pushes changes to the hg repository
* user A tries to push his changes to the hg remote

The last step causes this result in gitifyhg (similar to what one gets when the remote is a git repos):

 ! [rejected]        master -> master (non-fast-forward)
error: failed to push some refs to 'gitifyhg::URL'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Merge the remote changes (e.g. 'git pull')
hint: before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.


With remote-hg, you just force push the change, creating a new head in the remote repo. So, yeah, failed pushes which mess up the internal state don't happen anymore. But I rather have those than potentially mess up the upstream repository like that.


> the plan in remote-hg is to simply ignore those revisions, and only
> push the ones that have git refs. I have the code for that, but I'll
> not be pushing it to git.git for the time being.

I am not quite sure what you mean here, but I'll just wait for your code and hope it'll explain itself.

> 
>> * git notes are used to associate to each git commit the sha1 of the corresponding hg commit, to help users figure out that mapping
> 
> This is a minor feature. I've had the code for this for quite some
> time, but for the moment I think there are higher priorities.

Well, I guess it depends on how you use this. In my daily work with hg repositories, that's a quite important feature. 

Actually, I wonder: Are you actually using remote-hg yourself for day-to-day development? Or for any other regular activities? You seem to have quite different usage patterns from me and other people involved with gitifyhg.

> 
>> * internally, the marks are using the hg sha1s instead of the hg rev ids. The latter are not necessarily invariant, and using the sha1s makes it much easier to recover from semi-broken states.
> 
> I doubt this makes any difference (except for more wasted space).

I disagree. Especially since remote-hg does treat local hg repositories differently than remote ones, by not cloning them but rather working with them directly. As soon as somebody uses "hg strip" or "hg graft", etc. on that, your mark files will contain incorrect data.

With a  remote hg repository, that is of course not as much of an issue, since you have a local hg clone stored inside of .git/. In theory, the user would never touch that, so nothing in there changes. In reality, though, sometimes the user still has to dig in there and modify things (e.g. to undo a "bad" push; although of course for now you "fixed" that particular problem by force pushing). Of course in an ideal world, a user never should have to do that, but as long as there are bugs, users may have to. And based on painful experience, it is *much* easier to do that when the marks use the invariant hashes instead of the transient rev ids.

As for the wasted space: For a repository with 999,999 commits, the size of the "marks-hg file should increase from about 1.5 MB to about 5 MB. To use your words: Hardly worth mentioning.


> 
>> * Better handling of various hg errors, see e.g. [2]. More work is still needed there with both tools, though [3].
> 
> This is literally a three lines fix, and it simply makes one error
> nicer. Hardly worth mentioning.

Perhaps not for you, but for users who just want to use remote-hg resp. gitifyhg, getting a helpful error message instead of seeing an (to them) unreadable stack trace is quite important. I guess this just shows that we have quite different goals and ideas about how to interact with users of our respective tools :-).

Anyway, I hope you'll consider applying something like that three lines fix to remote-hg.

> 
>> * Support for creating hg tags from git (i.e. pushing light git tags to heavy hg tags)
> 
> remote-hg has the same.

Excellent.

> 
>> * The gitifyhg test suite is run after each push on Travis CI against several git / mercurial combinations [4].
>> In particular, unlike all other remote-hg implementations I know, we explicitly promise (and test) compatibility with a specific range of Mercurial versions (not just the one the dev happens to have installed right now). This has been a frequent issue for me with the msysgit remote-hg
> 
> I've personally checked against multiple versions of Mercurial. It's
> possible that some error might slip by, but it would get quickly
> noticed.

Really? This sounds close to some people who say things like "I don't need a test suite, I personally run some tests every now and then on my machine."
Of course clearly that is not at all how you operate. Rather you are very sensible and strive to provide a good testsuite which strives to test as much stuff as possible. As it should be. Which is why I am surprised that you compare (a) "personally" checking against some unspecified "multiple versions of Mercurial" at an unspecified frequency with (b) automatically running a testsuite after each push against a specific set of Mercurial versions. Versions which were specifically selected to match what Ubuntu 11.10, 12.04 LTS and 12.10 resp. Debian Wheezy provide, as well as recent versions. See also <https://github.com/buchuki/gitifyhg/blob/master/.travis.yml>. I would like to also let this check with multiple git versions, but that requires collecting / making suitable .debs which I have n
 ot yet gotten around to.

Furthermore, you don't seem to document what versions of Mercurial are supposed to work / not work. (Indeed, as far as I can tell, remote-hg has no documentation whatsoever, another difference. Granted, the gitifyhg README is not particular great at this point, but at least it exists and tells people how to get started and where to get help).

In contrast, the gitifyhg README clearly states that it "requires at least Mercurial 1.9". And its setup.py refuses to install it if the Mercurial version is too old. Again, for us devs that's not very important, but for users, I think it is. In recent months, I had to provide assistance with using hg to tons of people (*sigh*), and old Mercurial versions came up in a considerable portion of those (perhaps 30-40% or so). 

> 
>> * Renaming a gitifyhg remote just works [5]. Doing that with remote-hg triggers a re-clone of the remote repository (if it works at all, I don't remember).
> 
> Yeah, now you can change the alias of the remote, but you can't change
> the remote url.

That's simply wrong. You still can change the remote URL, it will just lead to the creation of a fresh separate local clone.

In contrast, with remote-hg, renaming the remote will create a fresh local clone, while changing the remote URL will *not* do that -- instead, the changes from the new remote will be pulled into the existing local clone. Which in some cases may be exactly what you want (e.g. if a repository just moved to a different remote URL). But in other cases, it won't be, namely when the repository at the new URL contains different content. This is not that unlikely when you consider that it comment Mercurial workflow to use separate repositories for different branches.


> This is not really an advantage, simply an almost
> imperceptible different choice.

These are indeed different design choices, but at least to me, they are very much perceptible :-). And we made this design choice quite consciously, after looking at how it is done in remote-hg, and not liking that.

Anyway: At least in my day-to-day operations, I occasionally rename a remote (very rare, but it happens). So far I never had to change a remote URL. Of course that is just me, perhaps others occasionally (well, more of than me :-) have to change remote URLs. But as I said, that's still possible in gitifyhg.

That said, the situation is certainly not ideal. The fact that remote helpers have no good way to notice if the remote name (or URL, or anything) have changed is the root problem at hand here, I'd say. But for now, we are happy enough with the solution we implemented in gitifyhg, at least for our purposes.


> I still don't see any good reason why a user might prefer gitifyhg,

Well, for me it is exactly the other way around :-). That said, gitifyhg certainly also still has issues and problems, but I am confident it will even better in the coming weeks. And it certainly wouldn't be were it is now without your great work on remote-hg, and also on related improvements you got into git.git. Kudos!


> even more importantly, why gitifyhg developers don't contribute to
> remote-hg.

I can only speak for myself, as a (minor) contributor to both remote-hg and gitifyhg. But here are my reasons why I prefer contributing to remote-hg over gitifyhg:

1) Apparent difference between your goals and mine / those of gitifyhg. I think this is quite visible if one looks at our exchange above. Features that are quite important to me, even crucial, are unimportant for you, or even outright rejected. In contrast, so far with the couple people working on gitifyhg we always managed to arrive at solutions that satisfy all of us. I.e. it seems our ideas and goals are much better aligned within that group.

2) Lack of reactions on pull requests and bug reports on your github pages. Perhaps you never intended this to be used for pull requests / bug reports, but I never (until recently) saw you state that, nor can one read such a statement on your github pages. 

3) Not shipping gitifyhg as part of git/contrib but rather as a separate package is important to me, too. It means that we can make new releases whenever we want, not tied to git. Users can install it easily via "pip", and I also think it gets a lot more visibility this way.


> 
> Also, unlike remote-hg, which basically passes all the tests of
> gitifyhg, gitifyhg barely passes any tests of remote-hg (three).

Heh, bad, but OK (as I said, my message was based on an older version of remote-hg, and actually also on an older gitifyhg). Thank you for the report, I'll look into it as soon as I can (or somebody else might).

BTW, I just pulled you hg-next branch, and run "make test" in that. The tests in test-hg-hg-git.sh actually all failed (with remote-hg). Do I need to do something special for those to work?


Cheers,
Max

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Wed, Apr 3, 2013 at 6:25 PM, Max Horn <max@quendi.de> wrote:
> On 03.04.2013, at 03:31, Felipe Contreras wrote:

>> I only learned about it recently, I've looked at the history and to me
>> it seems rather chaotic, and a lot of the code was simply copied from
>> git-remote-hg without comment.
>
> gitifyhg was scrapped and completely restarted from scratch at some point. Based largely on your git-remote-hg code. A bit more on its history can be read here:
>   http://archlinux.me/dusty/2013/01/06/gitifyhg-rewritten/

Yeah, I'm aware of that, but look at the README:

---
To the best of our knowledge, this is the most robust and usable git
to hg bridge currently available. It has a large test suite and better
documentation than the alternatives we know about. It has been tested
on several large mercurial repositories (including that of mercurial
itself and the pypy repository) that break with various other
git-to-hg bridge projects and is used daily in normal workflow
scenarios.
---

It reads as though gitifyhg was this marvelous piece of code developed
independently, and everything else (including remote-hg) pales in
comparison. There's no mention at all of remote-hg until the very
bitter end where it's mentioned that it's "heavily inspired by and
borrows code" from it, but it's much more than that, it's *based* on
this code. When I read the code, not only do I see it's the same
design, has the same function names, the same variables, the same
order, huge tracks of code are exactly the same, and in fact, I have
too look very carefully to see the differences.

Take a look at this commit:
http://github.com/buchuki/gitifyhg/commit/2c4fd7329fa0db8dd56df3ac8c62fa292580b74e

Geez, that's an awfully nice author sanitize function, but Dusty
Phillips was not the author, I am, he just added 2 lines of code.

But what really bothers me is that the first public version of
remote-hg happened after many many tries and commits, different
designs were discarded based on bad performance, feasibility, or
complexity, and gitifyhg just copies this code and runs with it.

Take a look at this function, which I took a long time to develop,
clean, and improve so it performs as efficiently as possible through
arduous profiling in huge repositories:

    def get_filechanges(self, context, parent):
        modified = set()
        added = set()
        removed = set()

        current = context.manifest()
        previous = self.repo[parent].manifest().copy()

        for fn in current:
            if fn in previous:
                if (current.flags(fn) != previous.flags(fn) or
current[fn] != previous[fn]):
                    modified.add(fn)
                del previous[fn]
            else:
                added.add(fn)
        removed |= set(previous.keys())

        return added | modified, removed

Nobody from gitifyhg even bothered to try to understand this function,
what it does, or why, just copied it:

http://github.com/buchuki/gitifyhg/commit/a46d518e2b8df5e8339c8caa9fa113642bc7ac3a

In fact, it's co clear the code was simply copied without much
thought, that the parent caller was introduced *before* the actual
function:

http://github.com/buchuki/gitifyhg/commit/54be060e1928b7e750bdf3981c32d5bc88851ed1

To me the strategy was clear: copy code of remote-hg until gitifyhg
works, then fix a few bugs, add a few features, and then claim
gitifyhg is superior, even though remote-hg will clearly fix those
bugs, and add those features soon enough. I don't see why nobody tried
to contact git.git, cooperate, and improve what is clearly the
original code of gitifyhg.

>>> * added many new test cases, sadly still including some xfails. Several of these (both passing and xfailing) also apply to remote-hg (i.e. the issue is also present in contrib's remote-hg)
>>
>> I ran these test-cases with remote-hg, and the same test-cases pass. I
>> only had to do minor modifications, most of the failures came from
>> subtle differences such as different strategies to sanitize authors,
>> and which branch to pick for HEAD.
>
> Yeah, that's because what I wrote was based on the situation before your recent patch series. Glad to see that git/contrib's remote-hg is improving!

No, that's not true, this test run is from v1.8.2 without the latest
fix patches, simply the compatibility ones:

========================================================================
test session starts
========================================================================
platform linux2 -- Python 2.7.3 -- pytest-2.3.4
collected 80 items

test/test_author.py ........F
test/test_clone.py ......xx.........x...x..
test/test_notes.py ..FxF
test/test_pull.py ....x..xx..
test/test_push.py ...FFFFF.FF...x....FFFFFFFFF
test/test_special_cases.py ...

=============================================================================
FAILURES ==============================================================================
/home/felipec/tmp/gitifyhg/test/helpers.py:118: assert 'totally
<bad...e used in hg>' == 'totally <unknown>'
/usr/lib/python2.7/site-packages/sh.py:309: ErrorReturnCode_128:
/home/felipec/tmp/gitifyhg/test/test_notes.py:107: assert not 'error'
in 'From hg::file:///tmp/pytest-0/test_simple_push_updates_notes_after_contentful_pull0/hg_base\n
  66d7a5d..a8745f0  mas...rror: refs/notes/hg does not point to a
valid object!\nerror: refs/notes/hg-origin does not point to a valid
object!\n'
/home/felipec/tmp/gitifyhg/test/test_push.py:77: assert -1 > 0
/home/felipec/tmp/gitifyhg/test/test_push.py:86: assert -1 > 0
/home/felipec/tmp/gitifyhg/test/test_push.py:97: assert -1 > 0
/home/felipec/tmp/gitifyhg/test/test_push.py:117: assert 'default' ==
'branch_one'
/home/felipec/tmp/gitifyhg/test/test_push.py:136: assert 'default' ==
'branch one'
/home/felipec/tmp/gitifyhg/test/test_push.py:168: assert 'default' ==
'branch_one'
/home/felipec/tmp/gitifyhg/test/test_push.py:181: assert -1 > 0
/home/felipec/tmp/gitifyhg/test/test_push.py:334: assert 'anewbranch'
in 'no bookmarks set\n'
/home/felipec/tmp/gitifyhg/test/test_push.py:346: assert
'this_is_a_tag' in 'tip
0:1b3b36fc0158\n'
/home/felipec/tmp/gitifyhg/test/test_push.py:358: assert
'this_is_a_tag' in 'tip
1:7ca4e066c47f\n'
/home/felipec/tmp/gitifyhg/test/test_push.py:376: assert
'this_is_a_tag' in 'tip
2:5b8c7a98dfa4\nan_old_tag                         0:af90a856304c\n'
/home/felipec/tmp/gitifyhg/test/test_push.py:391: assert
'this_is_a_tag' in 'tip
0:3c2288c2c93b\n'
/home/felipec/tmp/gitifyhg/test/test_push.py:407: assert
'this_is_a_tag' in 'tip
1:090867df4300\n'
/home/felipec/tmp/gitifyhg/test/test_push.py:420: assert 'this is a
tag' in 'tip                                0:b7c5bfb3bdbc\n'
/home/felipec/tmp/gitifyhg/test/test_push.py:432: assert
'this_is_a_tag' in 'tip
1:10504139dfc4\nan_old_tag                         0:b7c5bfb3bdbc\n'
/usr/lib/python2.7/site-packages/sh.py:309: ErrorReturnCode_128:
========================================================= 19 failed,
52 passed, 9 xfailed in 74.35 seconds
==========================================================

The only real bugs is that tags are not pushed, and named branches are
wrong, the rest are differences in how errors are reported. I doubt
whomever ran these tests spent much time analyzing the results.
Certainly there's no way to think that remote-hg was completely broken
and couldn't be fixed to pass these.

>>> * improved handling of hg user names (remote-hg is not able to deal with some pathological cases, failing to import commits). Sadly, mercurial allows arbitrary strings as usernames, git doesn't...
>>
>> I wouldn't call it improved. In some cases the remote-hg result is
>> better, in others gitifyhg is,
>
> I'd love to learn about cases where remote-hg's result is better in your opinion, so that I can see if the mapping in gitifyhg could be improved for those cases.

For example "no email quoting email@example.com" results in "no email
quoting <email@example.com>", but I already have the code that does
that in remote-hg.

> That said, this part is really quite subjective I guess. In the end, since Mercurial names can be *anything*, one can never get a "perfect" mapping. Luckily, for most real repositories out there, user names will be quite sane and remote-hg and gitifyhg will produce identical results. (Although some hg repos out there have some really weird stuff going on. Yuck.)

Indeed, which is why I advocate for an author mapping solution, so if
the user is not happy, they can manually fix the wrong authors.
Moreover, the author sanitizing should be done by 'git fast-import',
so that people don't have to implement and re-implement the same
sanitizing code all the time. Also, the mapping should be part of 'git
fast-import' so it works the same for all solutions.

>>> * failed pushes to hg are cleanly rolled back (using mq.strip() from the mq extension), instead of resulting in inconsistent internal state. This is quite important in real life, and has bitten me several times with remote-hg (and was the initial reason why I switched to gitifyhg). A typical way to reproduce this is to push to a remote repository that has commits not yet in my local clone.
>>
>> This is not an issue in remote-hg any more since now we force the
>> push. It's not nice, but there's no other way to push multiple
>> bookmarks (aka git branches) to the same branch (aka commit label).
>
> Uhh, what? The push is forced? That sounds to me like a much, much bigger issue with remote-hg than anything else ever was, from my point of view. That seems tobe akin to making "--force" default to "git push", and I don't think anybody here would consider that a good idea.

It's absolutely nothing like that; with 'git push --force' you
**loose** commits, with 'hg push --force' you don't, you simply have
another head. It's mercurial's fault that multiple bookmarks cannot be
pushed any other way.

>> I doubt these inconsistent states can happen any more, but if they do,
>
> Seriously? This is triggered quite frequently in real life. And it will very likely cause somebody to mess up a hg repository they work on. As long is this in, using remote-hg is a total no-go for me. Just consider the following scenario:
> * user A clones a hg repository into a git repository
> * user A commits some commits in the git clone
> * meanwhile, user B pushes changes to the hg repository
> * user A tries to push his changes to the hg remote
>
> The last step causes this result in gitifyhg (similar to what one gets when the remote is a git repos):
>
> With remote-hg, you just force push the change, creating a new head in the remote repo. So, yeah, failed pushes which mess up the internal state don't happen anymore. But I rather have those than potentially mess up the upstream repository like that.

Unfortunately that's per mercurial design; you can't have git-like
branches any other way.

>> the plan in remote-hg is to simply ignore those revisions, and only
>> push the ones that have git refs. I have the code for that, but I'll
>> not be pushing it to git.git for the time being.
>
> I am not quite sure what you mean here, but I'll just wait for your code and hope it'll explain itself.

Even if you end with a dangling head because of a failed push, you can
do 'hg push -r good_head', and the bad head will be ignored.

>>> * git notes are used to associate to each git commit the sha1 of the corresponding hg commit, to help users figure out that mapping
>>
>> This is a minor feature. I've had the code for this for quite some
>> time, but for the moment I think there are higher priorities.
>
> Well, I guess it depends on how you use this. In my daily work with hg repositories, that's a quite important feature.

It's more important to fix clone, fetch and push failures.

> Actually, I wonder: Are you actually using remote-hg yourself for day-to-day development? Or for any other regular activities? You seem to have quite different usage patterns from me and other people involved with gitifyhg.

No, I don't, but plenty of people use hg-git, and I'd say remote-hg is
superior, because it does the same as hg-git, and more.

>>> * internally, the marks are using the hg sha1s instead of the hg rev ids. The latter are not necessarily invariant, and using the sha1s makes it much easier to recover from semi-broken states.
>>
>> I doubt this makes any difference (except for more wasted space).
>
> I disagree. Especially since remote-hg does treat local hg repositories differently than remote ones, by not cloning them but rather working with them directly. As soon as somebody uses "hg strip" or "hg graft", etc. on that, your mark files will contain incorrect data.

That might be correct, but doing so on a public repository is not a
good idea, it causes problems even in git.

When I'm done with other tasks I will do some tests to see for sure
what are the real issues, what is merely hypothetical, and what is
likely. For the moment worrying a workflow that is actually a bad
practice is not sensible.

> With a  remote hg repository, that is of course not as much of an issue, since you have a local hg clone stored inside of .git/. In theory, the user would never touch that, so nothing in there changes. In reality, though, sometimes the user still has to dig in there and modify things (e.g. to undo a "bad" push; although of course for now you "fixed" that particular problem by force pushing). Of course in an ideal world, a user never should have to do that, but as long as there are bugs, users may have to. And based on painful experience, it is *much* easier to do that when the marks use the invariant hashes instead of the transient rev ids.

How do you end up with a bad push in remote-hg?

> As for the wasted space: For a repository with 999,999 commits, the size of the "marks-hg file should increase from about 1.5 MB to about 5 MB. To use your words: Hardly worth mentioning.

It is a parenthesis (it was hardly mentioned).

>>> * Better handling of various hg errors, see e.g. [2]. More work is still needed there with both tools, though [3].
>>
>> This is literally a three lines fix, and it simply makes one error
>> nicer. Hardly worth mentioning.
>
> Perhaps not for you, but for users who just want to use remote-hg resp. gitifyhg, getting a helpful error message instead of seeing an (to them) unreadable stack trace is quite important. I guess this just shows that we have quite different goals and ideas about how to interact with users of our respective tools :-).

I have a suspicion that gitifyhg converts dates wrong, how is getting
a helpful error message on something the user would rarely (if ever)
do, more important than that (something the user always does)? We
might have different priorities, but I think mine are correct.

And don't speak about the "goals", because as I said I already have
the fix, saying we have different goals is implying that I don't want
to fix this issue, as if somehow not fixing a trivial error message
would not allow me to achieve my "goals". The priorities might be
different, but the goals are the same, and I think remote-hg is much
closer.

Also, anybody can send patches to the git mailing list, so _my_ goals,
and _my_ priorities shouldn't matter much.

>>> * The gitifyhg test suite is run after each push on Travis CI against several git / mercurial combinations [4].
>>> In particular, unlike all other remote-hg implementations I know, we explicitly promise (and test) compatibility with a specific range of Mercurial versions (not just the one the dev happens to have installed right now). This has been a frequent issue for me with the msysgit remote-hg
>>
>> I've personally checked against multiple versions of Mercurial. It's
>> possible that some error might slip by, but it would get quickly
>> noticed.
>
> Really? This sounds close to some people who say things like "I don't need a test suite, I personally run some tests every now and then on my machine."

Do you see any compatibility issues reported in the git mailing list,
or my github[1]? No? KTHXBYE. There _were_ compatibility issues, and
those got reported, and fixed, not any more.

Also, having CI testing doesn't ensure a project has better code, it's
the actual code that matters, and so far you haven't proved why
remote-hg's code would be inherently inferior. Moreover, gitifyhg
doesn't seem to have bidirectionality tests, remote-hg does, and
gitifyhg basically fails all of them, not to mention the ones that
compare the output with hg-git's, which has had far many more years of
development and testing, and their extensive tests have been harnessed
by remote-hg.

> Furthermore, you don't seem to document what versions of Mercurial are supposed to work / not work. (Indeed, as far as I can tell, remote-hg has no documentation whatsoever, another difference. Granted, the gitifyhg README is not particular great at this point, but at least it exists and tells people how to get started and where to get help).

When a user has a problem with that, we can deal with it. It's not
hard to do, and in fact it was done only one month ago in gitifyhg.
Nothing inherently different.

> In contrast, the gitifyhg README clearly states that it "requires at least Mercurial 1.9". And its setup.py refuses to install it if the Mercurial version is too old. Again, for us devs that's not very important, but for users, I think it is. In recent months, I had to provide assistance with using hg to tons of people (*sigh*), and old Mercurial versions came up in a considerable portion of those (perhaps 30-40% or so).

remote-hg certainly works on versions older than 1.9, again, I find it
annoying that you claim to know what is important for users, as if
somehow knowing that gitifyhg doesn't work with the user's version of
mercurial (e.g. 1.8) is better than remote-hg's situation; where it
*actually works*, but it's not mentioned. Yeah, mentioning that it
doesn't work is better than working, right.

>>> * Renaming a gitifyhg remote just works [5]. Doing that with remote-hg triggers a re-clone of the remote repository (if it works at all, I don't remember).
>>
>> Yeah, now you can change the alias of the remote, but you can't change
>> the remote url.
>
> That's simply wrong. You still can change the remote URL, it will just lead to the creation of a fresh separate local clone.

Exactly the same will happen with remote-hg when you change the alias.
Either creating a new local clone is "working", or it's not, you can't
have your cake and eat it at the same time. remote-hg reclones on
alias change, gitifyhg reclones on URL change. There is no advantage
here.

> In contrast, with remote-hg, renaming the remote will create a fresh local clone, while changing the remote URL will *not* do that -- instead, the changes from the new remote will be pulled into the existing local clone. Which in some cases may be exactly what you want (e.g. if a repository just moved to a different remote URL). But in other cases, it won't be, namely when the repository at the new URL contains different content. This is not that unlikely when you consider that it comment Mercurial workflow to use separate repositories for different branches.

It doesn't matter. The "extra content" that you get from the different
URL will just create that; extra content, both in the local git, and
mercurial repos. Since remote-hg will selectively push specific
revisions, that extra content will simply remain there, ignored.

>> This is not really an advantage, simply an almost
>> imperceptible different choice.
>
> These are indeed different design choices, but at least to me, they are very much perceptible :-). And we made this design choice quite consciously, after looking at how it is done in remote-hg, and not liking that.

This is not design, a few lines of code would change the behavior. And
you still haven't shown a single *important* use case where a user
would decide, you know what, yeah, I'll use gitifyhg because of that;
"I rename remote aliases a lot", "My upstream repo gets constantly
rebased" are very, very, unlikely phrases any user will ever utter.

And ultimately it doesn't matter, because if it's somehow decided that
one way is better than the other, we can switch it rather easily
(because they are not design decisions). So again, no, there's nothing
inherently superior here, merely subtle different behavior.

> Anyway: At least in my day-to-day operations, I occasionally rename a remote (very rare, but it happens). So far I never had to change a remote URL. Of course that is just me, perhaps others occasionally (well, more of than me :-) have to change remote URLs. But as I said, that's still possible in gitifyhg.

And I've changed the remote URL plenty of times, and again, changing
the alias works just fine as well.

>> I still don't see any good reason why a user might prefer gitifyhg,
>
> Well, for me it is exactly the other way around :-). That said, gitifyhg certainly also still has issues and problems, but I am confident it will even better in the coming weeks. And it certainly wouldn't be were it is now without your great work on remote-hg, and also on related improvements you got into git.git. Kudos!

You know what would be a proper way to thank? Mention where most of
the code and design came from in the README, and _at least_ mention
why you decided to work on your own rather than cooperate. When I
announced the work on remote-hg I discussed with the authors of other
solutions to try to explain why this work was needed until I
demonstrated that this was indeed superior, and I didn't even borrowed
code. I also provided a public summary when I announced it in my
blog[2].

>> even more importantly, why gitifyhg developers don't contribute to
>> remote-hg.
>
> I can only speak for myself, as a (minor) contributor to both remote-hg and gitifyhg. But here are my reasons why I prefer contributing to remote-hg over gitifyhg:
>
> 1) Apparent difference between your goals and mine / those of gitifyhg. I think this is quite visible if one looks at our exchange above. Features that are quite important to me, even crucial, are unimportant for you, or even outright rejected. In contrast, so far with the couple people working on gitifyhg we always managed to arrive at solutions that satisfy all of us. I.e. it seems our ideas and goals are much better aligned within that group.

Again, my goals are irrelevant, if you present a good case for a
patch, it will be merged by the maintainer, Junio, I'm not the
maintainer. But be prepared for argumentation from me, and others.

And no, I haven't rejected any patch, I couldn't reject a patch sent
to the git mailing list even if I wanted. I have only argued against
proposals, and if you give up when somebody argues against your ideas,
perhaps they were not so good in the first place.

But most of the "features" that you mentioned I said I ALREADY HAVE
THE PATCH FOR IT. So don't say we have different goals.

> 2) Lack of reactions on pull requests and bug reports on your github pages. Perhaps you never intended this to be used for pull requests / bug reports, but I never (until recently) saw you state that, nor can one read such a statement on your github pages.

I was not working on remote-hg. But it doesn't matter, git's
development happens in the git mailing list, and anybody can send
patches.

> 3) Not shipping gitifyhg as part of git/contrib but rather as a separate package is important to me, too. It means that we can make new releases whenever we want, not tied to git. Users can install it easily via "pip", and I also think it gets a lot more visibility this way.

% wget http://raw.github.com/felipec/git/fc/remote/hg/contrib/remote-helpers/git-remote-hg
-O ~/bin/git-remote-hg

There, you now have the latest and greatest version of remote-hg, you
don't need to wait for git.git to make a release. In fact, I'm in
contact with a few people that are having issues that are running
precisely this version; the latest and greatest.

>> Also, unlike remote-hg, which basically passes all the tests of
>> gitifyhg, gitifyhg barely passes any tests of remote-hg (three).
>
> Heh, bad, but OK (as I said, my message was based on an older version of remote-hg, and actually also on an older gitifyhg). Thank you for the report, I'll look into it as soon as I can (or somebody else might).
>
> BTW, I just pulled you hg-next branch, and run "make test" in that. The tests in test-hg-hg-git.sh actually all failed (with remote-hg). Do I need to do something special for those to work?

No, they should just work. Perhaps you have an old version of hg-git
(I have v0.4). You can check the errors with
GIT_TEST_OPTS="--verbose".

Cheers.

[1] https://github.com/felipec/git/issues
[2] http://felipec.wordpress.com/2012/11/13/git-remote-hg-bzr-2/

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Thu, Apr 4, 2013 at 12:42 AM, Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> On Wed, Apr 3, 2013 at 6:25 PM, Max Horn <max@quendi.de> wrote:
>> On 03.04.2013, at 03:31, Felipe Contreras wrote:
>
>>> I only learned about it recently, I've looked at the history and to me
>>> it seems rather chaotic, and a lot of the code was simply copied from
>>> git-remote-hg without comment.
>>
>> gitifyhg was scrapped and completely restarted from scratch at some point. Based largely on your git-remote-hg code. A bit more on its history can be read here:
>>   http://archlinux.me/dusty/2013/01/06/gitifyhg-rewritten/

Please don't CC the gitifyhg mailing list, unlike vger mailing lists
(or any other sane list), it doesn't accept mail from non-subscribers,
which makes communication with outsiders much more difficult, as
demonstrated by this.

---
We're writing to let you know that the group you tried to contact
(gitifyhg) may not exist, or you may not have permission to post
messages to the group. A few more details on why you weren't able to
post:

 * You might have spelled or formatted the group name incorrectly.
 * The owner of the group may have removed this group.
 * You may need to join the group before receiving permission to post.
 * This group may not be open to posting.
---

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Max Horn]

On 04.04.2013, at 08:46, Felipe Contreras wrote:

> On Thu, Apr 4, 2013 at 12:42 AM, Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
>> On Wed, Apr 3, 2013 at 6:25 PM, Max Horn <max@quendi.de> wrote:
>>> On 03.04.2013, at 03:31, Felipe Contreras wrote:
>> 
>>>> I only learned about it recently, I've looked at the history and to me
>>>> it seems rather chaotic, and a lot of the code was simply copied from
>>>> git-remote-hg without comment.
>>> 
>>> gitifyhg was scrapped and completely restarted from scratch at some point. Based largely on your git-remote-hg code. A bit more on its history can be read here:
>>>  http://archlinux.me/dusty/2013/01/06/gitifyhg-rewritten/
> 
> Please don't CC the gitifyhg mailing list, unlike vger mailing lists
> (or any other sane list), it doesn't accept mail from non-subscribers,
> which makes communication with outsiders much more difficult, as
> demonstrated by this.

I changed the settings of the gitifyhg list settings to accept emails from anybody.

Moreover, I would appreciate if you could refrain from injecting all those snide side remarks, such as the one you just needlessly made about how moderated mailing lists are insane.

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Thu, Apr 4, 2013 at 3:07 AM, Max Horn <max@quendi.de> wrote:
>
> On 04.04.2013, at 08:46, Felipe Contreras wrote:
>
>> On Thu, Apr 4, 2013 at 12:42 AM, Felipe Contreras
>> <felipe.contreras@gmail.com> wrote:
>>> On Wed, Apr 3, 2013 at 6:25 PM, Max Horn <max@quendi.de> wrote:
>>>> On 03.04.2013, at 03:31, Felipe Contreras wrote:
>>>
>>>>> I only learned about it recently, I've looked at the history and to me
>>>>> it seems rather chaotic, and a lot of the code was simply copied from
>>>>> git-remote-hg without comment.
>>>>
>>>> gitifyhg was scrapped and completely restarted from scratch at some point. Based largely on your git-remote-hg code. A bit more on its history can be read here:
>>>>  http://archlinux.me/dusty/2013/01/06/gitifyhg-rewritten/
>>
>> Please don't CC the gitifyhg mailing list, unlike vger mailing lists
>> (or any other sane list), it doesn't accept mail from non-subscribers,
>> which makes communication with outsiders much more difficult, as
>> demonstrated by this.
>
> I changed the settings of the gitifyhg list settings to accept emails from anybody.

Cool.

> Moreover, I would appreciate if you could refrain from injecting all those snide side remarks, such as the one you just needlessly made about how moderated mailing lists are insane.

I did not say that, gitifyhg's mailing list was not _moderated_, it
*automatically* rejected all non-subscriber email without any
_moderation_; that is insane in my opinion, but a lot of mailing lists
do that.

-- 
Felipe Contreras

Re: [PATCH 07/13] remote-hg: redirect buggy mercurial output

[Felipe Contreras]

Junio C Hamano wrote:
> Felipe Contreras <felipe.contreras@gmail.com> writes:
> 
> > On Tue, Apr 2, 2013 at 1:58 PM, Junio C Hamano <gitster@pobox.com> wrote:
> >> Felipe Contreras <felipe.contreras@gmail.com> writes:
> >>
> >>> We can't use stdout for that in remote helpers.
> >>>
> >>> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
> >>> ---
> >>
> >> You may want to clarify "buggy output" a bit.  Will mercurial
> >> forever be broken?  Some versions of Hg emit [[[it is unclear for
> >> Junio to tell what it is to fill this blank]]] to its output that
> >> we want to ignore?
> >
> > The problem is that mercurial's code is kind of hardcoded to run under
> > mercurial's UI, so it throws messages around willynillingly, like:
> >
> > searching for changes
> > no changes found
> >
> > And they can't be turned off. Theoretically we could override
> > mercurial's UI class, but I think that has the potential to create
> > more problems, it's not worth at this point in time.
> 
> Oh, I totally agree with you _after_ reading that explanation.
> 
> You just shouldn't let me waste your time to explain that to me in
> this exchange, and you could have done so by writing a clearer log
> message.  That's all.

I saw that you update the commit message without consulting here first to:

---
remote-hg: redirect unnecessary mercurial output
    
Mercurial emits messages like "searching for changes", "no changes
found", etc. meant for the use of its own UI layer, which is of no
use for our remote helper.  Squelch them.
---

This is not correct. This patch does _not_ squelch the output, it's redirecting
it to standard error, so the user actually sees it now, and we do that not
because the output is "unnecessary", but because it *breaks* the pipe between
the transport helper and remote helper. I'll reroll with the updated commit message.

Cheers.

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Tue, Apr 2, 2013 at 4:23 PM, Max Horn <max@quendi.de> wrote:

> I'll try to list some of remaining differences, mostly (in my biased opinion) improvements on the gitifyhg side. Note that some of these might be outdated with felipe's recent changes, i.e. I have not yet had time to review and/or test them all. So please bear that in mind.

I've implemented a lot of these, cleaned them up, and pushed them, the
ones that will be integrated:
http://github.com/felipec/git/tree/fc/remote/hg-next

The ones that won't (at least not without more discussion):
http://github.com/felipec/git/tree/fc/remote/hg-gitifyhg-compat

> * added many new test cases, sadly still including some xfails. Several of these (both passing and xfailing) also apply to remote-hg (i.e. the issue is also present in contrib's remote-hg)

I don't think there's anything inherently better about these tests,
with the compatibility patches here are the results on v0.8 running
remote-hg:

=========================================================== test
session starts ===========================================================
platform linux2 -- Python 2.7.3 -- pytest-2.3.4
collected 80 items

test/test_author.py ........F
test/test_clone.py ......xx.........x...x..
test/test_notes.py ..Fx.
test/test_pull.py ....x..xx..
test/test_push.py ..........F...x........FF...
test/test_special_cases.py ...

============================================= 5 failed, 66 passed, 9
xfailed in 75.52 seconds =============================================

> * improved handling of hg user names (remote-hg is not able to deal with some pathological cases, failing to import commits). Sadly, mercurial allows arbitrary strings as usernames, git doesn't...

This is not true; after checking the code, remote-hg can't possibly
fail, if it does, so does gitifyhg. I guarantee it. The only
differences are cosmetic.

That being said, I'll integrate a patch that I believe produces
superior sanitation than gitifyhg's, and passes the gitifyhg test (as
you can see above) (for the most part):

https://github.com/felipec/git/commit/c0e363915eb6459233e37d5082fb2ff7c7c727b4

> * failed pushes to hg are cleanly rolled back (using mq.strip() from the mq extension), instead of resulting in inconsistent internal state. This is quite important in real life, and has bitten me several times with remote-hg (and was the initial reason why I switched to gitifyhg). A typical way to reproduce this is to push to a remote repository that has commits not yet in my local clone.

After the change to force=true, let's see if this happens any more in
remote-hg (Doubt it).

> * git notes are used to associate to each git commit the sha1 of the corresponding hg commit, to help users figure out that mapping

Easy:
https://github.com/felipec/git/commit/2294fb445f5c018a39f421cba70e4d8510c04c89

I will not integrate this for the moment, there must be a better way
to interact with transport-helper to update these.

> * internally, the marks are using the hg sha1s instead of the hg rev ids. The latter are not necessarily invariant, and using the sha1s makes it much easier to recover from semi-broken states.

I will investigate the pros and cons of this, but either way it's not
something people are going to immediately need (I doubt the
semi-broken states will happen again).

> * Better handling of various hg errors, see e.g. [2]. More work is still needed there with both tools, though [3].

No idea why something so trivial was mentioned:
https://github.com/felipec/git/commit/d12e35d23b9d26d384c3dbbce25a09720ccbceff

> * Support for creating hg tags from git (i.e. pushing light git tags to heavy hg tags)

This was already merged to git.git:
https://git.kernel.org/cgit/git/git.git/commit/?id=32f370f62177b505daf96aaf711c0249d881b6c0

(link might change)

> * The gitifyhg test suite is run after each push on Travis CI against several git / mercurial combinations [4].
> In particular, unlike all other remote-hg implementations I know, we explicitly promise (and test) compatibility with a specific range of Mercurial versions (not just the one the dev happens to have installed right now). This has been a frequent issue for me with the msysgit remote-hg

This is nice, but doesn't translate necessarily to anything tangible
for the user. remote-hg, like all git.git, has good development
practices, which minimizes the risks of regressions.

> * Renaming a gitifyhg remote just works [5]. Doing that with remote-hg triggers a re-clone of the remote repository (if it works at all, I don't remember).

Changing a remote-hg URL remote just works. Potato potato.

Cheers.

-- 
Felipe Contreras

Re: [PATCH 07/13] remote-hg: redirect buggy mercurial output

[Junio C Hamano]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> I saw that you update the commit message without consulting here first to:
>
> ---
> remote-hg: redirect unnecessary mercurial output
>     
> Mercurial emits messages like "searching for changes", "no changes
> found", etc. meant for the use of its own UI layer, which is of no
> use for our remote helper.  Squelch them.
> ---
>
> This is not correct. This patch does _not_ squelch the output, it's redirecting
> it to standard error, so the user actually sees it now, and we do that not
> because the output is "unnecessary", but because it *breaks* the pipe between
> the transport helper and remote helper. I'll reroll with the updated commit message.

I actually "consulted" by asking you what you meant by "buggy".  I
just misread/misunderstood your response in prose.

An update in the patch form obviously would not risk such a
misunderstanding ;-)

Thanks.

Re: [PATCH 00/13] remote-hg: general updates

[Jed Brown]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> I still don't see any good reason why a user might prefer gitifyhg,
> even more importantly, why gitifyhg developers don't contribute to
> remote-hg.

Felipe, I read your blog announcement [1] and got the impression that
remote-hg was ready for daily use.  When I tried to use it, it promptly
crashed in my first attempt to clone.  I opened up the script, fixed
whatever caused the first stack trace and made it slighly further before
it crashed again.  I couldn't tell what was expected to work, what was a
known problem, and what was an unknown problem.  Many things clearly did
not work and it had the look of a project that was not getting active
use.  I felt that it was wildly oversold and that putting it into
git.git was premature.

I tried gitifyhg later and it basically worked out of the box.  All
known problems were marked by 'xfail' test cases.  At that time,
remote-hg failed almost all the gitifyhg tests.  I contributed a few
things to gitifyhg, including the notes support (essential when talking
via email with other people using Mercurial).  Since then, the last
major project I'm involved with has switched to Git so I rarely need
gitifyhg or remote-hg any more.



FWIW, I also thought Dusty's original announcement oversold gitifyhg, but
it was closer to the truth and upon cloning the repo, it was more clear
what didn't work.  The early history of gitifyhg is quite chaotic and I
didn't realize at first how much code turned out to be borrowed from
remote-hg.  I don't know whether you wrote all of remote-hg or borrowed
significant parts from elsewhere.  To be honest, I don't really care,
but it would be good to coalesce around one project that is well-tested
and has documented behavior so that the poor folks stuck with Mercurial
upstreams can have dependable behavior.

[1] http://felipec.wordpress.com/2012/11/13/git-remote-hg-bzr-2/

Re: [PATCH 00/13] remote-hg: general updates

[Junio C Hamano]

Jed Brown <jed@59A2.org> writes:

> ...  I felt that it was wildly oversold and that putting it into
> git.git was premature.
>
> I tried gitifyhg later and it basically worked out of the box.  All
> known problems were marked by 'xfail' test cases.  At that time,
> remote-hg failed almost all the gitifyhg tests.  I contributed a few
> things to gitifyhg, including the notes support (essential when talking
> via email with other people using Mercurial).  Since then, the last
> major project I'm involved with has switched to Git so I rarely need
> gitifyhg or remote-hg any more.
>
> FWIW, I also thought Dusty's original announcement oversold gitifyhg, but
> it was closer to the truth and upon cloning the repo, it was more clear
> what didn't work.

So,... is there a concrete proposal for _me_ to act on?  Do you want
to see contrib/remtote-hg out of my tree, and have it compete with
the other one (which also shouldn't be in my tree) in the open?

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Thu, Apr 4, 2013 at 12:11 PM, Jed Brown <jed@59a2.org> wrote:
> Felipe Contreras <felipe.contreras@gmail.com> writes:
>
>> I still don't see any good reason why a user might prefer gitifyhg,
>> even more importantly, why gitifyhg developers don't contribute to
>> remote-hg.
>
> Felipe, I read your blog announcement [1] and got the impression that
> remote-hg was ready for daily use.  When I tried to use it, it promptly
> crashed in my first attempt to clone.  I opened up the script, fixed
> whatever caused the first stack trace and made it slighly further before
> it crashed again.  I couldn't tell what was expected to work, what was a
> known problem, and what was an unknown problem.  Many things clearly did
> not work and it had the look of a project that was not getting active
> use.  I felt that it was wildly oversold and that putting it into
> git.git was premature.

Where is the evidence? You say remote-hg doesn't work, I say it does,
the difference is that I have evidence to prove it.

There's no bug report, no mail, no log of any kind, no repository to
check, no nothing. What are developers supposed to do with this
comment? Nothing, it's not constructive.

> I tried gitifyhg later and it basically worked out of the box.  All
> known problems were marked by 'xfail' test cases.  At that time,
> remote-hg failed almost all the gitifyhg tests.

Because gitifyhg tests are testing gitifyhg, not proper behavior. For example:

def test_push_conflict_default(git_dir, hg_repo):
    git_repo = clone_repo(git_dir, hg_repo)
    sh.cd(hg_repo)
    make_hg_commit("b")
    sh.cd(git_repo)
    make_git_commit("c")
    assert sh.git.push(_ok_code=1).stderr.find("master -> master
(non-fast-forward)") > 0

remote-hg doesn't fail with the non-fast-forward error, in fact, it
doesn't fail at all, it pushes correctly, and that's reported as a
failure.

> I contributed a few
> things to gitifyhg, including the notes support (essential when talking
> via email with other people using Mercurial).  Since then, the last
> major project I'm involved with has switched to Git so I rarely need
> gitifyhg or remote-hg any more.

So what is the purpose of this anecdote? Are you going to provide
something of value, or is this just an attempt to root for the project
you have contributed to?

> FWIW, I also thought Dusty's original announcement oversold gitifyhg, but
> it was closer to the truth and upon cloning the repo, it was more clear
> what didn't work.

This is *one* data-point, you can't make such overreaching
generalizations from *one* data-point; you will often be wrong.

> The early history of gitifyhg is quite chaotic and I
> didn't realize at first how much code turned out to be borrowed from
> remote-hg.  I don't know whether you wrote all of remote-hg or borrowed
> significant parts from elsewhere.  To be honest, I don't really care,
> but it would be good to coalesce around one project that is well-tested
> and has documented behavior so that the poor folks stuck with Mercurial
> upstreams can have dependable behavior.

Indeed, and mails like this don't help one iota to achieve that. We
need data, arguments, and code, and you are not providing any.

Now, if you can go fetch the log of the error you had, that would be
great, otherwise I'll turn to something productive, like improving
remote-hg.

Cheers.

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Jed Brown]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> Where is the evidence? You say remote-hg doesn't work, I say it does,
> the difference is that I have evidence to prove it.

There are many projects that don't do what they claim.  I gave remote-hg
a few minutes and moved on since (at the time) it didn't seem
interesting enough to be worth the effort of making proper bug reports.
There's a lot of low-quality code in the world and I'm willing to
tolerate a certain false-positive rate.  I apologize for misdiagnosing
your project and I'm happy to believe that you have since fixed the
bugs.  I was merely answering you question of why some of us contributed
to gitifyhg in preference to remote-hg.

I see no value in dwelling on the value judgement I made a few months
ago.  Additionally, I have almost no use for either project any more.

> remote-hg doesn't fail with the non-fast-forward error, in fact, it
> doesn't fail at all, it pushes correctly, and that's reported as a
> failure.

I don't agree that force-pushing by default is "correct" behavior.

Re: [PATCH 00/13] remote-hg: general updates

[Jed Brown]

Junio C Hamano <gitster@pobox.com> writes:

> So,... is there a concrete proposal for _me_ to act on?  Do you want
> to see contrib/remtote-hg out of my tree, and have it compete with
> the other one (which also shouldn't be in my tree) in the open?

Three months ago, I would have said yes.  Now I don't know.  It looks
like remote-hg has improved and is perhaps stable enough to remain, but
I think it needs a much more complete test suite [1] and some visible
documentation about its mapping semantics.  There is no way a change
like "force-push by default" should come without the user knowing about
it.  (I don't think force-push should become the default in any case,
but something is wrong with the process if there isn't a good way to
communicate such behavioral turmoil.)  A separate project making its own
releases has a more visible place to indicate such changes.

[1] Max and I have no love for the "obfuscated shell scripting" in
gitifyhg's 'py.test'- and 'sh'-based test suite, and we expressed early
on that we'd rather have git-style shell scripts.  So while porting
these would provide a good start, they can't just be dropped into
git.git.

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Thu, Apr 4, 2013 at 1:23 PM, Jed Brown <jed@59a2.org> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> So,... is there a concrete proposal for _me_ to act on?  Do you want
>> to see contrib/remtote-hg out of my tree, and have it compete with
>> the other one (which also shouldn't be in my tree) in the open?
>
> Three months ago, I would have said yes.  Now I don't know.  It looks
> like remote-hg has improved and is perhaps stable enough to remain,

Really? There have been 22 changes since 3 months ago. You think a
project goes from droppable to decent in 22 commits? The fact that you
hit a bug doesn't make a project unworthy. I thought you said you were
not going to dwell in your value judgement.

> but
> I think it needs a much more complete test suite [1] and some visible
> documentation about its mapping semantics.

I don't see anything particularly valuable in gitifyhg's test suite.
Show me a single test-case that remote-hg fails that would make sense
to port, then we'll talk.

> There is no way a change
> like "force-push by default" should come without the user knowing about
> it.  (I don't think force-push should become the default in any case,
> but something is wrong with the process if there isn't a good way to
> communicate such behavioral turmoil.)  A separate project making its own
> releases has a more visible place to indicate such changes.

Tell me, what is the worst case scenario a forced push will create?
What would be this terrible turmoil?

> [1] Max and I have no love for the "obfuscated shell scripting" in
> gitifyhg's 'py.test'- and 'sh'-based test suite, and we expressed early
> on that we'd rather have git-style shell scripts.  So while porting
> these would provide a good start, they can't just be dropped into
> git.git.

Oh yes they can, it's very easy to port (although annoying), I ported
quite a few of them because it was impossible to debug through py.test
(because stderr was incompletely shown), so I was forced to do that,
but again, I didn't see much value in them and deleted them. Maybe if
we had some complex striping on errors, but not at the moment.

I would love to be proven wrong; show me a single test that is so
sorely needed in remote-hg.

Cheers.

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

Antoine Pelisse wrote:
> >> * internally, the marks are using the hg sha1s instead of the hg rev ids. The latter are not necessarily invariant, and using the sha1s makes it much easier to recover from semi-broken states.
> >
> > I doubt this makes any difference (except for more wasted space).
> 
> I think this is definitely wrong. If you happen to strip a changeset
> from the mercurial repository, and redo a completely different commit
> on top of it, the new commit will never be seen on git end (because it
> will have the same rev id and will thus be identified as identical
> from git point of view).

That is true. I've written the code to use SHA-1 nodeids inststead andd this
particular scenario works correctly, I just need to figure a way to discard the
old ones.

Cheers.

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Thu, Apr 4, 2013 at 10:14 AM, Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> On Tue, Apr 2, 2013 at 4:23 PM, Max Horn <max@quendi.de> wrote:

>> * internally, the marks are using the hg sha1s instead of the hg rev ids. The latter are not necessarily invariant, and using the sha1s makes it much easier to recover from semi-broken states.
>
> I will investigate the pros and cons of this, but either way it's not
> something people are going to immediately need (I doubt the
> semi-broken states will happen again).

And another one bites the dust:
https://github.com/felipec/git/commits/fc/remote/hg-noteids

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Max Horn]

<RANT>
While I am not really interested in exchanging any further emails or any other form of communication with Felipe, as I find his vitriolic style of communication unbearable, I feel compelled to reply to a few points. I'll probably regret this... anyway, I promise this will be my last mail in this thread. Even though I fully expect to receive a barrage of hostile and aggressive replies by Felipe. So be it, /dev/null has plenty space left.
</RANT>

OK, I'll try to keep a professional tone from now on :-).


On 04.04.2013, at 08:42, Felipe Contreras wrote:

[...]

>>>> * The gitifyhg test suite is run after each push on Travis CI against several git / mercurial combinations [4].
>>>> In particular, unlike all other remote-hg implementations I know, we explicitly promise (and test) compatibility with a specific range of Mercurial versions (not just the one the dev happens to have installed right now). This has been a frequent issue for me with the msysgit remote-hg
>>> 
>>> I've personally checked against multiple versions of Mercurial. It's
>>> possible that some error might slip by, but it would get quickly
>>> noticed.
>> 
>> Really? This sounds close to some people who say things like "I don't need a test suite, I personally run some tests every now and then on my machine."
> 
> Do you see any compatibility issues reported in the git mailing list,
> or my github[1]? No? KTHXBYE. There _were_ compatibility issues, and
> those got reported, and fixed, not any more.

Please consider that the willingness of people to collaborate with you in any way is directly related to how you treat them. That includes bug reports. The way you acted towards Jed, who was very calmly and matter-of-factly explaining things, was IMHO completely inappropriate and unacceptable. Indeed, I should augment my list of reasons why people might not want to contribute to remote-hg by one major bullet point: You. And please, don't feel to compelled to tell us that Junio is really the maintainer of remote-hg and not you: Whether this is true or not doesn't matter for this point.


> remote-hg certainly works on versions older than 1.9, again

That's plain wrong. Indeed, remote-hg is using hg interfaces that were only introduced in 1.9. As such, I would be quite surprised if remote-hg worked with older hg versions, but I don't see why I should bother to test... Hmm, wait, I see a reason:

> , I find it
> annoying that you claim to know what is important for users, as if
> somehow knowing that gitifyhg doesn't work with the user's version of
> mercurial (e.g. 1.8) is better than remote-hg's situation; where it
> *actually works*, but it's not mentioned. Yeah, mentioning that it
> doesn't work is better than working, right.

I'll leave it to everybody to read what you wrote there, and contrast it with the following, and draw their own conclusions...

The reason why I did not write what exactly is wrong with remote-hg in hg 1.8 and older is that it is so obvious that I didn't think anybody would need handholding to verify it or find out the details :-). But since you "asked" for it:


$ hg --version
Mercurial Distributed SCM (version 1.8.4)
(see http://mercurial.selenic.com for more information)

Copyright (C) 2005-2011 Matt Mackall and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ git clone hg::foobar/ foobar.git
Cloning into 'foobar.git'...
Traceback (most recent call last):
  File "/Users/mhorn/bin/git-remote-hg", line 846, in <module>
    sys.exit(main(sys.argv))
  File "/Users/mhorn/bin/git-remote-hg", line 819, in main
    fix_path(alias, peer or repo, url)
  File "/Users/mhorn/bin/git-remote-hg", line 765, in fix_path
    repo_url = util.url(repo.url())
AttributeError: 'module' object has no attribute 'url'

Indeed, util.url was only added in 1.9.3. OK, let's work around that. Then local clones work. But of course in reality, most users will want to interact with a remote repository. But that's still broken:

$ git clone hg::ssh://hg@bitbucket.org/fingolfin/test-gitifyhg
Cloning into 'test-gitifyhg'...
Traceback (most recent call last):
  File "/Users/mhorn/bin/git-remote-hg", line 1138, in <module>
    sys.exit(main(sys.argv))
  File "/Users/mhorn/bin/git-remote-hg", line 1107, in main
    repo = get_repo(url, alias)
  File "/Users/mhorn/bin/git-remote-hg", line 284, in get_repo
    peer, dstpeer = hg.clone(myui, {}, url, local_path, update=False, pull=True)
TypeError: clone() got multiple values for keyword argument 'pull'


Right, clone() changed. And some more stuff. See <https://github.com/fingolfin/gitifyhg/commit/d3d37a7a853f3c8a1907bdaf933844128d5e7d81>. 

There also was a good reason why I stopped at that point, but I don't remember the details right now. And quite frankly, I have zero incentive to even try to remember. But anyway, I don't think it's that useful to add support for 1.8, too, since one can't get back much further anyway. And upgrading to a newer Mercurial is (a) quite easy (even if you don't have root, just install it into $HOME), and (b) using a newer Mercurial version is a good idea for other reasons, too.

[...]

> 
>>> Also, unlike remote-hg, which basically passes all the tests of
>>> gitifyhg, gitifyhg barely passes any tests of remote-hg (three).
>> 
>> Heh, bad, but OK (as I said, my message was based on an older version of remote-hg, and actually also on an older gitifyhg). Thank you for the report, I'll look into it as soon as I can (or somebody else might).
>> 
>> BTW, I just pulled you hg-next branch, and run "make test" in that. The tests in test-hg-hg-git.sh actually all failed (with remote-hg). Do I need to do something special for those to work?
> 
> No, they should just work. Perhaps you have an old version of hg-git
> (I have v0.4). You can check the errors with
> GIT_TEST_OPTS="--verbose".

Yup, that was it, thanks. It would be kinda nice if the test code could check for suitable versions of mercurial and hg-git (and python, I guess) and warn the user if necessary.


As for your complaints about not getting proper credit in the gitifyhg README etc.: I agree that it is very much lacking in this regard, and will work towards rectifying this (indeed, I already suggested this to the other gitifyhg devs).

I'll also look into using sharness for gitifyhg test (which is based on the git test suite), as I also don't like the current test setup in gitifyhg, and indeed, the other gitifyhg devs agree. This would also make it easier to share and compare tests between remote-hg and gitifyhg if desired.

I won't reply to all the other stuff you wrote, as it just causes too much bile to raise up re-reading it, and I am not sure I could manage to reply in an even remotely neutral tone. So I'll refrain from attempting, as I am not interested in a fight between the two projects, or anybody for that matter. Nor do I see this as a competition where the "best" wins -- if somebody prefers remote-hg over gitifyhg, or the other way around, I don't care much, as long as at least one of the tools satisfy their needs. 

Rather, all *I* am interested is using git to access a couple hg repositories that I absolutely must access. And helping several friend and colleagues who are in the precise same situation. Well, Jed and me already explained why e.g. forced push makes remote-hg an absolute no-go for me and several other people. Whether you accept this or not is irrelevant -- it sadly won't change the reality I and others have to deal with at work and elsewhere.


Cheers,
Max

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Fri, Apr 5, 2013 at 4:30 PM, Max Horn <max@quendi.de> wrote:

> On 04.04.2013, at 08:42, Felipe Contreras wrote:

> Please consider that the willingness of people to collaborate with you in any way is directly related to how you treat them. That includes bug reports. The way you acted towards Jed, who was very calmly and matter-of-factly explaining things, was IMHO completely inappropriate and unacceptable

He did not provide any bug report at all, also, he was not willing to
collaborate, as he clearly stated; he won't work git-hg bridges no
more.

> Indeed, I should augment my list of reasons why people might not want to contribute to remote-hg by one major bullet point: You. And please, don't feel to compelled to tell us that Junio is really the maintainer of remote-hg and not you: Whether this is true or not doesn't matter for this point.

Oh but it does. The only reason my remote-hg managed to land in the
mainline is because I showed it was superior to the other
remote-hg[1], and that there really was no point in trying to salvage
the other code base. Maybe this turns out to be a mistake, and the
other remote-hg manages to improve and surpass my remote-hg, but it
seems rather unlikely at this point.

Similarly, you could *show* that gitifyhg is superior than remote-hg,
why it makes sense to use that instead of trying to improve this code
base, but you don't, because you can't, because it doesn't make sense.

Ultimately this is not about people, this is about the code. A
sensible person that is not emotionally attached to any code, would
simply look at the code, and if what you claim is true (that remote-hg
has many issues), this sensible person would start cherry picking
patches from gitifyhg, send them to the git mailing list explaining
why they make sense. Eventually, remote-hg in git.git would become
essentially gitifyhg (albeit better because it would have received
more review from other git developers). Codewise this would make
sense.

And this sensible person would replace me as the go-to person for
remote-hg. And he would probably have a friendly and wonderful
personality, and you can accept him as your messiah and what not. But
this wouldn't change the fact that codewise this is the best way to
proceed.

But the fact of the matter is that either a) remote-hg is perfectly
fine, or b) this sensible person doesn't exist.

>> , I find it
>> annoying that you claim to know what is important for users, as if
>> somehow knowing that gitifyhg doesn't work with the user's version of
>> mercurial (e.g. 1.8) is better than remote-hg's situation; where it
>> *actually works*, but it's not mentioned. Yeah, mentioning that it
>> doesn't work is better than working, right.
>
> I'll leave it to everybody to read what you wrote there, and contrast it with the following, and draw their own conclusions...

Keep in mind that it's not me the one that claims remote-hg is
superior because it supports 1.8, as you seem try to portray. Rather,
it's _you_ the one that claims superiority because gitifyhg *mentions*
support until 1.9 (which remote-hg also supports).

It's not important who supports the most ancient version of mercurial
that nobody uses anyway, what is important is that *mentioning* or not
mentioning which ancient version of mercurial is supported doesn't
make one project superior to the other.

But perhaps this point is too subtle for you to understand, so there:
https://github.com/felipec/git/wiki/Git-remote-hg

Now it's mentioned that remote-hg too, supports up to version 1.9. Can
we agree now, that nobody has any advantage, either on version
support, or the mentioning of version support?

> Indeed, util.url was only added in 1.9.3. OK, let's work around that. Then local clones work. But of course in reality, most users will want to interact with a remote repository. But that's still broken:

Fixed:
https://github.com/felipec/git/commit/df0ed732b56c6c7910cc76f3e930229816e27117

And it was _you_ the one that sent the commit that broke it to the git
mailing list to be pushed.

> Right, clone() changed. And some more stuff. See <https://github.com/fingolfin/gitifyhg/commit/d3d37a7a853f3c8a1907bdaf933844128d5e7d81>.

Indeed, that might be useful, but that doesn't mean remote-hg doesn't
work with 1.8; it works perfectly fine for local repositories, and all
the tests pass.

> There also was a good reason why I stopped at that point, but I don't remember the details right now. And quite frankly, I have zero incentive to even try to remember. But anyway, I don't think it's that useful to add support for 1.8, too, since one can't get back much further anyway. And upgrading to a newer Mercurial is (a) quite easy (even if you don't have root, just install it into $HOME), and (b) using a newer Mercurial version is a good idea for other reasons, too.

If supporting 1.8 is not useful because most people have newer
versions, then mentioning support for 1.9 doesn't give any advantage.
Thanks for wasting our time on something nobody cares anyway.

>>> BTW, I just pulled you hg-next branch, and run "make test" in that. The tests in test-hg-hg-git.sh actually all failed (with remote-hg). Do I need to do something special for those to work?
>>
>> No, they should just work. Perhaps you have an old version of hg-git
>> (I have v0.4). You can check the errors with
>> GIT_TEST_OPTS="--verbose".
>
> Yup, that was it, thanks. It would be kinda nice if the test code could check for suitable versions of mercurial and hg-git (and python, I guess) and warn the user if necessary.

Yes, but how many people run these tests? And how many people have
hg-git in the first place? And how many people care about
test-hg-hg-git? I'll guess not so many, so I'm not really looking
forward to see what are the issues with older versions of hg-git.

> As for your complaints about not getting proper credit in the gitifyhg README etc.: I agree that it is very much lacking in this regard, and will work towards rectifying this (indeed, I already suggested this to the other gitifyhg devs).

Thanks.

> I'll also look into using sharness for gitifyhg test (which is based on the git test suite), as I also don't like the current test setup in gitifyhg, and indeed, the other gitifyhg devs agree. This would also make it easier to share and compare tests between remote-hg and gitifyhg if desired.

That would be really good.

> Jed and me already explained why e.g. forced push makes remote-hg an absolute no-go for me and several other people.

% git config remote-hg.force-push false

Problem solved. You don't need to work on a another project to avoid this.

> Whether you accept this or not is irrelevant -- it sadly won't change the reality I and others have to deal with at work and elsewhere.

The reality is you lack imagination. I've yet to see a single argument
that shows a problem with using a dedicated branch to push bookmarks.

> I won't reply to all the other stuff you wrote, as it just causes too much bile to raise up re-reading it, and I am not sure I could manage to reply in an even remotely neutral tone. So I'll refrain from attempting, as I am not interested in a fight between the two projects, or anybody for that matter. Nor do I see this as a competition where the "best" wins -- if somebody prefers remote-hg over gitifyhg, or the other way around, I don't care much, as long as at least one of the tools satisfy their needs.
>
> Rather, all *I* am interested is using git to access a couple hg repositories that I absolutely must access. And helping several friend and colleagues who are in the precise same situation.

So you are unable to argue why is it that gitifyhg is necessary, or
good. I take this as evidence that you are too emotionally invested to
accept the possibility that there's no good reason for the existence
of gitifyhg.

gitifyhg is not technically, or in any way, superior to remote-hg. You
should just give it up, and try to move as much code as possible to
remote-hg (is there any that will be useful?), and perhaps maintain a
branch to make remote-hg behave as you want (I already provided an
example), but on top of remote-hg. You can have your own github repo,
with your own wiki, and your own releases as you have it now. Then it
would be easier to share code, and if gitifyhg turns out to be truly
superior, it would be easier to simply merge those patches that enable
gitifyhg behavior, which you claim make all the difference in the
world.

It would be great if we could all work together in remote-hg, but
given the fact that at no point in time did you consider to "fix"
remote-hg, and that you spend considerable amounts of time claiming
that gitifyhg is superior, specially in remote-hg's own bug tracker,
rather than sending patches for remote-hg, which would prove your
point much more easily (if it was correct), I don't think you are
allowing for that possibility.

Cheers.

[1] https://github.com/msysgit/msysgit/wiki/Guide-to-git-remote-hg

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Junio C Hamano]

Max Horn <max@quendi.de> writes:

> OK, I'll try to keep a professional tone from now on :-).
>
> Please consider that the willingness of people to collaborate with
> you in any way is directly related to how you treat them. That
> includes bug reports. The way you acted towards Jed, who was very
> calmly and matter-of-factly explaining things, was IMHO completely
> inappropriate and unacceptable. Indeed, I should augment my list
> of reasons why people might not want to contribute to remote-hg by
> one major bullet point: You. And please, don't feel to compelled
> to tell us that Junio is really the maintainer of remote-hg and
> not you: Whether this is true or not doesn't matter for this
> point.

Only on this point, as the top-level maintainer.  I do not have any
opinion on technical merits between the two Hg gateways myself.

A tool that is in contrib/ follows the contrib/README rule.

I do not maintain it. Maintenance is up to the person who asked to
include it there.  I do ask the people who propose to add something
in contrib/ to promise that they arrange it to be maintained.

I do not even guarantee that they are the best in the breed in their
respective category. When something is added to contrib/, others can
raise objections by proposing alternatives, by arguing that tools of
the nature are better kept out of my tree, etc.  When remote-hg was
added, I didn't see specific objections against it.

There is one generic objection to adding anything new in contrib/ I
have myself, though.

In early days of Git, almost all users, who might be interested in
improving their Git experience by helping to polish third-party
tools, had clones of my tree and did not hesitate to come to this
list. Back then, having a copy of an emerging third-party tool in my
tree in contrib/ was a good way to give more exposure to it, and to
give those interested in it a place to meet and join forces to
improve it. Because Git population was small, almost everybody was
here, and it was an efficient distribution mechanism.

Git is now reasonably well known and has big enough user base, and
many users, even those who are inclined to help improving their Git
experience by contributing to third-party tools, do not necessarily
have a clone of my tree.  A third-party tool around Git, if it is
any good, is likely to have much much better chance to thrive as a
free-standing project with its own community, compared to those
early days.

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Fri, Apr 5, 2013 at 7:28 PM, Junio C Hamano <gitster@pobox.com> wrote:

> A tool that is in contrib/ follows the contrib/README rule.
>
> I do not maintain it. Maintenance is up to the person who asked to
> include it there.  I do ask the people who propose to add something
> in contrib/ to promise that they arrange it to be maintained.

That's true, but I meant that you are the gatekeeper. Ultimately you
decide which patches go in. If Max, or anybody else, wants a patch
into contrib, you can get it in, even if I disagree with it. You can
also decide that somebody from gitifyhg might be a more suitable
person as a maintainer of remote-hg.

Either way, I think if things go well, remote-hg will prove it's worth
and move out of contrib and into git's core.

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Philip Oakley]

From: "Felipe Contreras" <felipe.contreras@gmail.com>
Sent: Saturday, April 06, 2013 1:45 AM
> On Fri, Apr 5, 2013 at 4:30 PM, Max Horn <max@quendi.de> wrote:
>
>> On 04.04.2013, at 08:42, Felipe Contreras wrote:
>
>> Please consider [...]

> Ultimately this is not about people, this is about the code.

In the case of helper functions this is not the case.

The question would be better framed:
"Does this, or that, helper function make users (people) feel helped, or 
frustrated (or somewhere in limbo)?".

I've called IT help desks and often felt frustrated, and some times I've 
got one of the good girls/guys who worked with me to improve my 
situation (often despite official policies). I get back to those folks 
(even if they 'failed').

It's not a binary black/white issue when real users need help. It's no 
good keeping with the faith (e.g. the Git ideal, the coders ideal, ..) 
when the users (a mixed group) environmental doctine differs.

>    A sensible person that is not emotionally attached to any code,
  [I'm thinking users here, they are emotionally attached to their 
original problem, and sense doesn't come into it]

>  would simply look at the code,
Unfortunately, even for reasonable coders, looking at the code isn't 
usually the case because of lack of time, unfamiliarity with the code, 
extent of the code, availablity of the code (they may be simply running 
a packaged/compiled 'app'), this is not that likely to happen. We should 
be thankful when folk do look.

It's hard enough to get "good" bug reports from fellow coders (they are 
only human / no more human than us) that tell us what _we_ want to know 
(rather than what _they_ remember, or was important to them). ;-)


I don't use Hg, but as I read the discussion, there are 
incomaptibilities between Git, and Hg. Thus neither helper can ever be 
perfect. The winners will be those who solve a user need with enough 
documentation and error capture to make them (their user group) feel 
happy. At the moment it looks like the discussion is stratifying into 
various "it worked for me" camps, each with their own problem children 
repos that won't respond to parental advice, even with a --force from 
social services.

Philip
[As they say back home: Between thee and me, ther's nowt so queer as 
fowk, and I ain't so sure about thee] 

Re: [PATCH 00/13] remote-hg: general updates

[Felipe Contreras]

On Sat, Apr 6, 2013 at 8:09 AM, Philip Oakley <philipoakley@iee.org> wrote:
> From: "Felipe Contreras" <felipe.contreras@gmail.com>
> Sent: Saturday, April 06, 2013 1:45 AM

>> Ultimately this is not about people, this is about the code.
>
>
> In the case of helper functions this is not the case.
>
> The question would be better framed:
> "Does this, or that, helper function make users (people) feel helped, or
> frustrated (or somewhere in limbo)?".

And that question is ultimately answered by code.

> I've called IT help desks and often felt frustrated, and some times I've got
> one of the good girls/guys who worked with me to improve my situation (often
> despite official policies). I get back to those folks (even if they
> 'failed').

That is not an issue when you don't have a situation that needs
support, when everything just works. Having one issue with bad support
is better than having 5 issues with good support.

> It's not a binary black/white issue when real users need help. It's no good
> keeping with the faith (e.g. the Git ideal, the coders ideal, ..) when the
> users (a mixed group) environmental doctine differs.

And neither Max or Jed are users, so this doesn't apply to them.

As for actual users, well show me, which remote-hg users have had bad support?

https://github.com/felipec/git/issues

>>    A sensible person that is not emotionally attached to any code,
>
>  [I'm thinking users here, they are emotionally attached to their original
> problem, and sense doesn't come into it]

That's irrelevant, a sensible developer would maximize the amount of
issues that do **not** hit the users.

>>  would simply look at the code,
>
> Unfortunately, even for reasonable coders, looking at the code isn't usually
> the case because of lack of time, unfamiliarity with the code, extent of the
> code, availablity of the code (they may be simply running a
> packaged/compiled 'app'), this is not that likely to happen. We should be
> thankful when folk do look.

If this sensible developer doesn't have time to look at remote-hg
code, he has less time to make a possibly inferior alternative work on
par or better. Thus a sensible developer would either look at
remote-hg code, or do not develop.

> It's hard enough to get "good" bug reports from fellow coders (they are only
> human / no more human than us) that tell us what _we_ want to know (rather
> than what _they_ remember, or was important to them). ;-)

Nobody is asking for that.

> I don't use Hg, but as I read the discussion, there are incomaptibilities
> between Git, and Hg. Thus neither helper can ever be perfect. The winners
> will be those who solve a user need with enough documentation and error
> capture to make them (their user group) feel happy. At the moment it looks
> like the discussion is stratifying into various "it worked for me" camps,
> each with their own problem children repos that won't respond to parental
> advice, even with a --force from social services.

I don't think so. The discussion has been about hypothetical, not real
users. The one person that did claim was hit by a bug, had no evidence
for it nor cared, so it hardly matters. The rest is in pondering such
as if the user does this and that, and somebody else in the team does
that, and then the user does this, and the team has that policy, they
might end in a bit of a kerfuffle. Not something that has _actually_
happened.

And I'm confident that with time it will be shown that in terms of
real issues, remote-hg would have less, and be fixed faster.

Cheers.

-- 
Felipe Contreras

Re: [PATCH 00/13] remote-hg: general updates

[Junio C Hamano]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> On Fri, Apr 5, 2013 at 7:28 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
>> A tool that is in contrib/ follows the contrib/README rule.
>>
>> I do not maintain it. Maintenance is up to the person who asked to
>> include it there.  I do ask the people who propose to add something
>> in contrib/ to promise that they arrange it to be maintained.
>
> That's true, but I meant that you are the gatekeeper. Ultimately you
> decide which patches go in. If Max, or anybody else, wants a patch
> into contrib, you can get it in, even if I disagree with it.

In an ideal fantasy world, it could be true, but when I say "I do
not maintain it, and people who put it in contrib/ is responsible
for it", I really mean it.

I may find the behaviour/performance of the sub-maintainer of a part
in contrib/ unsatisfactory and have to take an administrative action
(e.g. to remove the problematic part out of contrib/), but I would
rather not to do that kind of thing, if possible.  Instead I expect
people who have any code in my tree (even in contrib/) to act as a
responsible adult, taking and responding to constructive criticisms
well, and more importantly, nudging those whose utterance you find
are mostly noise into raising a more concrete and actionable issues
that would be useful to you as an input.

> Either way, I think if things go well, remote-hg will prove it's worth
> and move out of contrib and into git's core.

That was what you promised when we started carrying it in contrib/;
I am still hoping to see it happen when it matures.