[PATCH 0/8] remote-helper: fixes

[PATCH 0/8] remote-helper: fixes

[Felipe Contreras]

A few trivial updates, a few important fixes.


Daniel Liew (1):
  git-remote-hg: use internal clone's hgrc

Felipe Contreras (7):
  remote-hg: fix parsing of custom committer
  remote-hg: update to 'public' phase when pushing
  remote-{hg,bzr}: store marks only on success
  remote-hg: properly detect missing contexts
  t: remote-hg: split into setup test
  remote-hg: make sure we omit multiple heads
  remote-hg: trivial cleanups

 git-remote-bzr.py    |  8 +++---
 git-remote-hg.py     | 72 ++++++++++++++++++++++++++++++++++++++++++----------
 t/t5810-remote-hg.sh | 65 +++++++----------------------------------------
 3 files changed, 70 insertions(+), 75 deletions(-)

-- 
1.9.2+fc1.3.gade8541

[PATCH 1/8] remote-hg: fix parsing of custom committer

[Felipe Contreras]

Other tools use the 'committer' extra field differently, so let's make
the parsing more reliable and don't assume it's in a certain format.

Reported-by: Kevin Cox <kevincox@kevincox.ca>
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/git-remote-hg.py b/git-remote-hg.py
index 34cda02..c849abb 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -461,8 +461,12 @@ def export_ref(repo, name, kind, head):
 
         author = "%s %d %s" % (fixup_user(user), time, gittz(tz))
         if 'committer' in extra:
-            user, time, tz = extra['committer'].rsplit(' ', 2)
-            committer = "%s %s %s" % (user, time, gittz(int(tz)))
+            try:
+                cuser, ctime, ctz = extra['committer'].rsplit(' ', 2)
+                committer = "%s %s %s" % (cuser, ctime, gittz(int(ctz)))
+            except ValueError:
+                cuser = extra['committer']
+                committer = "%s %d %s" % (fixup_user(cuser), time, gittz(tz))
         else:
             committer = author
 
-- 
1.9.2+fc1.3.gade8541

[PATCH 2/8] remote-hg: update to 'public' phase when pushing

[Felipe Contreras]

This is what Mercurial does.

Reported-by: Nathan Palmer
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/git-remote-hg.py b/git-remote-hg.py
index c849abb..204ceeb 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -995,9 +995,17 @@ def push_unsafe(repo, remote, parsed_refs, p_revs):
     if unbundle:
         if force:
             remoteheads = ['force']
-        return remote.unbundle(cg, remoteheads, 'push')
+        ret = remote.unbundle(cg, remoteheads, 'push')
     else:
-        return remote.addchangegroup(cg, 'push', repo.url())
+        ret = remote.addchangegroup(cg, 'push', repo.url())
+
+    phases = remote.listkeys('phases')
+    if phases:
+        for head in p_revs:
+            # update to public
+            remote.pushkey('phases', hghex(head), '1', '0')
+
+    return ret
 
 def push(repo, remote, parsed_refs, p_revs):
     if hasattr(remote, 'canpush') and not remote.canpush():
-- 
1.9.2+fc1.3.gade8541

[PATCH 3/8] remote-{hg,bzr}: store marks only on success

[Felipe Contreras]

Commit 2594a79 (remote-hg: fix bad state issue) originally introduced
this code in order to avoid synchronization issues while pushing,
because `git fast-export` might end up writing the marks before a crash
in the remote helper occurs.

However, the problem is in `git fast-export`; the marks should only be
written after both have finished successfully.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-bzr.py | 8 +++-----
 git-remote-hg.py  | 8 +++-----
 2 files changed, 6 insertions(+), 10 deletions(-)

diff --git a/git-remote-bzr.py b/git-remote-bzr.py
index 9abb58e..71b138e 100755
--- a/git-remote-bzr.py
+++ b/git-remote-bzr.py
@@ -971,12 +971,10 @@ def main(args):
             die('unhandled command: %s' % line)
         sys.stdout.flush()
 
+    marks.store()
+
 def bye():
-    if not marks:
-        return
-    if not is_tmp:
-        marks.store()
-    else:
+    if is_tmp:
         shutil.rmtree(dirname)
 
 atexit.register(bye)
diff --git a/git-remote-hg.py b/git-remote-hg.py
index 204ceeb..8aca6dd 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -1258,12 +1258,10 @@ def main(args):
             die('unhandled command: %s' % line)
         sys.stdout.flush()
 
+    marks.store()
+
 def bye():
-    if not marks:
-        return
-    if not is_tmp:
-        marks.store()
-    else:
+    if is_tmp:
         shutil.rmtree(dirname)
 
 atexit.register(bye)
-- 
1.9.2+fc1.3.gade8541

[PATCH 4/8] remote-hg: properly detect missing contexts

[Felipe Contreras]

This can happen when there's a synchronization issue between marks-git
and marks-hg; a key is missing in marks-hg, and when we receive a reset
command the value of ctx basically comes from None.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/git-remote-hg.py b/git-remote-hg.py
index 8aca6dd..1972f7f 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -917,6 +917,11 @@ def checkheads_bmark(repo, ref, ctx):
 
     ctx_old = bmarks[bmark]
     ctx_new = ctx
+
+    if not ctx.rev():
+        print "error %s unknown" % ref
+        return False
+
     if not repo.changelog.descendant(ctx_old.rev(), ctx_new.rev()):
         if force_push:
             print "ok %s forced update" % ref
-- 
1.9.2+fc1.3.gade8541

[PATCH 5/8] t: remote-hg: split into setup test

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 t/t5810-remote-hg.sh | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/t/t5810-remote-hg.sh b/t/t5810-remote-hg.sh
index 594a0a1..ba8b2d8 100755
--- a/t/t5810-remote-hg.sh
+++ b/t/t5810-remote-hg.sh
@@ -105,17 +105,18 @@ setup () {
 
 setup
 
-test_expect_success 'cloning' '
-	test_when_finished "rm -rf gitrepo*" &&
-
+test_expect_success 'setup' '
 	(
 	hg init hgrepo &&
 	cd hgrepo &&
 	echo zero >content &&
 	hg add content &&
 	hg commit -m zero
-	) &&
+	)
+'
 
+test_expect_success 'cloning' '
+	test_when_finished "rm -rf gitrepo*" &&
 	git clone "hg::hgrepo" gitrepo &&
 	check gitrepo HEAD zero
 '
-- 
1.9.2+fc1.3.gade8541

[PATCH 6/8] git-remote-hg: use internal clone's hgrc

[Felipe Contreras]

From: Daniel Liew <delcypher@gmail.com>

Use the hgrc configuration file in the internal mercurial repository in
addition to the other system wide hgrc files. This is done by using the
'ui' object from the 'repository' object which will have loaded the
repository hgrc file if it exists.

Signed-off-by: Dan Liew <delcypher@gmail.com>
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/git-remote-hg.py b/git-remote-hg.py
index 1972f7f..cd3d79e 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -422,7 +422,7 @@ def get_repo(url, alias):
 
         repo = hg.repository(myui, local_path)
         try:
-            peer = hg.peer(myui, {}, url)
+            peer = hg.peer(repo.ui, {}, url)
         except:
             die('Repository error')
         repo.pull(peer, heads=None, force=True)
-- 
1.9.2+fc1.3.gade8541

[PATCH 7/8] remote-hg: make sure we omit multiple heads

[Felipe Contreras]

We want to ignore secondary heads, otherwise we will import revisions
that won't have any ref pointing to them and might eventually be pruned,
which would cause problems with the synchronization of marks.

This can only be expressed properly as '::b - ::a', but that's not
efficient, specially in older versions of Mercurial.
'ancestor(a,b)::b - ancestor(a,b)::a' might be better, but it's not
particularly pretty. Mercurial v3.0 will have a new 'only(b, a)' that
does the same, but that hasn't even been released yet. Either way all of
these require repo.revs() which is only available after Mercurial v2.1.

Also, we would need special considerations for when there's no base
revision (importing from root).

It's much better to implement our own function to get a range of
revisions.

The new gitrange() is inspired by Mercurial's revset.missingancestors().

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py | 35 ++++++++++++++++++++++++++++++++---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/git-remote-hg.py b/git-remote-hg.py
index cd3d79e..402b92f 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -437,16 +437,45 @@ def rev_to_mark(rev):
 def mark_to_rev(mark):
     return marks.to_rev(mark)
 
+# Get a range of revisions in the form of a..b (git committish)
+def gitrange(repo, a, b):
+    positive = []
+    pending = set([int(b)])
+    negative = set([int(a)])
+    for cur in xrange(b, -1, -1):
+        if not pending:
+            break
+
+        parents = [p for p in repo.changelog.parentrevs(cur) if p >= 0]
+
+        if cur in pending:
+            positive.append(cur)
+            pending.remove(cur)
+            for p in parents:
+                if not p in negative:
+                    pending.add(p)
+        elif cur in negative:
+            negative.remove(cur)
+            for p in parents:
+                if not p in pending:
+                    negative.add(p)
+                else:
+                    pending.discard(p)
+
+    positive.reverse()
+    return positive
+
 def export_ref(repo, name, kind, head):
     ename = '%s/%s' % (kind, name)
     try:
         tip = marks.get_tip(ename)
-        tip = repo[tip].rev()
+        tip = repo[tip]
     except:
-        tip = 0
+        tip = repo[-1]
 
-    revs = xrange(tip, head.rev() + 1)
+    revs = gitrange(repo, tip, head)
     total = len(revs)
+    tip = tip.rev()
 
     for rev in revs:
 
-- 
1.9.2+fc1.3.gade8541

[PATCH 8/8] remote-hg: trivial cleanups

[Felipe Contreras]

Cleanup 51be46e (remote-hg: do not fail on invalid bookmarks).

Having a 40-characters string is not ideal, and having three tests for
basically the same relatively rare situation is overkill.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 git-remote-hg.py     |  2 +-
 t/t5810-remote-hg.sh | 56 ++++------------------------------------------------
 2 files changed, 5 insertions(+), 53 deletions(-)

diff --git a/git-remote-hg.py b/git-remote-hg.py
index 402b92f..74f2a2e 100755
--- a/git-remote-hg.py
+++ b/git-remote-hg.py
@@ -677,7 +677,7 @@ def do_list(parser):
             print "? refs/heads/branches/%s" % gitref(branch)
 
     for bmark in bmarks:
-        if  bmarks[bmark].hex() == '0000000000000000000000000000000000000000':
+        if  bmarks[bmark].hex() == '0' * 40:
             warn("Ignoring invalid bookmark '%s'", bmark)
         else:
             print "? refs/heads/%s" % gitref(bmark)
diff --git a/t/t5810-remote-hg.sh b/t/t5810-remote-hg.sh
index ba8b2d8..9946f57 100755
--- a/t/t5810-remote-hg.sh
+++ b/t/t5810-remote-hg.sh
@@ -772,7 +772,7 @@ test_expect_success 'remote double failed push' '
 	)
 '
 
-test_expect_success 'clone remote with master null bookmark, then push to the bookmark' '
+test_expect_success 'clone remote with null bookmark, then push' '
 	test_when_finished "rm -rf gitrepo* hgrepo*" &&
 
 	hg init hgrepo &&
@@ -781,67 +781,19 @@ test_expect_success 'clone remote with master null bookmark, then push to the bo
 		echo a >a &&
 		hg add a &&
 		hg commit -m a &&
-		hg bookmark -r null master
+		hg bookmark -r null bookmark
 	) &&
 
 	git clone "hg::hgrepo" gitrepo &&
 	check gitrepo HEAD a &&
 	(
 		cd gitrepo &&
-		git checkout --quiet -b master &&
-		echo b >b &&
-		git add b &&
-		git commit -m b &&
-		git push origin master
-	)
-'
-
-test_expect_success 'clone remote with default null bookmark, then push to the bookmark' '
-	test_when_finished "rm -rf gitrepo* hgrepo*" &&
-
-	hg init hgrepo &&
-	(
-		cd hgrepo &&
-		echo a >a &&
-		hg add a &&
-		hg commit -m a &&
-		hg bookmark -r null -f default
-	) &&
-
-	git clone "hg::hgrepo" gitrepo &&
-	check gitrepo HEAD a &&
-	(
-		cd gitrepo &&
-		git checkout --quiet -b default &&
-		echo b >b &&
-		git add b &&
-		git commit -m b &&
-		git push origin default
-	)
-'
-
-test_expect_success 'clone remote with generic null bookmark, then push to the bookmark' '
-	test_when_finished "rm -rf gitrepo* hgrepo*" &&
-
-	hg init hgrepo &&
-	(
-		cd hgrepo &&
-		echo a >a &&
-		hg add a &&
-		hg commit -m a &&
-		hg bookmark -r null bmark
-	) &&
-
-	git clone "hg::hgrepo" gitrepo &&
-	check gitrepo HEAD a &&
-	(
-		cd gitrepo &&
-		git checkout --quiet -b bmark &&
+		git checkout --quiet -b bookmark &&
 		git remote -v &&
 		echo b >b &&
 		git add b &&
 		git commit -m b &&
-		git push origin bmark
+		git push origin bookmark
 	)
 '
 
-- 
1.9.2+fc1.3.gade8541

Re: [PATCH 8/8] remote-hg: trivial cleanups

[Philip Oakley]

From: "Felipe Contreras" <felipe.contreras@gmail.com>
Subject: [PATCH 8/8] remote-hg: trivial cleanups

It's useful, as a reader of the shortlog and email message subject 
lines, to know what sort of triviality is being tidied.
Usually they are 'spelling' or 'variable naming' or some such that can 
easily be squeezed on the end without breaking the ~50 char guide. In 
this case there were two types, so I ended up with this suggestion for 
the message heading

    remote-hg: cleanup 40*{0} string, and de-dup tests

which adds just enough (for me) to get a feel for the style of what's 
inside

regards

Philip

> Cleanup 51be46e (remote-hg: do not fail on invalid bookmarks).
>
> Having a 40-characters string is not ideal, and having three tests for
> basically the same relatively rare situation is overkill.
>
> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
> ---
> git-remote-hg.py     |  2 +-
> t/t5810-remote-hg.sh | 56 
> ++++------------------------------------------------
> 2 files changed, 5 insertions(+), 53 deletions(-)
>
> diff --git a/git-remote-hg.py b/git-remote-hg.py
> index 402b92f..74f2a2e 100755
> --- a/git-remote-hg.py
> +++ b/git-remote-hg.py
> @@ -677,7 +677,7 @@ def do_list(parser):
>             print "? refs/heads/branches/%s" % gitref(branch)
>
>     for bmark in bmarks:
> -        if  bmarks[bmark].hex() == 
> '0000000000000000000000000000000000000000':
> +        if  bmarks[bmark].hex() == '0' * 40:
>             warn("Ignoring invalid bookmark '%s'", bmark)
>         else:
>             print "? refs/heads/%s" % gitref(bmark)
> diff --git a/t/t5810-remote-hg.sh b/t/t5810-remote-hg.sh
> index ba8b2d8..9946f57 100755
> --- a/t/t5810-remote-hg.sh
> +++ b/t/t5810-remote-hg.sh
> @@ -772,7 +772,7 @@ test_expect_success 'remote double failed push' '
>  )
> '
>
> -test_expect_success 'clone remote with master null bookmark, then 
> push to the bookmark' '
> +test_expect_success 'clone remote with null bookmark, then push' '
>  test_when_finished "rm -rf gitrepo* hgrepo*" &&
>
>  hg init hgrepo &&
> @@ -781,67 +781,19 @@ test_expect_success 'clone remote with master 
> null bookmark, then push to the bo
>  echo a >a &&
>  hg add a &&
>  hg commit -m a &&
> - hg bookmark -r null master
> + hg bookmark -r null bookmark
>  ) &&
>
>  git clone "hg::hgrepo" gitrepo &&
>  check gitrepo HEAD a &&
>  (
>  cd gitrepo &&
> - git checkout --quiet -b master &&
> - echo b >b &&
> - git add b &&
> - git commit -m b &&
> - git push origin master
> - )
> -'
> -
> -test_expect_success 'clone remote with default null bookmark, then 
> push to the bookmark' '
> - test_when_finished "rm -rf gitrepo* hgrepo*" &&
> -
> - hg init hgrepo &&
> - (
> - cd hgrepo &&
> - echo a >a &&
> - hg add a &&
> - hg commit -m a &&
> - hg bookmark -r null -f default
> - ) &&
> -
> - git clone "hg::hgrepo" gitrepo &&
> - check gitrepo HEAD a &&
> - (
> - cd gitrepo &&
> - git checkout --quiet -b default &&
> - echo b >b &&
> - git add b &&
> - git commit -m b &&
> - git push origin default
> - )
> -'
> -
> -test_expect_success 'clone remote with generic null bookmark, then 
> push to the bookmark' '
> - test_when_finished "rm -rf gitrepo* hgrepo*" &&
> -
> - hg init hgrepo &&
> - (
> - cd hgrepo &&
> - echo a >a &&
> - hg add a &&
> - hg commit -m a &&
> - hg bookmark -r null bmark
> - ) &&
> -
> - git clone "hg::hgrepo" gitrepo &&
> - check gitrepo HEAD a &&
> - (
> - cd gitrepo &&
> - git checkout --quiet -b bmark &&
> + git checkout --quiet -b bookmark &&
>  git remote -v &&
>  echo b >b &&
>  git add b &&
>  git commit -m b &&
> - git push origin bmark
> + git push origin bookmark
>  )
> '
>
> -- 
> 1.9.2+fc1.3.gade8541
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>