[PATCH v5 00/14] New remote-hg helper

[PATCH v5 00/14] New remote-hg helper

[Felipe Contreras]

Hi,

I've ported the tests from hg-git and made sure that the output from remote-hg
matches the output of hg-git. With these extensive tests I would consider this
one ready for wide use. Not only do the tests pass, I've compared the generated
repos of a few projects, and the SHA-1's are exactly the same :)

This remote-hg has advantages other tools don't have:

 * Uses transport-helper (git clone hg::path)
 * The code is small
 * The code is simple
 * No external dependencies (other than mercurial)
 * It's easy to install (put into your path)
 * Has extensive tests
 * Active development
 * Has compatibility with hg-git
 * The required patches are available
 * No changes necesary to git core

One important alternative is the one written by Sverre Rabbelier that is now
maintained and distributed in msysgit. It's hard to evaluate this option as
there isn't a branch specific to this remote helper so it would be possible to
evaluate the necessary patches.

To use it add it to your $PATH (e.g. ~/bin).

 % git clone hd:///full/path/or/url/to/hg/repo

To run the tests:

 % make -C contrib/remote-hg tests

The only caveat is that you need 'python' in your $PATH.

Changes since v4:

 * Add new simple biridectional tests (no need for hg-git)
 * Move tests to contrib directory
 * Add dependency checks to the tests
 * Trivial fixes

Changes since v3:

 * New extensive tests
 * Add compatibility mode with hg-git
 * Added support for boomkars
 * Add mercurial information to the git msg (branch, renames, extra, etc.)

Changes since v2:

 * Added support for pushing
 * Tests copied from original remote-hg
 * Custom default -> master renames removed
 * Code reorganized

Changes since v1:

 * Improved documentation
 * Use more common 'python' binary
 * Warn, don't barf when a branch has multiple heads
 * Fixed marks to fetch after cloned
 * Support for cloning/pulling remote repositories
 * Use a more appropriate internal directory (e.g. .git/hg/origin)
 * Fixes for python3

Felipe Contreras (14):
  Add new remote-hg transport helper
  remote-hg: add support for bookmarks
  remote-hg: add support for pushing
  remote-hg: add support for remote pushing
  remote-hg: add support to push URLs
  remote-hg: make sure the encoding is correct
  remote-hg: match hg merge behavior
  remote-hg: add support for hg-git compat mode
  remote-hg: add compat for hg-git author fixes
  remote-hg: fake bookmark when there's none
  remote-hg: add support for fake remote
  remote-hg: add biridectional tests
  remote-hg: add tests to compare with hg-git
  remote-hg: add extra author test

 contrib/remote-hg/Makefile       |  13 +
 contrib/remote-hg/git-remote-hg  | 773 +++++++++++++++++++++++++++++++++++++++
 contrib/remote-hg/test-hg-git.sh | 464 +++++++++++++++++++++++
 contrib/remote-hg/test.sh        | 241 ++++++++++++
 4 files changed, 1491 insertions(+)
 create mode 100644 contrib/remote-hg/Makefile
 create mode 100755 contrib/remote-hg/git-remote-hg
 create mode 100755 contrib/remote-hg/test-hg-git.sh
 create mode 100755 contrib/remote-hg/test.sh

-- 
1.8.0

[PATCH v5 01/14] Add new remote-hg transport helper

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/git-remote-hg | 359 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 359 insertions(+)
 create mode 100755 contrib/remote-hg/git-remote-hg

diff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg
new file mode 100755
index 0000000..67d39fa
--- /dev/null
+++ b/contrib/remote-hg/git-remote-hg
@@ -0,0 +1,359 @@
+#!/usr/bin/env python
+#
+# Copyright (c) 2012 Felipe Contreras
+#
+
+# Inspired by Rocco Rutte's hg-fast-export
+
+# Just copy to your ~/bin, or anywhere in your $PATH.
+# Then you can clone with:
+# git clone hg::/path/to/mercurial/repo/
+
+from mercurial import hg, ui
+
+import re
+import sys
+import os
+import json
+
+NAME_RE = re.compile('^([^<>]+)')
+AUTHOR_RE = re.compile('^([^<>]+?)? ?<([^<>]+)>$')
+
+def die(msg, *args):
+    sys.stderr.write('ERROR: %s\n' % (msg % args))
+    sys.exit(1)
+
+def warn(msg, *args):
+    sys.stderr.write('WARNING: %s\n' % (msg % args))
+
+def gitmode(flags):
+    return 'l' in flags and '120000' or 'x' in flags and '100755' or '100644'
+
+def gittz(tz):
+    return '%+03d%02d' % (-tz / 3600, -tz % 3600 / 60)
+
+class Marks:
+
+    def __init__(self, path):
+        self.path = path
+        self.tips = {}
+        self.marks = {}
+        self.last_mark = 0
+
+        self.load()
+
+    def load(self):
+        if not os.path.exists(self.path):
+            return
+
+        tmp = json.load(open(self.path))
+
+        self.tips = tmp['tips']
+        self.marks = tmp['marks']
+        self.last_mark = tmp['last-mark']
+
+    def dict(self):
+        return { 'tips': self.tips, 'marks': self.marks, 'last-mark' : self.last_mark }
+
+    def store(self):
+        json.dump(self.dict(), open(self.path, 'w'))
+
+    def __str__(self):
+        return str(self.dict())
+
+    def from_rev(self, rev):
+        return self.marks[str(rev)]
+
+    def next_mark(self, rev):
+        self.last_mark += 1
+        self.marks[str(rev)] = self.last_mark
+        return self.last_mark
+
+    def is_marked(self, rev):
+        return self.marks.has_key(str(rev))
+
+    def get_tip(self, branch):
+        return self.tips.get(branch, 0)
+
+    def set_tip(self, branch, tip):
+        self.tips[branch] = tip
+
+class Parser:
+
+    def __init__(self, repo):
+        self.repo = repo
+        self.line = self.get_line()
+
+    def get_line(self):
+        return sys.stdin.readline().strip()
+
+    def __getitem__(self, i):
+        return self.line.split()[i]
+
+    def check(self, word):
+        return self.line.startswith(word)
+
+    def each_block(self, separator):
+        while self.line != separator:
+            yield self.line
+            self.line = self.get_line()
+
+    def __iter__(self):
+        return self.each_block('')
+
+    def next(self):
+        self.line = self.get_line()
+        if self.line == 'done':
+            self.line = None
+
+def export_file(fc):
+    d = fc.data()
+    print "M %s inline %s" % (gitmode(fc.flags()), fc.path())
+    print "data %d" % len(d)
+    print d
+
+def get_filechanges(repo, ctx, parents):
+    l = [repo.status(p, ctx)[:3] for p in parents]
+    changed, added, removed = [set(sum(e, [])) for e in zip(*l)]
+    return added | changed, removed
+
+def fixup_user(user):
+    user = user.replace('"', '')
+    name = mail = None
+    m = AUTHOR_RE.match(user)
+    if m:
+        name = m.group(1)
+        mail = m.group(2).strip()
+    else:
+        m = NAME_RE.match(user)
+        if m:
+            name = m.group(1).strip()
+
+    if not name:
+        name = 'Unknown'
+    if not mail:
+        mail = 'unknown'
+
+    return '%s <%s>' % (name, mail)
+
+def get_repo(url, alias):
+    global dirname
+
+    myui = ui.ui()
+    myui.setconfig('ui', 'interactive', 'off')
+
+    if hg.islocal(url):
+        repo = hg.repository(myui, url)
+    else:
+        local_path = os.path.join(dirname, 'clone')
+        if not os.path.exists(local_path):
+            peer, dstpeer = hg.clone(myui, {}, url, local_path, update=False, pull=True)
+            repo = dstpeer.local()
+        else:
+            repo = hg.repository(myui, local_path)
+            peer = hg.peer(myui, {}, url)
+            repo.pull(peer, heads=None, force=True)
+
+    return repo
+
+def rev_to_mark(rev):
+    global marks
+    return marks.from_rev(rev)
+
+def export_ref(repo, name, kind, head):
+    global prefix, marks
+
+    ename = '%s/%s' % (kind, name)
+    tip = marks.get_tip(ename)
+
+    # mercurial takes too much time checking this
+    if tip and tip == head.rev():
+        # nothing to do
+        return
+    revs = repo.revs('%u:%u' % (tip, head))
+    count = 0
+
+    revs = [rev for rev in revs if not marks.is_marked(rev)]
+
+    for rev in revs:
+
+        c = repo[rev]
+        (manifest, user, (time, tz), files, desc, extra) = repo.changelog.read(c.node())
+        rev_branch = extra['branch']
+
+        author = "%s %d %s" % (fixup_user(user), time, gittz(tz))
+        if 'committer' in extra:
+            user, time, tz = extra['committer'].rsplit(' ', 2)
+            committer = "%s %s %s" % (user, time, gittz(int(tz)))
+        else:
+            committer = author
+
+        parents = [p for p in repo.changelog.parentrevs(rev) if p >= 0]
+
+        if len(parents) == 0:
+            modified = c.manifest().keys()
+            removed = []
+        else:
+            modified, removed = get_filechanges(repo, c, parents)
+
+        if len(parents) == 0 and rev:
+            print 'reset %s/%s' % (prefix, ename)
+
+        print "commit %s/%s" % (prefix, ename)
+        print "mark :%d" % (marks.next_mark(rev))
+        print "author %s" % (author)
+        print "committer %s" % (committer)
+        print "data %d" % (len(desc))
+        print desc
+
+        if len(parents) > 0:
+            print "from :%s" % (rev_to_mark(parents[0]))
+            if len(parents) > 1:
+                print "merge :%s" % (rev_to_mark(parents[1]))
+
+        for f in removed:
+            print "D %s" % (f)
+        for f in modified:
+            export_file(c.filectx(f))
+        print
+
+        count += 1
+        if (count % 100 == 0):
+            print "progress revision %d '%s' (%d/%d)" % (rev, name, count, len(revs))
+            print "#############################################################"
+
+    # make sure the ref is updated
+    print "reset %s/%s" % (prefix, ename)
+    print "from :%u" % rev_to_mark(rev)
+    print
+
+    marks.set_tip(ename, rev)
+
+def export_tag(repo, tag):
+    export_ref(repo, tag, 'tags', repo[tag])
+
+def export_branch(repo, branch):
+    tip = get_branch_tip(repo, branch)
+    head = repo[tip]
+    export_ref(repo, branch, 'branches', head)
+
+def export_head(repo):
+    global g_head
+    export_ref(repo, g_head[0], g_head[1], g_head[2])
+
+def do_capabilities(parser):
+    global prefix, dirname
+
+    print "import"
+    print "refspec refs/heads/branches/*:%s/branches/*" % prefix
+    print "refspec refs/tags/*:%s/tags/*" % prefix
+    print
+
+def get_branch_tip(repo, branch):
+    global branches
+
+    heads = branches.get(branch, None)
+    if not heads:
+        return None
+
+    # verify there's only one head
+    if (len(heads) > 1):
+        warn("Branch '%s' has more than one head, consider merging" % branch)
+        return repo.branchtip(branch)
+
+    return heads[0]
+
+def list_branch_head(repo, cur):
+    global g_head
+
+    tip = get_branch_tip(repo, cur)
+    head = 'branches/' + cur
+    print "@refs/heads/%s HEAD" % head
+    g_head = (head, 'branches', repo[tip])
+
+def do_list(parser):
+    global branches
+
+    repo = parser.repo
+    for branch in repo.branchmap():
+        heads = repo.branchheads(branch)
+        if len(heads):
+            branches[branch] = heads
+
+    cur = repo.dirstate.branch()
+
+    list_branch_head(repo, cur)
+    for branch in branches:
+        print "? refs/heads/branches/%s" % branch
+
+    for tag, node in repo.tagslist():
+        if tag == 'tip':
+            continue
+        print "? refs/tags/%s" % tag
+
+    print
+
+def do_import(parser):
+    repo = parser.repo
+
+    path = os.path.join(dirname, 'marks-git')
+
+    print "feature done"
+    if os.path.exists(path):
+        print "feature import-marks=%s" % path
+    print "feature export-marks=%s" % path
+    sys.stdout.flush()
+
+    # lets get all the import lines
+    while parser.check('import'):
+        ref = parser[1]
+
+        if (ref == 'HEAD'):
+            export_head(repo)
+        elif ref.startswith('refs/heads/branches/'):
+            branch = ref[len('refs/heads/branches/'):]
+            export_branch(repo, branch)
+        elif ref.startswith('refs/tags/'):
+            tag = ref[len('refs/tags/'):]
+            export_tag(repo, tag)
+
+        parser.next()
+
+    print 'done'
+
+def main(args):
+    global prefix, dirname, marks, branches
+
+    alias = args[1]
+    url = args[2]
+
+    gitdir = os.environ['GIT_DIR']
+    dirname = os.path.join(gitdir, 'hg', alias)
+    branches = {}
+
+    repo = get_repo(url, alias)
+    prefix = 'refs/hg/%s' % alias
+
+    if not os.path.exists(dirname):
+        os.makedirs(dirname)
+
+    marks_path = os.path.join(dirname, 'marks-hg')
+    marks = Marks(marks_path)
+
+    parser = Parser(repo)
+    for line in parser:
+        if parser.check('capabilities'):
+            do_capabilities(parser)
+        elif parser.check('list'):
+            do_list(parser)
+        elif parser.check('import'):
+            do_import(parser)
+        elif parser.check('export'):
+            do_export(parser)
+        else:
+            die('unhandled command: %s' % line)
+        sys.stdout.flush()
+
+    marks.store()
+
+sys.exit(main(sys.argv))
-- 
1.8.0

[PATCH v5 02/14] remote-hg: add support for bookmarks

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/git-remote-hg | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg
index 67d39fa..9e29daa 100755
--- a/contrib/remote-hg/git-remote-hg
+++ b/contrib/remote-hg/git-remote-hg
@@ -9,7 +9,7 @@
 # Then you can clone with:
 # git clone hg::/path/to/mercurial/repo/
 
-from mercurial import hg, ui
+from mercurial import hg, ui, bookmarks
 
 import re
 import sys
@@ -232,6 +232,10 @@ def export_ref(repo, name, kind, head):
 def export_tag(repo, tag):
     export_ref(repo, tag, 'tags', repo[tag])
 
+def export_bookmark(repo, bmark):
+    head = bmarks[bmark]
+    export_ref(repo, bmark, 'bookmarks', head)
+
 def export_branch(repo, branch):
     tip = get_branch_tip(repo, branch)
     head = repo[tip]
@@ -246,6 +250,7 @@ def do_capabilities(parser):
 
     print "import"
     print "refspec refs/heads/branches/*:%s/branches/*" % prefix
+    print "refspec refs/heads/*:%s/bookmarks/*" % prefix
     print "refspec refs/tags/*:%s/tags/*" % prefix
     print
 
@@ -272,7 +277,7 @@ def list_branch_head(repo, cur):
     g_head = (head, 'branches', repo[tip])
 
 def do_list(parser):
-    global branches
+    global branches, bmarks
 
     repo = parser.repo
     for branch in repo.branchmap():
@@ -280,11 +285,16 @@ def do_list(parser):
         if len(heads):
             branches[branch] = heads
 
+    for bmark, node in bookmarks.listbookmarks(repo).iteritems():
+        bmarks[bmark] = repo[node]
+
     cur = repo.dirstate.branch()
 
     list_branch_head(repo, cur)
     for branch in branches:
         print "? refs/heads/branches/%s" % branch
+    for bmark in bmarks:
+        print "? refs/heads/%s" % bmark
 
     for tag, node in repo.tagslist():
         if tag == 'tip':
@@ -313,6 +323,9 @@ def do_import(parser):
         elif ref.startswith('refs/heads/branches/'):
             branch = ref[len('refs/heads/branches/'):]
             export_branch(repo, branch)
+        elif ref.startswith('refs/heads/'):
+            bmark = ref[len('refs/heads/'):]
+            export_bookmark(repo, bmark)
         elif ref.startswith('refs/tags/'):
             tag = ref[len('refs/tags/'):]
             export_tag(repo, tag)
@@ -322,7 +335,7 @@ def do_import(parser):
     print 'done'
 
 def main(args):
-    global prefix, dirname, marks, branches
+    global prefix, dirname, marks, branches, bmarks
 
     alias = args[1]
     url = args[2]
@@ -330,6 +343,7 @@ def main(args):
     gitdir = os.environ['GIT_DIR']
     dirname = os.path.join(gitdir, 'hg', alias)
     branches = {}
+    bmarks = {}
 
     repo = get_repo(url, alias)
     prefix = 'refs/hg/%s' % alias
-- 
1.8.0

[PATCH v5 03/14] remote-hg: add support for pushing

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/git-remote-hg | 215 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 213 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg
index 9e29daa..337ba40 100755
--- a/contrib/remote-hg/git-remote-hg
+++ b/contrib/remote-hg/git-remote-hg
@@ -9,7 +9,7 @@
 # Then you can clone with:
 # git clone hg::/path/to/mercurial/repo/
 
-from mercurial import hg, ui, bookmarks
+from mercurial import hg, ui, bookmarks, context
 
 import re
 import sys
@@ -18,6 +18,7 @@ import json
 
 NAME_RE = re.compile('^([^<>]+)')
 AUTHOR_RE = re.compile('^([^<>]+?)? ?<([^<>]+)>$')
+RAW_AUTHOR_RE = re.compile('^(\w+) (?:(.+)? )?<(.+)> (\d+) ([+-]\d+)')
 
 def die(msg, *args):
     sys.stderr.write('ERROR: %s\n' % (msg % args))
@@ -32,12 +33,17 @@ def gitmode(flags):
 def gittz(tz):
     return '%+03d%02d' % (-tz / 3600, -tz % 3600 / 60)
 
+def hgmode(mode):
+    m = { '0100755': 'x', '0120000': 'l' }
+    return m.get(mode, '')
+
 class Marks:
 
     def __init__(self, path):
         self.path = path
         self.tips = {}
         self.marks = {}
+        self.rev_marks = {}
         self.last_mark = 0
 
         self.load()
@@ -52,6 +58,9 @@ class Marks:
         self.marks = tmp['marks']
         self.last_mark = tmp['last-mark']
 
+        for rev, mark in self.marks.iteritems():
+            self.rev_marks[mark] = int(rev)
+
     def dict(self):
         return { 'tips': self.tips, 'marks': self.marks, 'last-mark' : self.last_mark }
 
@@ -64,11 +73,19 @@ class Marks:
     def from_rev(self, rev):
         return self.marks[str(rev)]
 
+    def to_rev(self, mark):
+        return self.rev_marks[mark]
+
     def next_mark(self, rev):
         self.last_mark += 1
         self.marks[str(rev)] = self.last_mark
         return self.last_mark
 
+    def new_mark(self, rev, mark):
+        self.marks[str(rev)] = mark
+        self.rev_marks[mark] = rev
+        self.last_mark = mark
+
     def is_marked(self, rev):
         return self.marks.has_key(str(rev))
 
@@ -106,6 +123,35 @@ class Parser:
         if self.line == 'done':
             self.line = None
 
+    def get_mark(self):
+        i = self.line.index(':') + 1
+        return int(self.line[i:])
+
+    def get_data(self):
+        if not self.check('data'):
+            return None
+        i = self.line.index(' ') + 1
+        size = int(self.line[i:])
+        return sys.stdin.read(size)
+
+    def get_author(self):
+        m = RAW_AUTHOR_RE.match(self.line)
+        if not m:
+            return None
+        _, name, email, date, tz = m.groups()
+
+        if email != 'unknown':
+            if name:
+                user = '%s <%s>' % (name, email)
+            else:
+                user = '<%s>' % (email)
+        else:
+            user = name
+
+        tz = int(tz)
+        tz = ((tz / 100) * 3600) + ((tz % 100) * 60)
+        return (user, int(date), -tz)
+
 def export_file(fc):
     d = fc.data()
     print "M %s inline %s" % (gitmode(fc.flags()), fc.path())
@@ -160,6 +206,10 @@ def rev_to_mark(rev):
     global marks
     return marks.from_rev(rev)
 
+def mark_to_rev(mark):
+    global marks
+    return marks.to_rev(mark)
+
 def export_ref(repo, name, kind, head):
     global prefix, marks
 
@@ -249,9 +299,17 @@ def do_capabilities(parser):
     global prefix, dirname
 
     print "import"
+    print "export"
     print "refspec refs/heads/branches/*:%s/branches/*" % prefix
     print "refspec refs/heads/*:%s/bookmarks/*" % prefix
     print "refspec refs/tags/*:%s/tags/*" % prefix
+
+    path = os.path.join(dirname, 'marks-git')
+
+    if os.path.exists(path):
+        print "*import-marks %s" % path
+    print "*export-marks %s" % path
+
     print
 
 def get_branch_tip(repo, branch):
@@ -334,8 +392,159 @@ def do_import(parser):
 
     print 'done'
 
+def parse_blob(parser):
+    global blob_marks
+
+    parser.next()
+    mark = parser.get_mark()
+    parser.next()
+    data = parser.get_data()
+    blob_marks[mark] = data
+    parser.next()
+    return
+
+def parse_commit(parser):
+    global marks, blob_marks, bmarks, parsed_refs
+
+    from_mark = merge_mark = None
+
+    a = parser.line.split(' ')
+    ref = a[1]
+    parser.next()
+
+    commit_mark = parser.get_mark()
+    parser.next()
+    author = parser.get_author()
+    parser.next()
+    committer = parser.get_author()
+    parser.next()
+    data = parser.get_data()
+    parser.next()
+    if parser.check('from'):
+        from_mark = parser.get_mark()
+        parser.next()
+    if parser.check('merge'):
+        merge_mark = parser.get_mark()
+        parser.next()
+        if parser.check('merge'):
+            die('octopus merges are not supported yet')
+
+    files = {}
+
+    for line in parser:
+        if parser.check('M'):
+            t, m, mark_ref, path = line.split(' ')
+            mark = int(mark_ref[1:])
+            f = { 'mode' : hgmode(m), 'data' : blob_marks[mark] }
+        elif parser.check('D'):
+            t, path = line.split(' ')
+            f = { 'deleted' : True }
+        else:
+            die('Unknown file command: %s' % line)
+        files[path] = f
+
+    def getfilectx(repo, memctx, f):
+        of = files[f]
+        if 'deleted' in of:
+            raise IOError
+        is_exec = of['mode'] == 'x'
+        is_link = of['mode'] == 'l'
+        return context.memfilectx(f, of['data'], is_link, is_exec, None)
+
+    repo = parser.repo
+
+    user, date, tz = author
+    extra = {}
+
+    if committer != author:
+        extra['committer'] = "%s %u %u" % committer
+
+    if from_mark:
+        p1 = repo.changelog.node(mark_to_rev(from_mark))
+    else:
+        p1 = '\0' * 20
+
+    if merge_mark:
+        p2 = repo.changelog.node(mark_to_rev(merge_mark))
+    else:
+        p2 = '\0' * 20
+
+    ctx = context.memctx(repo, (p1, p2), data,
+            files.keys(), getfilectx,
+            user, (date, tz), extra)
+
+    node = repo.commitctx(ctx)
+
+    rev = repo[node].rev()
+
+    parsed_refs[ref] = node
+
+    marks.new_mark(rev, commit_mark)
+
+def parse_reset(parser):
+    a = parser.line.split(' ')
+    ref = a[1]
+    parser.next()
+    # ugh
+    if parser.check('commit'):
+        parse_commit(parser)
+        return
+    if not parser.check('from'):
+        return
+    from_mark = parser.get_mark()
+    parser.next()
+
+    node = parser.repo.changelog.node(mark_to_rev(from_mark))
+    parsed_refs[ref] = node
+
+def parse_tag(parser):
+    a = parser.line.split(' ')
+    name = a[1]
+    parser.next()
+    from_mark = parser.get_mark()
+    parser.next()
+    tagger = parser.get_author()
+    parser.next()
+    data = parser.get_data()
+    parser.next()
+
+    # nothing to do
+
+def do_export(parser):
+    global parsed_refs
+
+    parser.next()
+
+    for line in parser.each_block('done'):
+        if parser.check('blob'):
+            parse_blob(parser)
+        elif parser.check('commit'):
+            parse_commit(parser)
+        elif parser.check('reset'):
+            parse_reset(parser)
+        elif parser.check('tag'):
+            parse_tag(parser)
+        elif parser.check('feature'):
+            pass
+        else:
+            die('unhandled export command: %s' % line)
+
+    for ref, node in parsed_refs.iteritems():
+        if ref.startswith('refs/heads/branches'):
+            pass
+        elif ref.startswith('refs/heads/'):
+            bmark = ref[len('refs/heads/'):]
+            bookmarks.pushbookmark(parser.repo, bmark, '', node)
+        elif ref.startswith('refs/tags/'):
+            tag = ref[len('refs/tags/'):]
+            parser.repo.tag([tag], node, None, True, None, {})
+        print "ok %s" % ref
+
+    print
+
 def main(args):
-    global prefix, dirname, marks, branches, bmarks
+    global prefix, dirname, branches, bmarks
+    global marks, blob_marks, parsed_refs
 
     alias = args[1]
     url = args[2]
@@ -344,6 +553,8 @@ def main(args):
     dirname = os.path.join(gitdir, 'hg', alias)
     branches = {}
     bmarks = {}
+    blob_marks = {}
+    parsed_refs = {}
 
     repo = get_repo(url, alias)
     prefix = 'refs/hg/%s' % alias
-- 
1.8.0

[PATCH v5 04/14] remote-hg: add support for remote pushing

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/git-remote-hg | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg
index 337ba40..959ab80 100755
--- a/contrib/remote-hg/git-remote-hg
+++ b/contrib/remote-hg/git-remote-hg
@@ -183,7 +183,7 @@ def fixup_user(user):
     return '%s <%s>' % (name, mail)
 
 def get_repo(url, alias):
-    global dirname
+    global dirname, peer
 
     myui = ui.ui()
     myui.setconfig('ui', 'interactive', 'off')
@@ -511,7 +511,7 @@ def parse_tag(parser):
     # nothing to do
 
 def do_export(parser):
-    global parsed_refs
+    global parsed_refs, peer
 
     parser.next()
 
@@ -542,12 +542,17 @@ def do_export(parser):
 
     print
 
+    if peer:
+        parser.repo.push(peer, force=False)
+
 def main(args):
     global prefix, dirname, branches, bmarks
     global marks, blob_marks, parsed_refs
+    global peer
 
     alias = args[1]
     url = args[2]
+    peer = None
 
     gitdir = os.environ['GIT_DIR']
     dirname = os.path.join(gitdir, 'hg', alias)
-- 
1.8.0

[PATCH v5 05/14] remote-hg: add support to push URLs

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/git-remote-hg | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg
index 959ab80..4d49923 100755
--- a/contrib/remote-hg/git-remote-hg
+++ b/contrib/remote-hg/git-remote-hg
@@ -9,12 +9,13 @@
 # Then you can clone with:
 # git clone hg::/path/to/mercurial/repo/
 
-from mercurial import hg, ui, bookmarks, context
+from mercurial import hg, ui, bookmarks, context, util
 
 import re
 import sys
 import os
 import json
+import shutil
 
 NAME_RE = re.compile('^([^<>]+)')
 AUTHOR_RE = re.compile('^([^<>]+?)? ?<([^<>]+)>$')
@@ -554,6 +555,12 @@ def main(args):
     url = args[2]
     peer = None
 
+    if alias[4:] == url:
+        is_tmp = True
+        alias = util.sha1(alias).hexdigest()
+    else:
+        is_tmp = False
+
     gitdir = os.environ['GIT_DIR']
     dirname = os.path.join(gitdir, 'hg', alias)
     branches = {}
@@ -584,6 +591,9 @@ def main(args):
             die('unhandled command: %s' % line)
         sys.stdout.flush()
 
-    marks.store()
+    if not is_tmp:
+        marks.store()
+    else:
+        shutil.rmtree(dirname)
 
 sys.exit(main(sys.argv))
-- 
1.8.0

[PATCH v5 06/14] remote-hg: make sure the encoding is correct

[Felipe Contreras]

Independently of the environment.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/git-remote-hg | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg
index 4d49923..29824be 100755
--- a/contrib/remote-hg/git-remote-hg
+++ b/contrib/remote-hg/git-remote-hg
@@ -9,7 +9,7 @@
 # Then you can clone with:
 # git clone hg::/path/to/mercurial/repo/
 
-from mercurial import hg, ui, bookmarks, context, util
+from mercurial import hg, ui, bookmarks, context, util, encoding
 
 import re
 import sys
@@ -373,6 +373,9 @@ def do_import(parser):
     print "feature export-marks=%s" % path
     sys.stdout.flush()
 
+    tmp = encoding.encoding
+    encoding.encoding = 'utf-8'
+
     # lets get all the import lines
     while parser.check('import'):
         ref = parser[1]
@@ -391,6 +394,8 @@ def do_import(parser):
 
         parser.next()
 
+    encoding.encoding = tmp
+
     print 'done'
 
 def parse_blob(parser):
@@ -474,8 +479,13 @@ def parse_commit(parser):
             files.keys(), getfilectx,
             user, (date, tz), extra)
 
+    tmp = encoding.encoding
+    encoding.encoding = 'utf-8'
+
     node = repo.commitctx(ctx)
 
+    encoding.encoding = tmp
+
     rev = repo[node].rev()
 
     parsed_refs[ref] = node
-- 
1.8.0

[PATCH v5 07/14] remote-hg: match hg merge behavior

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/git-remote-hg | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg
index 29824be..c28f4b4 100755
--- a/contrib/remote-hg/git-remote-hg
+++ b/contrib/remote-hg/git-remote-hg
@@ -409,6 +409,12 @@ def parse_blob(parser):
     parser.next()
     return
 
+def get_merge_files(repo, p1, p2, files):
+    for e in repo[p1].files():
+        if e not in files:
+            f = { 'ctx' : repo[p1][e] }
+            files[e] = f
+
 def parse_commit(parser):
     global marks, blob_marks, bmarks, parsed_refs
 
@@ -453,6 +459,8 @@ def parse_commit(parser):
         of = files[f]
         if 'deleted' in of:
             raise IOError
+        if 'ctx' in of:
+            return of['ctx']
         is_exec = of['mode'] == 'x'
         is_link = of['mode'] == 'l'
         return context.memfilectx(f, of['data'], is_link, is_exec, None)
@@ -475,6 +483,13 @@ def parse_commit(parser):
     else:
         p2 = '\0' * 20
 
+    #
+    # If files changed from any of the parents, hg wants to know, but in git if
+    # nothing changed from the first parent, nothing changed.
+    #
+    if merge_mark:
+        get_merge_files(repo, p1, p2, files)
+
     ctx = context.memctx(repo, (p1, p2), data,
             files.keys(), getfilectx,
             user, (date, tz), extra)
-- 
1.8.0

[PATCH v5 08/14] remote-hg: add support for hg-git compat mode

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/git-remote-hg | 102 +++++++++++++++++++++++++++++++++++++---
 1 file changed, 95 insertions(+), 7 deletions(-)

diff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg
index c28f4b4..e49f9ed 100755
--- a/contrib/remote-hg/git-remote-hg
+++ b/contrib/remote-hg/git-remote-hg
@@ -16,6 +16,22 @@ import sys
 import os
 import json
 import shutil
+import subprocess
+
+#
+# If you want to switch to hg-git compatibility mode:
+# git config --global remote-hg.hg-git-compat true
+#
+# git:
+# Sensible defaults for git.
+# hg bookmarks are exported as git branches, hg branches are prefixed
+# with 'branches/'.
+#
+# hg:
+# Emulate hg-git.
+# Only hg bookmarks are exported as git branches.
+# Commits are modified to preserve hg information and allow biridectionality.
+#
 
 NAME_RE = re.compile('^([^<>]+)')
 AUTHOR_RE = re.compile('^([^<>]+?)? ?<([^<>]+)>$')
@@ -212,7 +228,7 @@ def mark_to_rev(mark):
     return marks.to_rev(mark)
 
 def export_ref(repo, name, kind, head):
-    global prefix, marks
+    global prefix, marks, mode
 
     ename = '%s/%s' % (kind, name)
     tip = marks.get_tip(ename)
@@ -247,6 +263,33 @@ def export_ref(repo, name, kind, head):
         else:
             modified, removed = get_filechanges(repo, c, parents)
 
+        if mode == 'hg':
+            extra_msg = ''
+
+            if rev_branch != 'default':
+                extra_msg += 'branch : %s\n' % rev_branch
+
+            renames = []
+            for f in c.files():
+                if f not in c.manifest():
+                    continue
+                rename = c.filectx(f).renamed()
+                if rename:
+                    renames.append((rename[0], f))
+
+            for e in renames:
+                extra_msg += "rename : %s => %s\n" % e
+
+            for key, value in extra.iteritems():
+                if key in ('author', 'committer', 'encoding', 'message', 'branch', 'hg-git'):
+                    continue
+                else:
+                    extra_msg += "extra : %s : %s\n" % (key, urllib.quote(value))
+
+            desc += '\n'
+            if extra_msg:
+                desc += '\n--HG--\n' + extra_msg
+
         if len(parents) == 0 and rev:
             print 'reset %s/%s' % (prefix, ename)
 
@@ -335,8 +378,18 @@ def list_branch_head(repo, cur):
     print "@refs/heads/%s HEAD" % head
     g_head = (head, 'branches', repo[tip])
 
+def list_bookmark_head(repo, cur):
+    global g_head
+
+    head = bookmarks.readcurrent(repo)
+    if not head:
+        return
+    node = repo[head]
+    print "@refs/heads/%s HEAD" % head
+    g_head = (head, 'bookmarks', node)
+
 def do_list(parser):
-    global branches, bmarks
+    global branches, bmarks, mode
 
     repo = parser.repo
     for branch in repo.branchmap():
@@ -349,9 +402,13 @@ def do_list(parser):
 
     cur = repo.dirstate.branch()
 
-    list_branch_head(repo, cur)
-    for branch in branches:
-        print "? refs/heads/branches/%s" % branch
+    if mode != 'hg':
+        list_branch_head(repo, cur)
+        for branch in branches:
+            print "? refs/heads/branches/%s" % branch
+    else:
+        list_bookmark_head(repo, cur)
+
     for bmark in bmarks:
         print "? refs/heads/%s" % bmark
 
@@ -417,6 +474,7 @@ def get_merge_files(repo, p1, p2, files):
 
 def parse_commit(parser):
     global marks, blob_marks, bmarks, parsed_refs
+    global mode
 
     from_mark = merge_mark = None
 
@@ -463,7 +521,9 @@ def parse_commit(parser):
             return of['ctx']
         is_exec = of['mode'] == 'x'
         is_link = of['mode'] == 'l'
-        return context.memfilectx(f, of['data'], is_link, is_exec, None)
+        rename = of.get('rename', None)
+        return context.memfilectx(f, of['data'],
+                is_link, is_exec, rename)
 
     repo = parser.repo
 
@@ -490,6 +550,21 @@ def parse_commit(parser):
     if merge_mark:
         get_merge_files(repo, p1, p2, files)
 
+    if mode == 'hg':
+        i = data.find('\n--HG--\n')
+        if i >= 0:
+            tmp = data[i + len('\n--HG--\n'):].strip()
+            for k, v in [e.split(' : ') for e in tmp.split('\n')]:
+                if k == 'rename':
+                    old, new = v.split(' => ', 1)
+                    files[new]['rename'] = old
+                elif k == 'branch':
+                    extra[k] = v
+                elif k == 'extra':
+                    ek, ev = v.split(' : ', 1)
+                    extra[ek] = urllib.unquote(ev)
+            data = data[:i]
+
     ctx = context.memctx(repo, (p1, p2), data,
             files.keys(), getfilectx,
             user, (date, tz), extra)
@@ -574,12 +649,25 @@ def do_export(parser):
 def main(args):
     global prefix, dirname, branches, bmarks
     global marks, blob_marks, parsed_refs
-    global peer
+    global peer, mode
 
     alias = args[1]
     url = args[2]
     peer = None
 
+    cmd = ['git', 'config', '--get', 'remote-hg.hg-git-compat']
+    hg_git_compat = False
+    try:
+        if subprocess.check_output(cmd) == 'true\n':
+            hg_git_compat = True
+    except subprocess.CalledProcessError:
+        pass
+
+    if hg_git_compat:
+        mode = 'hg'
+    else:
+        mode = 'git'
+
     if alias[4:] == url:
         is_tmp = True
         alias = util.sha1(alias).hexdigest()
-- 
1.8.0

[PATCH v5 09/14] remote-hg: add compat for hg-git author fixes

[Felipe Contreras]

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/git-remote-hg | 59 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 53 insertions(+), 6 deletions(-)

diff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg
index e49f9ed..c2efadf 100755
--- a/contrib/remote-hg/git-remote-hg
+++ b/contrib/remote-hg/git-remote-hg
@@ -17,6 +17,7 @@ import os
 import json
 import shutil
 import subprocess
+import urllib
 
 #
 # If you want to switch to hg-git compatibility mode:
@@ -35,6 +36,7 @@ import subprocess
 
 NAME_RE = re.compile('^([^<>]+)')
 AUTHOR_RE = re.compile('^([^<>]+?)? ?<([^<>]+)>$')
+AUTHOR_HG_RE = re.compile('^(.*?) ?<(.+?)(?:>(.+)?)?$')
 RAW_AUTHOR_RE = re.compile('^(\w+) (?:(.+)? )?<(.+)> (\d+) ([+-]\d+)')
 
 def die(msg, *args):
@@ -152,12 +154,20 @@ class Parser:
         return sys.stdin.read(size)
 
     def get_author(self):
+        global bad_mail
+
+        ex = None
         m = RAW_AUTHOR_RE.match(self.line)
         if not m:
             return None
         _, name, email, date, tz = m.groups()
+        if name and 'ext:' in name:
+            m = re.match('^(.+?) ext:\((.+)\)$', name)
+            if m:
+                name = m.group(1)
+                ex = urllib.unquote(m.group(2))
 
-        if email != 'unknown':
+        if email != bad_mail:
             if name:
                 user = '%s <%s>' % (name, email)
             else:
@@ -165,6 +175,9 @@ class Parser:
         else:
             user = name
 
+        if ex:
+            user += ex
+
         tz = int(tz)
         tz = ((tz / 100) * 3600) + ((tz % 100) * 60)
         return (user, int(date), -tz)
@@ -180,9 +193,9 @@ def get_filechanges(repo, ctx, parents):
     changed, added, removed = [set(sum(e, [])) for e in zip(*l)]
     return added | changed, removed
 
-def fixup_user(user):
-    user = user.replace('"', '')
+def fixup_user_git(user):
     name = mail = None
+    user = user.replace('"', '')
     m = AUTHOR_RE.match(user)
     if m:
         name = m.group(1)
@@ -191,11 +204,41 @@ def fixup_user(user):
         m = NAME_RE.match(user)
         if m:
             name = m.group(1).strip()
+    return (name, mail)
+
+def fixup_user_hg(user):
+    def sanitize(name):
+        # stole this from hg-git
+        return re.sub('[<>\n]', '?', name.lstrip('< ').rstrip('> '))
+
+    m = AUTHOR_HG_RE.match(user)
+    if m:
+        name = sanitize(m.group(1))
+        mail = sanitize(m.group(2))
+        ex = m.group(3)
+        if ex:
+            name += ' ext:(' + urllib.quote(ex) + ')'
+    else:
+        name = sanitize(user)
+        if '@' in user:
+            mail = name
+        else:
+            mail = None
+
+    return (name, mail)
+
+def fixup_user(user):
+    global mode, bad_mail
+
+    if mode == 'git':
+        name, mail = fixup_user_git(user)
+    else:
+        name, mail = fixup_user_hg(user)
 
     if not name:
-        name = 'Unknown'
+        name = bad_name
     if not mail:
-        mail = 'unknown'
+        mail = bad_mail
 
     return '%s <%s>' % (name, mail)
 
@@ -649,7 +692,7 @@ def do_export(parser):
 def main(args):
     global prefix, dirname, branches, bmarks
     global marks, blob_marks, parsed_refs
-    global peer, mode
+    global peer, mode, bad_mail, bad_name
 
     alias = args[1]
     url = args[2]
@@ -665,8 +708,12 @@ def main(args):
 
     if hg_git_compat:
         mode = 'hg'
+        bad_mail = 'none@none'
+        bad_name = ''
     else:
         mode = 'git'
+        bad_mail = 'unknown'
+        bad_name = 'Unknown'
 
     if alias[4:] == url:
         is_tmp = True
-- 
1.8.0

[PATCH v5 10/14] remote-hg: fake bookmark when there's none

[Felipe Contreras]

Or at least no current bookmark.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/git-remote-hg | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg
index c2efadf..c41ec95 100755
--- a/contrib/remote-hg/git-remote-hg
+++ b/contrib/remote-hg/git-remote-hg
@@ -422,12 +422,20 @@ def list_branch_head(repo, cur):
     g_head = (head, 'branches', repo[tip])
 
 def list_bookmark_head(repo, cur):
-    global g_head
+    global g_head, bmarks
 
     head = bookmarks.readcurrent(repo)
-    if not head:
-        return
-    node = repo[head]
+    if head:
+        node = repo[head]
+    else:
+        # fake bookmark from current branch
+        head = cur
+        tip = get_branch_tip(repo, head)
+        if not tip:
+            return
+        node = repo[tip]
+        bmarks[head] = node
+
     print "@refs/heads/%s HEAD" % head
     g_head = (head, 'bookmarks', node)
 
-- 
1.8.0

[PATCH v5 11/14] remote-hg: add support for fake remote

[Felipe Contreras]

Helpful while testing.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/git-remote-hg | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-hg/git-remote-hg b/contrib/remote-hg/git-remote-hg
index c41ec95..61c1072 100755
--- a/contrib/remote-hg/git-remote-hg
+++ b/contrib/remote-hg/git-remote-hg
@@ -248,7 +248,13 @@ def get_repo(url, alias):
     myui = ui.ui()
     myui.setconfig('ui', 'interactive', 'off')
 
-    if hg.islocal(url):
+    if url.startswith("remote://"):
+        remote = True
+        url = "file://%s" % url[9:]
+    else:
+        remote = False
+
+    if hg.islocal(url) and not remote:
         repo = hg.repository(myui, url)
     else:
         local_path = os.path.join(dirname, 'clone')
-- 
1.8.0

[PATCH v5 12/14] remote-hg: add biridectional tests

[Felipe Contreras]

Base commands from hg-git tests:
https://bitbucket.org/durin42/hg-git/src

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/Makefile |  13 +++
 contrib/remote-hg/test.sh  | 241 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 254 insertions(+)
 create mode 100644 contrib/remote-hg/Makefile
 create mode 100755 contrib/remote-hg/test.sh

diff --git a/contrib/remote-hg/Makefile b/contrib/remote-hg/Makefile
new file mode 100644
index 0000000..9a76575
--- /dev/null
+++ b/contrib/remote-hg/Makefile
@@ -0,0 +1,13 @@
+TESTS := $(wildcard test*.sh)
+
+export T := $(addprefix $(CURDIR)/,$(TESTS))
+export MAKE := $(MAKE) -e
+export PATH := $(CURDIR):$(PATH)
+
+test:
+	$(MAKE) -C ../../t $@
+
+$(TESTS):
+	$(MAKE) -C ../../t $(CURDIR)/$@
+
+.PHONY: $(TESTS)
diff --git a/contrib/remote-hg/test.sh b/contrib/remote-hg/test.sh
new file mode 100755
index 0000000..4ea2b24
--- /dev/null
+++ b/contrib/remote-hg/test.sh
@@ -0,0 +1,241 @@
+#!/bin/sh
+#
+# Copyright (c) 2012 Felipe Contreras
+#
+# Base commands from hg-git tests:
+# https://bitbucket.org/durin42/hg-git/src
+#
+
+test_description='Test remote-hg'
+
+. ./test-lib.sh
+
+if ! test_have_prereq PYTHON; then
+	skip_all='skipping remote-hg tests; python not available'
+	test_done
+fi
+
+if ! "$PYTHON_PATH" -c 'import mercurial'; then
+	skip_all='skipping remote-hg tests; mercurial not available'
+	test_done
+fi
+
+# clone to a git repo
+git_clone () {
+	hg -R $1 bookmark -f -r tip master &&
+	git clone -q "hg::$PWD/$1" $2
+}
+
+# clone to an hg repo
+hg_clone () {
+	(
+	hg init $2 &&
+	cd $1 &&
+	git push -q "hg::$PWD/../$2" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*'
+	) &&
+
+	(cd $2 && hg -q update)
+}
+
+# push an hg repo
+hg_push () {
+	(
+	cd $2
+	old=$(git symbolic-ref --short HEAD)
+	git checkout -q -b tmp &&
+	git fetch -q "hg::$PWD/../$1" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' &&
+	git checkout -q $old &&
+	git branch -q -D tmp 2> /dev/null || true
+	)
+}
+
+hg_log () {
+	hg -R $1 log --graph --debug | grep -v 'tag: *default/'
+}
+
+test_expect_success 'setup' '
+	(
+	echo "[ui]"
+	echo "username = A U Thor <author@example.com>"
+	echo "[defaults]"
+	echo "backout = -d \"0 0\""
+	echo "commit = -d \"0 0\""
+	echo "debugrawcommit = -d \"0 0\""
+	echo "tag = -d \"0 0\""
+	) >> "$HOME"/.hgrc &&
+	git config --global remote-hg.hg-git-compat true
+
+	export HGEDITOR=/usr/bin/true
+
+	export GIT_AUTHOR_DATE="2007-01-01 00:00:00 +0230"
+	export GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE"
+'
+
+test_expect_success 'encoding' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	git init -q gitrepo &&
+	cd gitrepo &&
+
+	echo alpha > alpha &&
+	git add alpha &&
+	git commit -m "add älphà" &&
+
+	export GIT_AUTHOR_NAME="tést èncödîng" &&
+	echo beta > beta &&
+	git add beta &&
+	git commit -m "add beta" &&
+
+	echo gamma > gamma &&
+	git add gamma &&
+	git commit -m "add gämmâ" &&
+
+	: TODO git config i18n.commitencoding latin-1 &&
+	echo delta > delta &&
+	git add delta &&
+	git commit -m "add déltà"
+	) &&
+
+	hg_clone gitrepo hgrepo &&
+	git_clone hgrepo gitrepo2 &&
+	hg_clone gitrepo2 hgrepo2 &&
+
+	HGENCODING=utf-8 hg_log hgrepo > expected &&
+	HGENCODING=utf-8 hg_log hgrepo2 > actual &&
+
+	test_cmp expected actual
+'
+
+test_expect_success 'file removal' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	git init -q gitrepo &&
+	cd gitrepo &&
+	echo alpha > alpha &&
+	git add alpha &&
+	git commit -m "add alpha" &&
+	echo beta > beta &&
+	git add beta &&
+	git commit -m "add beta"
+	mkdir foo &&
+	echo blah > foo/bar &&
+	git add foo &&
+	git commit -m "add foo" &&
+	git rm alpha &&
+	git commit -m "remove alpha" &&
+	git rm foo/bar &&
+	git commit -m "remove foo/bar"
+	) &&
+
+	hg_clone gitrepo hgrepo &&
+	git_clone hgrepo gitrepo2 &&
+	hg_clone gitrepo2 hgrepo2 &&
+
+	hg_log hgrepo > expected &&
+	hg_log hgrepo2 > actual &&
+
+	test_cmp expected actual
+'
+
+test_expect_success 'git tags' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	git init -q gitrepo &&
+	cd gitrepo &&
+	git config receive.denyCurrentBranch ignore &&
+	echo alpha > alpha &&
+	git add alpha &&
+	git commit -m "add alpha" &&
+	git tag alpha &&
+
+	echo beta > beta &&
+	git add beta &&
+	git commit -m "add beta" &&
+	git tag -a -m "added tag beta" beta
+	) &&
+
+	hg_clone gitrepo hgrepo &&
+	git_clone hgrepo gitrepo2 &&
+	hg_clone gitrepo2 hgrepo2 &&
+
+	hg_log hgrepo > expected &&
+	hg_log hgrepo2 > actual &&
+
+	test_cmp expected actual
+'
+
+test_expect_success 'hg branch' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	git init -q gitrepo &&
+	cd gitrepo &&
+
+	echo alpha > alpha &&
+	git add alpha &&
+	git commit -q -m "add alpha" &&
+	git checkout -q -b not-master
+	) &&
+
+	(
+	hg_clone gitrepo hgrepo &&
+
+	cd hgrepo &&
+	hg -q co master &&
+	hg mv alpha beta &&
+	hg -q commit -m "rename alpha to beta" &&
+	hg branch gamma | grep -v "permanent and global" &&
+	hg -q commit -m "started branch gamma"
+	) &&
+
+	hg_push hgrepo gitrepo &&
+	hg_clone gitrepo hgrepo2 &&
+
+	: TODO, avoid "master" bookmark &&
+	(cd hgrepo2 && hg checkout gamma) &&
+
+	hg_log hgrepo > expected &&
+	hg_log hgrepo2 > actual &&
+
+	test_cmp expected actual
+'
+
+test_expect_success 'hg tags' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	git init -q gitrepo &&
+	cd gitrepo &&
+
+	echo alpha > alpha &&
+	git add alpha &&
+	git commit -m "add alpha" &&
+	git checkout -q -b not-master
+	) &&
+
+	(
+	hg_clone gitrepo hgrepo &&
+
+	cd hgrepo &&
+	hg co master &&
+	hg tag alpha
+	) &&
+
+	hg_push hgrepo gitrepo &&
+	hg_clone gitrepo hgrepo2 &&
+
+	hg_log hgrepo > expected &&
+	hg_log hgrepo2 > actual &&
+
+	test_cmp expected actual
+'
+
+test_done
-- 
1.8.0

Re: [PATCH v5 12/14] remote-hg: add biridectional tests

[Felipe Contreras]

On Tue, Oct 30, 2012 at 5:47 AM, Junio C Hamano <gitster@pobox.com> wrote:
> What's the copyright status of the part this borrows from? Is there an
> in-file copyright notice needed to *name* the original author?
>
> The set-up part may become easier to read if done with here document.
>
> Pardon terseness, typo and HTML from a tablet.

I'm the original author. Some chunks are borrowed from the hg-git
project, but they had no copyright, I'll contact them and ask.

Cheers.

-- 
Felipe Contreras

[PATCH v5 13/14] remote-hg: add tests to compare with hg-git

[Felipe Contreras]

The base commands come from the tests of the hg-git project.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/test-hg-git.sh | 460 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 460 insertions(+)
 create mode 100755 contrib/remote-hg/test-hg-git.sh

diff --git a/contrib/remote-hg/test-hg-git.sh b/contrib/remote-hg/test-hg-git.sh
new file mode 100755
index 0000000..2b7acb0
--- /dev/null
+++ b/contrib/remote-hg/test-hg-git.sh
@@ -0,0 +1,460 @@
+#!/bin/sh
+#
+# Copyright (c) 2012 Felipe Contreras
+#
+# Base commands from hg-git tests:
+# https://bitbucket.org/durin42/hg-git/src
+#
+
+test_description='Test remote-hg output compared to hg-git'
+
+. ./test-lib.sh
+
+if ! test_have_prereq PYTHON; then
+	skip_all='skipping remote-hg tests; python not available'
+	test_done
+fi
+
+if ! "$PYTHON_PATH" -c 'import mercurial'; then
+	skip_all='skipping remote-hg tests; mercurial not available'
+	test_done
+fi
+
+if ! "$PYTHON_PATH" -c 'import hggit'; then
+	skip_all='skipping remote-hg tests; hg-git not available'
+	test_done
+fi
+
+# clone to a git repo with git
+git_clone_git () {
+	hg -R $1 bookmark -f -r tip master &&
+	git clone -q "hg::$PWD/$1" $2
+}
+
+# clone to an hg repo with git
+hg_clone_git () {
+	(
+	hg init $2 &&
+	cd $1 &&
+	git push -q "hg::$PWD/../$2" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*'
+	) &&
+
+	(cd $2 && hg -q update)
+}
+
+# clone to a git repo with hg
+git_clone_hg () {
+	(
+	git init -q $2 &&
+	cd $1 &&
+	hg bookmark -f -r tip master &&
+	hg -q push -r master ../$2 || true
+	)
+}
+
+# clone to an hg repo with hg
+hg_clone_hg () {
+	hg -q clone $1 $2
+}
+
+# push an hg repo with git
+hg_push_git () {
+	(
+	cd $2
+	old=$(git symbolic-ref --short HEAD)
+	git checkout -q -b tmp &&
+	git fetch -q "hg::$PWD/../$1" 'refs/tags/*:refs/tags/*' 'refs/heads/*:refs/heads/*' &&
+	git checkout -q $old &&
+	git branch -q -D tmp 2> /dev/null || true
+	)
+}
+
+# push an hg git repo with hg
+hg_push_hg () {
+	(
+	cd $1 &&
+	hg -q push ../$2 || true
+	)
+}
+
+hg_log () {
+	hg -R $1 log --graph --debug | grep -v 'tag: *default/'
+}
+
+git_log () {
+	git --git-dir=$1/.git fast-export --branches
+}
+
+test_expect_success 'setup' '
+	(
+	echo "[ui]"
+	echo "username = A U Thor <author@example.com>"
+	echo "[defaults]"
+	echo "backout = -d \"0 0\""
+	echo "commit = -d \"0 0\""
+	echo "debugrawcommit = -d \"0 0\""
+	echo "tag = -d \"0 0\""
+	echo "[extensions]"
+	echo "hgext.bookmarks ="
+	echo "hggit ="
+	) >> "$HOME"/.hgrc &&
+	git config --global receive.denycurrentbranch warn
+	git config --global remote-hg.hg-git-compat true
+
+	export HGEDITOR=/usr/bin/true
+
+	export GIT_AUTHOR_DATE="2007-01-01 00:00:00 +0230"
+	export GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE"
+'
+
+test_expect_success 'merge conflict 1' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	hg init hgrepo1 &&
+	cd hgrepo1 &&
+	echo A > afile &&
+	hg add afile &&
+	hg ci -m "origin" &&
+
+	echo B > afile &&
+	hg ci -m "A->B" &&
+
+	hg up -r0 &&
+	echo C > afile &&
+	hg ci -m "A->C" &&
+
+	hg merge -r1 || true &&
+	echo C > afile &&
+	hg resolve -m afile &&
+	hg ci -m "merge to C"
+	) &&
+
+	for x in hg git; do
+		git_clone_$x hgrepo1 gitrepo-$x &&
+		hg_clone_$x gitrepo-$x hgrepo2-$x &&
+		hg_log hgrepo2-$x > hg-log-$x &&
+		git_log gitrepo-$x > git-log-$x
+	done &&
+
+	test_cmp hg-log-hg hg-log-git &&
+	test_cmp git-log-hg git-log-git
+'
+
+test_expect_success 'merge conflict 2' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	hg init hgrepo1 &&
+	cd hgrepo1 &&
+	echo A > afile &&
+	hg add afile &&
+	hg ci -m "origin" &&
+
+	echo B > afile &&
+	hg ci -m "A->B" &&
+
+	hg up -r0 &&
+	echo C > afile &&
+	hg ci -m "A->C" &&
+
+	hg merge -r1 || true &&
+	echo B > afile &&
+	hg resolve -m afile &&
+	hg ci -m "merge to B"
+	) &&
+
+	for x in hg git; do
+		git_clone_$x hgrepo1 gitrepo-$x &&
+		hg_clone_$x gitrepo-$x hgrepo2-$x &&
+		hg_log hgrepo2-$x > hg-log-$x &&
+		git_log gitrepo-$x > git-log-$x
+	done &&
+
+	test_cmp hg-log-hg hg-log-git &&
+	test_cmp git-log-hg git-log-git
+'
+
+test_expect_success 'converged merge' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	hg init hgrepo1 &&
+	cd hgrepo1 &&
+	echo A > afile &&
+	hg add afile &&
+	hg ci -m "origin" &&
+
+	echo B > afile &&
+	hg ci -m "A->B" &&
+
+	echo C > afile &&
+	hg ci -m "B->C" &&
+
+	hg up -r0 &&
+	echo C > afile &&
+	hg ci -m "A->C" &&
+
+	hg merge -r2 || true &&
+	hg ci -m "merge"
+	) &&
+
+	for x in hg git; do
+		git_clone_$x hgrepo1 gitrepo-$x &&
+		hg_clone_$x gitrepo-$x hgrepo2-$x &&
+		hg_log hgrepo2-$x > hg-log-$x &&
+		git_log gitrepo-$x > git-log-$x
+	done &&
+
+	test_cmp hg-log-hg hg-log-git &&
+	test_cmp git-log-hg git-log-git
+'
+
+test_expect_success 'encoding' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	git init -q gitrepo &&
+	cd gitrepo &&
+
+	echo alpha > alpha &&
+	git add alpha &&
+	git commit -m "add älphà" &&
+
+	export GIT_AUTHOR_NAME="tést èncödîng" &&
+	echo beta > beta &&
+	git add beta &&
+	git commit -m "add beta" &&
+
+	echo gamma > gamma &&
+	git add gamma &&
+	git commit -m "add gämmâ" &&
+
+	: TODO git config i18n.commitencoding latin-1 &&
+	echo delta > delta &&
+	git add delta &&
+	git commit -m "add déltà"
+	) &&
+
+	for x in hg git; do
+		hg_clone_$x gitrepo hgrepo-$x &&
+		git_clone_$x hgrepo-$x gitrepo2-$x &&
+
+		HGENCODING=utf-8 hg_log hgrepo-$x > hg-log-$x &&
+		git_log gitrepo2-$x > git-log-$x
+	done &&
+
+	test_cmp hg-log-hg hg-log-git &&
+	test_cmp git-log-hg git-log-git
+'
+
+test_expect_success 'file removal' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	git init -q gitrepo &&
+	cd gitrepo &&
+	echo alpha > alpha &&
+	git add alpha &&
+	git commit -m "add alpha" &&
+	echo beta > beta &&
+	git add beta &&
+	git commit -m "add beta"
+	mkdir foo &&
+	echo blah > foo/bar &&
+	git add foo &&
+	git commit -m "add foo" &&
+	git rm alpha &&
+	git commit -m "remove alpha" &&
+	git rm foo/bar &&
+	git commit -m "remove foo/bar"
+	) &&
+
+	for x in hg git; do
+		(
+		hg_clone_$x gitrepo hgrepo-$x &&
+		cd hgrepo-$x &&
+		hg_log . &&
+		hg manifest -r 3 &&
+		hg manifest
+		) > output-$x &&
+
+		git_clone_$x hgrepo-$x gitrepo2-$x &&
+		git_log gitrepo2-$x > log-$x
+	done &&
+
+	test_cmp output-hg output-git &&
+	test_cmp log-hg log-git
+'
+
+test_expect_success 'git tags' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	(
+	git init -q gitrepo &&
+	cd gitrepo &&
+	git config receive.denyCurrentBranch ignore &&
+	echo alpha > alpha &&
+	git add alpha &&
+	git commit -m "add alpha" &&
+	git tag alpha &&
+
+	echo beta > beta &&
+	git add beta &&
+	git commit -m "add beta" &&
+	git tag -a -m "added tag beta" beta
+	) &&
+
+	for x in hg git; do
+		hg_clone_$x gitrepo hgrepo-$x &&
+		hg_log hgrepo-$x > log-$x
+	done &&
+
+	test_cmp log-hg log-git
+'
+
+test_expect_success 'hg author' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	for x in hg git; do
+		(
+		git init -q gitrepo-$x &&
+		cd gitrepo-$x &&
+
+		echo alpha > alpha &&
+		git add alpha &&
+		git commit -m "add alpha" &&
+		git checkout -q -b not-master
+		) &&
+
+		(
+		hg_clone_$x gitrepo-$x hgrepo-$x &&
+		cd hgrepo-$x &&
+
+		hg co master &&
+		echo beta > beta &&
+		hg add beta &&
+		hg commit -u "test" -m "add beta" &&
+
+		echo gamma >> beta &&
+		hg commit -u "test <test@example.com> (comment)" -m "modify beta" &&
+
+		echo gamma > gamma &&
+		hg add gamma &&
+		hg commit -u "<test@example.com>" -m "add gamma" &&
+
+		echo delta > delta &&
+		hg add delta &&
+		hg commit -u "name<test@example.com>" -m "add delta" &&
+
+		echo epsilon > epsilon &&
+		hg add epsilon &&
+		hg commit -u "name <test@example.com" -m "add epsilon" &&
+
+		echo zeta > zeta &&
+		hg add zeta &&
+		hg commit -u " test " -m "add zeta" &&
+
+		echo eta > eta &&
+		hg add eta &&
+		hg commit -u "test < test@example.com >" -m "add eta" &&
+
+		echo theta > theta &&
+		hg add theta &&
+		hg commit -u "test >test@example.com>" -m "add theta"
+		) &&
+
+		hg_push_$x hgrepo-$x gitrepo-$x &&
+		hg_clone_$x gitrepo-$x hgrepo2-$x &&
+
+		hg_log hgrepo2-$x > hg-log-$x &&
+		git_log gitrepo-$x > git-log-$x
+	done &&
+
+	test_cmp git-log-hg git-log-git &&
+
+	test_cmp hg-log-hg hg-log-git &&
+	test_cmp git-log-hg git-log-git
+'
+
+test_expect_success 'hg branch' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	for x in hg git; do
+		(
+		git init -q gitrepo-$x &&
+		cd gitrepo-$x &&
+
+		echo alpha > alpha &&
+		git add alpha &&
+		git commit -q -m "add alpha" &&
+		git checkout -q -b not-master
+		) &&
+
+		(
+		hg_clone_$x gitrepo-$x hgrepo-$x &&
+
+		cd hgrepo-$x &&
+		hg -q co master &&
+		hg mv alpha beta &&
+		hg -q commit -m "rename alpha to beta" &&
+		hg branch gamma | grep -v "permanent and global" &&
+		hg -q commit -m "started branch gamma"
+		) &&
+
+		hg_push_$x hgrepo-$x gitrepo-$x &&
+		hg_clone_$x gitrepo-$x hgrepo2-$x &&
+
+		hg_log hgrepo2-$x > hg-log-$x &&
+		git_log gitrepo-$x > git-log-$x
+	done &&
+
+	test_cmp hg-log-hg hg-log-git &&
+	test_cmp git-log-hg git-log-git
+'
+
+test_expect_success 'hg tags' '
+	mkdir -p tmp && cd tmp &&
+	test_when_finished "cd .. && rm -rf tmp" &&
+
+	for x in hg git; do
+		(
+		git init -q gitrepo-$x &&
+		cd gitrepo-$x &&
+
+		echo alpha > alpha &&
+		git add alpha &&
+		git commit -m "add alpha" &&
+		git checkout -q -b not-master
+		) &&
+
+		(
+		hg_clone_$x gitrepo-$x hgrepo-$x &&
+
+		cd hgrepo-$x &&
+		hg co master &&
+		hg tag alpha
+		) &&
+
+		hg_push_$x hgrepo-$x gitrepo-$x &&
+		hg_clone_$x gitrepo-$x hgrepo2-$x &&
+
+		(
+		git --git-dir=gitrepo-$x/.git tag -l &&
+		hg_log hgrepo2-$x &&
+		cat hgrepo2-$x/.hgtags
+		) > output-$x
+	done &&
+
+	test_cmp output-hg output-git
+'
+
+test_done
-- 
1.8.0

[PATCH v5 14/14] remote-hg: add extra author test

[Felipe Contreras]

For hg.hg.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
---
 contrib/remote-hg/test-hg-git.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/contrib/remote-hg/test-hg-git.sh b/contrib/remote-hg/test-hg-git.sh
index 2b7acb0..a9f5cb2 100755
--- a/contrib/remote-hg/test-hg-git.sh
+++ b/contrib/remote-hg/test-hg-git.sh
@@ -368,7 +368,11 @@ test_expect_success 'hg author' '
 
 		echo theta > theta &&
 		hg add theta &&
-		hg commit -u "test >test@example.com>" -m "add theta"
+		hg commit -u "test >test@example.com>" -m "add theta" &&
+
+		echo iota > iota &&
+		hg add iota &&
+		hg commit -u "test <test <at> example <dot> com>" -m "add iota"
 		) &&
 
 		hg_push_$x hgrepo-$x gitrepo-$x &&
-- 
1.8.0

Re: [PATCH v5 00/14] New remote-hg helper

[Chris Webb]

Hi. I routinely work with projects in both hg and git, so I'm really
interested in this. Thanks for working on it! I grabbed the latest version
from

  https://github.com/felipec/git/blob/fc-remote-hg/contrib/remote-hg/git-remote-hg

and have been trying it out. For the most part, it seems to work very nicely
for the hg repos I have access to and can test against. I've spotted a couple
of issues along the way that I thought would be worth reporting.

The first is really a symptom of a general difference between hg and git: an hg
repository can have multiple heads, whereas a git repo has exactly one head. To
demonstrate:

  $ hg init hgtest && cd hgtest
  $ echo zero >foo && hg add foo && hg commit -m zero
  $ echo one >foo && hg commit -m one
  $ hg checkout -r 0
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  $ echo two >foo && hg commit -m two
  created new head
  $ hg log --graph
  @  changeset:   2:ca09651009cb
  |  tag:         tip
  |  parent:      0:9f552c53d116
  |  user:        Chris Webb <chris@arachsys.com>
  |  date:        Tue Oct 30 09:33:38 2012 +0000
  |  summary:     two
  |
  | o  changeset:   1:58fad8998339
  |/   user:        Chris Webb <chris@arachsys.com>
  |    date:        Tue Oct 30 09:33:25 2012 +0000
  |    summary:     one
  |
  o  changeset:   0:9f552c53d116
     user:        Chris Webb <chris@arachsys.com>
     date:        Tue Oct 30 09:33:08 2012 +0000
     summary:     zero

  $ cd ..

Now if I try to convert this:

  $ git clone hg::$PWD/hgtest gittest
  Cloning into 'gittest'...
  WARNING: Branch 'default' has more than one head, consider merging
  Traceback (most recent call last):
    File "/home/chris/bin/git-remote-hg", line 773, in <module>
      sys.exit(main(sys.argv))
    File "/home/chris/bin/git-remote-hg", line 759, in main
      do_list(parser)
    File "/home/chris/bin/git-remote-hg", line 463, in do_list
      list_branch_head(repo, cur)
    File "/home/chris/bin/git-remote-hg", line 425, in list_branch_head
      tip = get_branch_tip(repo, cur)
    File "/home/chris/bin/git-remote-hg", line 418, in get_branch_tip
      return repo.branchtip(branch)
  AttributeError: 'mqrepo' object has no attribute 'branchtip'

Strip the second head and it's fine:

  $ hg -R hgtest strip 2
  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
  saved backup bundle to /tmp/hgtest/hgtest/.hg/strip-backup/ca09651009cb-backup.hg
  $ git clone hg::$PWD/hgtest gittest
  Cloning into 'gittest'...
  $

Not sure what the most friendly thing to do here is. Perhaps refuse to
clone/pull from a repo with multiple heads unless you name the specific head
you want?


The second thing I spotted is the behaviour of bookmarks on push:

  $ hg init hgtest && cd hgtest
  $ echo zero >foo && hg add foo && hg commit -m zero
  $ hg bookmark development
  $ cd ..
  $ git clone hg::$PWD/hgtest gittest && cd gittest
  Cloning into 'gittest'...
  $ git checkout development
  Branch development set up to track remote branch development from origin.
  Switched to a new branch 'development'
  $ echo one >foo && git add foo && git commit -m one
  [development 9f67dc4] one
   1 file changed, 1 insertion(+), 1 deletion(-)
  $ git status
  # On branch development
  # Your branch is ahead of 'origin/development' by 1 commit.
  #
  nothing to commit
  $ git push
  warning: helper reported unexpected status of refs/hg/origin/bookmarks/development
  To hg::/tmp/hgtest/hgtest
   * [new branch]      branches/default -> branches/default
   * [new branch]      development -> development
  $ hg log -R ../hgtest
  changeset:   1:1c0714d93864
  tag:         tip
  user:        Chris Webb <chris@arachsys.com>
  date:        Tue Oct 30 09:51:51 2012 +0000
  summary:     one

  changeset:   0:f56c463398ea
  bookmark:    development
  user:        Chris Webb <chris@arachsys.com>
  date:        Tue Oct 30 09:50:53 2012 +0000
  summary:     zero

i.e. the development bookmark hasn't been updated by the push. This might be
connected to the warning message

  warning: helper reported unexpected status of refs/hg/origin/bookmarks/development

I'm testing with hg 2.2.2 and current git master, so I expect this could be a
python api change in the more recent versions of hg if you don't see the same
behaviour.

Best wishes,

Chris.

Re: [PATCH v5 00/14] New remote-hg helper

[Chris Webb]

Chris Webb <chris@arachsys.com> writes:

> The first is really a symptom of a general difference between hg and git: an hg
> repository can have multiple heads, whereas a git repo has exactly one head.

By this I mean an hg repository without bookmarks or branches can still have
multiple heads, whereas a git branch points at exactly one place. Sorry,
very vague description there!

Cheers,

Chris.

Re: [PATCH v5 00/14] New remote-hg helper

[Felipe Contreras]

On Tue, Oct 30, 2012 at 11:25 AM, Chris Webb <chris@arachsys.com> wrote:
> Hi. I routinely work with projects in both hg and git, so I'm really
> interested in this. Thanks for working on it! I grabbed the latest version
> from
>
>   https://github.com/felipec/git/blob/fc-remote-hg/contrib/remote-hg/git-remote-hg
>
> and have been trying it out. For the most part, it seems to work very nicely
> for the hg repos I have access to and can test against. I've spotted a couple
> of issues along the way that I thought would be worth reporting.

Great!

> The first is really a symptom of a general difference between hg and git: an hg
> repository can have multiple heads, whereas a git repo has exactly one head. To
> demonstrate:

> Now if I try to convert this:
>
>   $ git clone hg::$PWD/hgtest gittest
>   Cloning into 'gittest'...
>   WARNING: Branch 'default' has more than one head, consider merging
>   Traceback (most recent call last):
>     File "/home/chris/bin/git-remote-hg", line 773, in <module>
>       sys.exit(main(sys.argv))
>     File "/home/chris/bin/git-remote-hg", line 759, in main
>       do_list(parser)
>     File "/home/chris/bin/git-remote-hg", line 463, in do_list
>       list_branch_head(repo, cur)
>     File "/home/chris/bin/git-remote-hg", line 425, in list_branch_head
>       tip = get_branch_tip(repo, cur)
>     File "/home/chris/bin/git-remote-hg", line 418, in get_branch_tip
>       return repo.branchtip(branch)
>   AttributeError: 'mqrepo' object has no attribute 'branchtip'

Yes, it seems this is an API issue; repo.branchtip doesn't exist in
python 2.2. I've added a check for that, and it should work fine now.
We'll be picking a random head (the first one), but the user has been
warned anyway.

> The second thing I spotted is the behaviour of bookmarks on push:

> i.e. the development bookmark hasn't been updated by the push. This might be
> connected to the warning message

This is not an API issue, this is a bug; bookmarks are not updated
(only the first creation works). I've fixed this as well, and added a
test with the example you put above:

http://github.com/felipec/git/commit/d006d54fc444484707dffa24d9fad053e574918d

Both issues should be fixed now :)

Cheers.

-- 
Felipe Contreras

Re: [PATCH v5 00/14] New remote-hg helper

[Chris Webb]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> Yes, it seems this is an API issue; repo.branchtip doesn't exist in
> python 2.2.

Hi. Presumably this is a problem with old mercurial not a problem with old
python as mentioned in the commit?

> Both issues should be fixed now :)

They are indeed, and it now works nicely on all the repos I've tested it
with, including http://selenic.com/hg: very impressive!

I wonder whether it's worth ignoring heads with bookmarks pointing to them
when it comes to considering heads of branches, or at least allowing the
hg branch tracking to be easily disabled?

A common idiom when working with hg bookmarks is to completely ignore the
(not very useful) hg branches (i.e. all commits are on the default hg
branch) and have a bookmark for each line of development used exactly as a
git branch would be.

On such a repository, at the moment you will always get a warning about
multiple heads on branches/default, even though you actually don't care
about branches/default (and would prefer it not to exist) and just want the
branches coming from the bookmarks.

I've also seen repositories with no hg branches, but with a single
unbookmarked tip and bookmarks on some other heads to mark non-mainline
development. It would be nice for branches/default to track the unbookmarked
tip in this case, without warning about the other, bookmarked heads.

Finally, on a simple repo with no branches and where there's no clash with a
bookmark called master, I'd love to be able to a get a more idiomatic
origin/master rather than origin/branches/default.

Just some idle thoughts...

Best wishes,

Chris.

Re: [PATCH v5 00/14] New remote-hg helper

[Chris Webb]

Chris Webb <chris@arachsys.com> writes:

> A common idiom when working with hg bookmarks is to completely ignore the
> (not very useful) hg branches (i.e. all commits are on the default hg
> branch) and have a bookmark for each line of development used exactly as a
> git branch would be.
> 
> On such a repository, at the moment you will always get a warning about
> multiple heads on branches/default, even though you actually don't care
> about branches/default (and would prefer it not to exist) and just want the
> branches coming from the bookmarks.

Something which you can do with hg clone is

  hg clone http://my.repo/foo#master

to clone just the history behind the master bookmark from foo. This works
nicely with git-remote-hg too:

  git clone hg::http://my.repo/foo#master

gives you just origin/master and origin/branches/default, not
origin/otherbookmark. This is a case where it would be particularly nice to
be able to kill origin/branches/default and just keep the identical
origin/master.

Cheers,

Chris.

Re: [PATCH v5 00/14] New remote-hg helper

[Felipe Contreras]

On Tue, Oct 30, 2012 at 7:00 PM, Chris Webb <chris@arachsys.com> wrote:
> Felipe Contreras <felipe.contreras@gmail.com> writes:
>
>> Yes, it seems this is an API issue; repo.branchtip doesn't exist in
>> python 2.2.
>
> Hi. Presumably this is a problem with old mercurial not a problem with old
> python as mentioned in the commit?
>
>> Both issues should be fixed now :)
>
> They are indeed, and it now works nicely on all the repos I've tested it
> with, including http://selenic.com/hg: very impressive!
>
> I wonder whether it's worth ignoring heads with bookmarks pointing to them
> when it comes to considering heads of branches, or at least allowing the
> hg branch tracking to be easily disabled?
>
> A common idiom when working with hg bookmarks is to completely ignore the
> (not very useful) hg branches (i.e. all commits are on the default hg
> branch) and have a bookmark for each line of development used exactly as a
> git branch would be.
>
> On such a repository, at the moment you will always get a warning about
> multiple heads on branches/default, even though you actually don't care
> about branches/default (and would prefer it not to exist) and just want the
> branches coming from the bookmarks.
>
> I've also seen repositories with no hg branches, but with a single
> unbookmarked tip and bookmarks on some other heads to mark non-mainline
> development. It would be nice for branches/default to track the unbookmarked
> tip in this case, without warning about the other, bookmarked heads.
>
> Finally, on a simple repo with no branches and where there's no clash with a
> bookmark called master, I'd love to be able to a get a more idiomatic
> origin/master rather than origin/branches/default.

Sounds like you might want to try this:
% git config --global remote-hg.hg-git-compat true

This would match the behavior of hg-git, which seems to be what you
are looking for: branches will be ignored. But additionally there will
be other obtrusive changes:

 1) Extra information will be stored in the commit message: branch,
renames, extra info.

Personally I don't like this, which is why it's only enabled with the
hg-git compat mode. Eventually there will be support for this in the
normal mode through git notes, and eventually (hopefully), hg-git will
use notes as well.

 2) Authors will be fixed trying to preserve as much information as possible


The problem with this mode is what happens when there are no
bookmarks, and what I decided to do is create a fake bookmark to the
current branch (e.g. default), so you would get 'origin/default', but
you will be warned if there are multiple heads. We could try to create
a 'master' ref to the 'tip', but that might not be what the user
wants: (s)he might switch to a certain branch, and clone the
repository in git; currently the result working directory would be the
same as in hg, but if we point to 'tip', then it would not.

Maybe we should have a branch pointing to 'tip' either way.

The reason I decided on having 'branches/default' on the normal mode
is that users unfamiliar with mercurial might expect to see all the
branches somewhere. But perhaps that should happen only for other
branches, while 'default' gets a special treatment and gets promoted
to 'origin/master' but only if there's no 'master' bookmark, as you
say. Now the tricky part would be handling the case where there was no
'master' bookmark, and suddenly one gets created, but it might be
overkill to think about that right now.

Either way, try the hg-git-compat mode, should be close :)

Cheers.

-- 
Felipe Contreras

Re: [PATCH v5 00/14] New remote-hg helper

[Felipe Contreras]

On Tue, Oct 30, 2012 at 7:00 PM, Chris Webb <chris@arachsys.com> wrote:
> Felipe Contreras <felipe.contreras@gmail.com> writes:
>
>> Yes, it seems this is an API issue; repo.branchtip doesn't exist in
>> python 2.2.
>
> Hi. Presumably this is a problem with old mercurial not a problem with old
> python as mentioned in the commit?

Yeah, mercurial.

>> Both issues should be fixed now :)
>
> They are indeed, and it now works nicely on all the repos I've tested it
> with, including http://selenic.com/hg: very impressive!
>
> I wonder whether it's worth ignoring heads with bookmarks pointing to them
> when it comes to considering heads of branches, or at least allowing the
> hg branch tracking to be easily disabled?
>
> A common idiom when working with hg bookmarks is to completely ignore the
> (not very useful) hg branches (i.e. all commits are on the default hg
> branch) and have a bookmark for each line of development used exactly as a
> git branch would be.
>
> On such a repository, at the moment you will always get a warning about
> multiple heads on branches/default, even though you actually don't care
> about branches/default (and would prefer it not to exist) and just want the
> branches coming from the bookmarks.
>
> I've also seen repositories with no hg branches, but with a single
> unbookmarked tip and bookmarks on some other heads to mark non-mainline
> development. It would be nice for branches/default to track the unbookmarked
> tip in this case, without warning about the other, bookmarked heads.

Implemented now. I'm not handling the 'tip' revision, but most likely
it's also the '.' revision. In this case a fake 'master' bookmark will
be created to track that revision.

> Finally, on a simple repo with no branches and where there's no clash with a
> bookmark called master, I'd love to be able to a get a more idiomatic
> origin/master rather than origin/branches/default.

Yeah, you can get that now. If there are no branches or bookmarks,
'master' will point to '.'.

As for your preference of not tracking branches, there's a new option:

% git config --global remote-hg.track-branches false

So you don't have to use hg-git-compat mode :)

Cheers.

-- 
Felipe Contreras

Re: [PATCH v5 00/14] New remote-hg helper

[Chris Webb]

Felipe Contreras <felipe.contreras@gmail.com> writes:

> Implemented now. I'm not handling the 'tip' revision, but most likely
> it's also the '.' revision. In this case a fake 'master' bookmark will
> be created to track that revision.

Hi Felipe. Sorry for the slow response, I've been snowed under with work and
have only just got around to testing your latest version.

The new remote-hg.track-branches=false option is great and does exactly what
I was hoping for. For the benefit of the list archives, one natural way to
use it is

  git clone -c remote-hg.track-branches=false hg::foo

when cloning the relevant repositories, if you don't want the setting
globally for every hg-remote clone.

During testing, I've seen some strange behaviour which I think is caused by
using the . revision instead of tip:

$ hg init h
$ hg init h2
$ ( cd h && touch foo && hg add foo && hg commit -m foo && hg push ../h2 )
pushing to ../h2
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
$ git clone hg::h g
Cloning into 'g'...
$ git clone hg::h2 g2
Cloning into 'g2'...
warning: remote HEAD refers to nonexistent ref, unable to checkout.
$

The reason for this is that by default . == null (not tip) in the repo h2
which we pushed into from h. The hg equivalent of a bare repo typically has a
null checkout like this. (Actually, the checkout of HEAD seems to break
whenever . is different from tip, not just when it's null as in this example.)

Apart from this, everything seems solid and works well. Really useful; thanks!

Best wishes,

Chris.

Re: [PATCH v5 00/14] New remote-hg helper

[Felipe Contreras]

On Sun, Nov 11, 2012 at 11:17 PM, Chris Webb <chris@arachsys.com> wrote:
> Felipe Contreras <felipe.contreras@gmail.com> writes:
>
>> Implemented now. I'm not handling the 'tip' revision, but most likely
>> it's also the '.' revision. In this case a fake 'master' bookmark will
>> be created to track that revision.
>
> Hi Felipe. Sorry for the slow response, I've been snowed under with work and
> have only just got around to testing your latest version.
>
> The new remote-hg.track-branches=false option is great and does exactly what
> I was hoping for. For the benefit of the list archives, one natural way to
> use it is
>
>   git clone -c remote-hg.track-branches=false hg::foo
>
> when cloning the relevant repositories, if you don't want the setting
> globally for every hg-remote clone.

Cool :)

> During testing, I've seen some strange behaviour which I think is caused by
> using the . revision instead of tip:
>
> $ hg init h
> $ hg init h2
> $ ( cd h && touch foo && hg add foo && hg commit -m foo && hg push ../h2 )
> pushing to ../h2
> searching for changes
> adding changesets
> adding manifests
> adding file changes
> added 1 changesets with 1 changes to 1 files
> $ git clone hg::h g
> Cloning into 'g'...
> $ git clone hg::h2 g2
> Cloning into 'g2'...
> warning: remote HEAD refers to nonexistent ref, unable to checkout.
> $
>
> The reason for this is that by default . == null (not tip) in the repo h2
> which we pushed into from h. The hg equivalent of a bare repo typically has a
> null checkout like this. (Actually, the checkout of HEAD seems to break
> whenever . is different from tip, not just when it's null as in this example.)

Well, I thought in those cases we didn't want HEAD to be updated.
People can still use the repo and checkout whatever branch they want.
But '.' is not  really the equivalent of HEAD. Since there's no
equivalent, I think it makes sense to try first '.', and then 'tip'.
This means that we would never have a HEAD pointing to nowhere.

I've added a patch for that.

Cheers.
-- 
Felipe Contreras

Re: [PATCH v5 00/14] New remote-hg helper

[Johannes Schindelin]

Hi Chris,

On Tue, 30 Oct 2012, Chris Webb wrote:

> I routinely work with projects in both hg and git, so I'm really
> interested in this. Thanks for working on it! I grabbed the latest
> version from
> 
>   https://github.com/felipec/git/blob/fc-remote-hg/contrib/remote-hg/git-remote-hg
> 
> and have been trying it out.

You might also want to try out the 'devel' branch of
https://github.com/msysgit/git. It is in production use here.

Ciao,
Johannes