git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [StGit PATCH] Add the --merged option to goto
@ 2009-03-20 16:15 Catalin Marinas
  2009-03-23  8:45 ` Karl Hasselström
  0 siblings, 1 reply; 10+ messages in thread
From: Catalin Marinas @ 2009-03-20 16:15 UTC (permalink / raw)
  To: git, Karl Hasselström

This patch adds support for checking which patches were already merged
upstream. This checking is done by trying to reverse-apply the patches
in the index before pushing them onto the stack. The trivial merge cases
in Index.merge() are ignored when performing this operation otherwise
the results could be wrong (e.g. a patch adding a hunk and a subsequent
patch canceling the previous change would both be considered merged).

Signed-off-by: Catalin Marinas <catalin.marinas@gmail.com>
---

This is in preparation for the updating of the push command where we
have this functionality (I think we had it for goto as well but was lost
with the update to stgit.lib). Test cases with --merged are already done
for the push command, so I haven't added any for goto (but I'll push
this patch only after push is updated).

 stgit/argparse.py        |    4 ++++
 stgit/commands/goto.py   |   12 +++++++++---
 stgit/lib/git.py         |   15 ++++++++-------
 stgit/lib/transaction.py |   42 +++++++++++++++++++++++++++++++++++-------
 4 files changed, 56 insertions(+), 17 deletions(-)

diff --git a/stgit/argparse.py b/stgit/argparse.py
index 85ee6e3..765579c 100644
--- a/stgit/argparse.py
+++ b/stgit/argparse.py
@@ -225,6 +225,10 @@ def keep_option():
                 short = 'Keep the local changes',
                 default = config.get('stgit.autokeep') == 'yes')]
 
+def merged_option():
+    return [opt('-m', '--merged', action = 'store_true',
+                short = 'Check for patches merged upstream')]
+
 class CompgenBase(object):
     def actions(self, var): return set()
     def words(self, var): return set()
diff --git a/stgit/commands/goto.py b/stgit/commands/goto.py
index 66f49df..839b75c 100644
--- a/stgit/commands/goto.py
+++ b/stgit/commands/goto.py
@@ -28,7 +28,7 @@ Push/pop patches to/from the stack until the one given on the command
 line becomes current."""
 
 args = [argparse.other_applied_patches, argparse.unapplied_patches]
-options = argparse.keep_option()
+options = argparse.keep_option() + argparse.merged_option()
 
 directory = common.DirectoryHasRepositoryLib()
 
@@ -47,8 +47,14 @@ def func(parser, options, args):
         assert not trans.pop_patches(lambda pn: pn in to_pop)
     elif patch in trans.unapplied:
         try:
-            for pn in trans.unapplied[:trans.unapplied.index(patch)+1]:
-                trans.push_patch(pn, iw, allow_interactive = True)
+            to_push = trans.unapplied[:trans.unapplied.index(patch)+1]
+            if options.merged:
+                merged = set(trans.check_merged(to_push))
+            else:
+                merged = set()
+            for pn in to_push:
+                trans.push_patch(pn, iw, allow_interactive = True,
+                                 already_merged = pn in merged)
         except transaction.TransactionHalted:
             pass
     elif patch in trans.hidden:
diff --git a/stgit/lib/git.py b/stgit/lib/git.py
index e0a3c96..875e352 100644
--- a/stgit/lib/git.py
+++ b/stgit/lib/git.py
@@ -732,7 +732,7 @@ class Index(RunWithEnv):
         # to use --binary.
         self.apply(self.__repository.diff_tree(tree1, tree2, ['--full-index']),
                    quiet)
-    def merge(self, base, ours, theirs, current = None):
+    def merge(self, base, ours, theirs, current = None, check_trivial = True):
         """Use the index (and only the index) to do a 3-way merge of the
         L{Tree}s C{base}, C{ours} and C{theirs}. The merge will either
         succeed (in which case the first half of the return value is
@@ -752,12 +752,13 @@ class Index(RunWithEnv):
         assert current == None or isinstance(current, Tree)
 
         # Take care of the really trivial cases.
-        if base == ours:
-            return (theirs, current)
-        if base == theirs:
-            return (ours, current)
-        if ours == theirs:
-            return (ours, current)
+        if check_trivial:
+            if base == ours:
+                return (theirs, current)
+            if base == theirs:
+                return (ours, current)
+            if ours == theirs:
+                return (ours, current)
 
         if current == theirs:
             # Swap the trees. It doesn't matter since merging is
diff --git a/stgit/lib/transaction.py b/stgit/lib/transaction.py
index b146648..8cd5e50 100644
--- a/stgit/lib/transaction.py
+++ b/stgit/lib/transaction.py
@@ -297,7 +297,8 @@ class StackTransaction(object):
                     out.info('Deleted %s%s' % (pn, s))
         return popped
 
-    def push_patch(self, pn, iw = None, allow_interactive = False):
+    def push_patch(self, pn, iw = None, allow_interactive = False,
+                   already_merged = False):
         """Attempt to push the named patch. If this results in conflicts,
         halts the transaction. If index+worktree are given, spill any
         conflicts to them."""
@@ -305,11 +306,14 @@ class StackTransaction(object):
         cd = orig_cd.set_committer(None)
         oldparent = cd.parent
         cd = cd.set_parent(self.top)
-        base = oldparent.data.tree
-        ours = cd.parent.data.tree
-        theirs = cd.tree
-        tree, self.temp_index_tree = self.temp_index.merge(
-            base, ours, theirs, self.temp_index_tree)
+        if already_merged:
+            tree = cd.tree
+        else:
+            base = oldparent.data.tree
+            ours = cd.parent.data.tree
+            theirs = cd.tree
+            tree, self.temp_index_tree = self.temp_index.merge(
+                base, ours, theirs, self.temp_index_tree)
         s = ''
         merge_conflict = False
         if not tree:
@@ -341,7 +345,9 @@ class StackTransaction(object):
         else:
             comm = None
             s = ' (unmodified)'
-        if not merge_conflict and cd.is_nochange():
+        if already_merged:
+            s = ' (merged)'
+        elif not merge_conflict and cd.is_nochange():
             s = ' (empty)'
         out.info('Pushed %s%s' % (pn, s))
         def update():
@@ -379,3 +385,25 @@ class StackTransaction(object):
         assert set(self.unapplied + self.hidden) == set(unapplied + hidden)
         self.unapplied = unapplied
         self.hidden = hidden
+
+    def check_merged(self, patches):
+        """Return a subset of patches already merged."""
+        merged = []
+        temp_index = self.__stack.repository.temp_index()
+        temp_index_tree = None
+        ours = self.stack.head.data.tree
+
+        for pn in reversed(patches):
+            # check whether patch changes can be reversed in the current tree
+            cd = self.patches[pn].data
+            base = cd.tree
+            theirs = cd.parent.data.tree
+            tree, temp_index_tree = \
+                    temp_index.merge(base, ours, theirs, temp_index_tree,
+                                     check_trivial = False)
+            if tree:
+                merged.append(pn)
+                ours = tree
+
+        temp_index.delete()
+        return merged

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [StGit PATCH] Add the --merged option to goto
  2009-03-20 16:15 [StGit PATCH] Add the --merged option to goto Catalin Marinas
@ 2009-03-23  8:45 ` Karl Hasselström
  2009-03-23 16:33   ` Catalin Marinas
  0 siblings, 1 reply; 10+ messages in thread
From: Karl Hasselström @ 2009-03-23  8:45 UTC (permalink / raw)
  To: Catalin Marinas; +Cc: git

On 2009-03-20 16:15:45 +0000, Catalin Marinas wrote:

> This patch adds support for checking which patches were already
> merged upstream. This checking is done by trying to reverse-apply
> the patches in the index before pushing them onto the stack. The
> trivial merge cases in Index.merge() are ignored when performing
> this operation otherwise the results could be wrong (e.g. a patch
> adding a hunk and a subsequent patch canceling the previous change
> would both be considered merged).
>
> Signed-off-by: Catalin Marinas <catalin.marinas@gmail.com>
> ---
>
> This is in preparation for the updating of the push command where we
> have this functionality (I think we had it for goto as well but was
> lost with the update to stgit.lib). Test cases with --merged are
> already done for the push command, so I haven't added any for goto
> (but I'll push this patch only after push is updated).

Looks good, except for a few things:

> @@ -732,7 +732,7 @@ class Index(RunWithEnv):
>          # to use --binary.
>          self.apply(self.__repository.diff_tree(tree1, tree2, ['--full-index']),
>                     quiet)
> -    def merge(self, base, ours, theirs, current = None):
> +    def merge(self, base, ours, theirs, current = None, check_trivial = True):
>          """Use the index (and only the index) to do a 3-way merge of the
>          L{Tree}s C{base}, C{ours} and C{theirs}. The merge will either
>          succeed (in which case the first half of the return value is

Please update the documentation with your new option. :-)

> @@ -752,12 +752,13 @@ class Index(RunWithEnv):
>          assert current == None or isinstance(current, Tree)
>  
>          # Take care of the really trivial cases.
> -        if base == ours:
> -            return (theirs, current)
> -        if base == theirs:
> -            return (ours, current)
> -        if ours == theirs:
> -            return (ours, current)
> +        if check_trivial:
> +            if base == ours:
> +                return (theirs, current)
> +            if base == theirs:
> +                return (ours, current)
> +            if ours == theirs:
> +                return (ours, current)

Uh, what? What's the point of not doing this unconditionally?

> @@ -379,3 +385,25 @@ class StackTransaction(object):
>          assert set(self.unapplied + self.hidden) == set(unapplied + hidden)
>          self.unapplied = unapplied
>          self.hidden = hidden
> +
> +    def check_merged(self, patches):
> +        """Return a subset of patches already merged."""
> +        merged = []
> +        temp_index = self.__stack.repository.temp_index()
> +        temp_index_tree = None

There's no need to create a new temp index here. The transaction
object already has one.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [StGit PATCH] Add the --merged option to goto
  2009-03-23  8:45 ` Karl Hasselström
@ 2009-03-23 16:33   ` Catalin Marinas
  2009-03-24 13:16     ` Karl Hasselström
  0 siblings, 1 reply; 10+ messages in thread
From: Catalin Marinas @ 2009-03-23 16:33 UTC (permalink / raw)
  To: Karl Hasselström; +Cc: git

2009/3/23 Karl Hasselström <kha@treskal.com>:
> On 2009-03-20 16:15:45 +0000, Catalin Marinas wrote:
>> @@ -752,12 +752,13 @@ class Index(RunWithEnv):
>>          assert current == None or isinstance(current, Tree)
>>
>>          # Take care of the really trivial cases.
>> -        if base == ours:
>> -            return (theirs, current)
>> -        if base == theirs:
>> -            return (ours, current)
>> -        if ours == theirs:
>> -            return (ours, current)
>> +        if check_trivial:
>> +            if base == ours:
>> +                return (theirs, current)
>> +            if base == theirs:
>> +                return (ours, current)
>> +            if ours == theirs:
>> +                return (ours, current)
>
> Uh, what? What's the point of not doing this unconditionally?

There are a few cases where my algorithm failed because the reverse
applying of patches fell on one of those special cases (otherwise they
wouldn't apply). The check_merged() function assumes that if a patch
can be reversed in a given tree, it was already included in that tree.

Let's assume that the tree corresponding to the top patch is T1. We
have the following cases for reverse-applying a patch which fall under
the trivial cases above (patch expressed as bottom_tree..top_tree):

The empty patch cases should be ignored from such test (not done currently):

T1..T1 => merge(T1, T1, T1) == T1
T2..T2 => merge(T2, T1, T2) == T1

The non-empty patch situations:

T1..T2 => merge(T2, T1, T1) == T1
T2..T1 => merge(T1, T1, T2) == T2

The T1..T2 is pretty common and happens when the base of a patch
wasn't modified. Reverse-applying such patch should not normally
succeed but the merge() here uses one of those special cases. The
merge() result is correct since we want two trees merged, T1 and T1,
with a common base, T2, used a helper.

The T2..T1 cases would succeed with both trivial checks and
apply_treediff() and that's probably OK since if a patch generates the
same tree when applied, the changes it makes were probably already
included.

Now I understand it better :-). Reading my explanation above, it seems
that only the T1..T2 case matters and it can be taken care of in the
check_merged() function. Checking whether the tree returned by merge()
is different than "ours" should be enough for all the above cases.

>> @@ -379,3 +385,25 @@ class StackTransaction(object):
>>          assert set(self.unapplied + self.hidden) == set(unapplied + hidden)
>>          self.unapplied = unapplied
>>          self.hidden = hidden
>> +
>> +    def check_merged(self, patches):
>> +        """Return a subset of patches already merged."""
>> +        merged = []
>> +        temp_index = self.__stack.repository.temp_index()
>> +        temp_index_tree = None
>
> There's no need to create a new temp index here. The transaction
> object already has one.

I had the impression that an Index object would hold some state and
didn't want to break it. It seems OK to use as long as I don't touch
self.temp_index_tree. See below for an updated patch:


Add the --merged option to goto

From: Catalin Marinas <catalin.marinas@gmail.com>

This patch adds support for checking which patches were already merged
upstream. This checking is done by trying to reverse-apply the patches
in the index before pushing them onto the stack.

Signed-off-by: Catalin Marinas <catalin.marinas@gmail.com>
---
 stgit/argparse.py        |    4 ++++
 stgit/commands/goto.py   |   12 +++++++++---
 stgit/lib/git.py         |    2 +-
 stgit/lib/transaction.py |   38 +++++++++++++++++++++++++++++++-------
 4 files changed, 45 insertions(+), 11 deletions(-)

diff --git a/stgit/argparse.py b/stgit/argparse.py
index 85ee6e3..765579c 100644
--- a/stgit/argparse.py
+++ b/stgit/argparse.py
@@ -225,6 +225,10 @@ def keep_option():
                 short = 'Keep the local changes',
                 default = config.get('stgit.autokeep') == 'yes')]

+def merged_option():
+    return [opt('-m', '--merged', action = 'store_true',
+                short = 'Check for patches merged upstream')]
+
 class CompgenBase(object):
     def actions(self, var): return set()
     def words(self, var): return set()
diff --git a/stgit/commands/goto.py b/stgit/commands/goto.py
index 66f49df..839b75c 100644
--- a/stgit/commands/goto.py
+++ b/stgit/commands/goto.py
@@ -28,7 +28,7 @@ Push/pop patches to/from the stack until the one
given on the command
 line becomes current."""

 args = [argparse.other_applied_patches, argparse.unapplied_patches]
-options = argparse.keep_option()
+options = argparse.keep_option() + argparse.merged_option()

 directory = common.DirectoryHasRepositoryLib()

@@ -47,8 +47,14 @@ def func(parser, options, args):
         assert not trans.pop_patches(lambda pn: pn in to_pop)
     elif patch in trans.unapplied:
         try:
-            for pn in trans.unapplied[:trans.unapplied.index(patch)+1]:
-                trans.push_patch(pn, iw, allow_interactive = True)
+            to_push = trans.unapplied[:trans.unapplied.index(patch)+1]
+            if options.merged:
+                merged = set(trans.check_merged(to_push))
+            else:
+                merged = set()
+            for pn in to_push:
+                trans.push_patch(pn, iw, allow_interactive = True,
+                                 already_merged = pn in merged)
         except transaction.TransactionHalted:
             pass
     elif patch in trans.hidden:
diff --git a/stgit/lib/git.py b/stgit/lib/git.py
index e0a3c96..fcac918 100644
--- a/stgit/lib/git.py
+++ b/stgit/lib/git.py
@@ -732,7 +732,7 @@ class Index(RunWithEnv):
         # to use --binary.
         self.apply(self.__repository.diff_tree(tree1, tree2, ['--full-index']),
                    quiet)
-    def merge(self, base, ours, theirs, current = None):
+    def merge(self, base, ours, theirs, current = None, check_trivial = True):
         """Use the index (and only the index) to do a 3-way merge of the
         L{Tree}s C{base}, C{ours} and C{theirs}. The merge will either
         succeed (in which case the first half of the return value is
diff --git a/stgit/lib/transaction.py b/stgit/lib/transaction.py
index b146648..9fa75c1 100644
--- a/stgit/lib/transaction.py
+++ b/stgit/lib/transaction.py
@@ -297,7 +297,8 @@ class StackTransaction(object):
                     out.info('Deleted %s%s' % (pn, s))
         return popped

-    def push_patch(self, pn, iw = None, allow_interactive = False):
+    def push_patch(self, pn, iw = None, allow_interactive = False,
+                   already_merged = False):
         """Attempt to push the named patch. If this results in conflicts,
         halts the transaction. If index+worktree are given, spill any
         conflicts to them."""
@@ -305,11 +306,14 @@ class StackTransaction(object):
         cd = orig_cd.set_committer(None)
         oldparent = cd.parent
         cd = cd.set_parent(self.top)
-        base = oldparent.data.tree
-        ours = cd.parent.data.tree
-        theirs = cd.tree
-        tree, self.temp_index_tree = self.temp_index.merge(
-            base, ours, theirs, self.temp_index_tree)
+        if already_merged:
+            tree = cd.tree
+        else:
+            base = oldparent.data.tree
+            ours = cd.parent.data.tree
+            theirs = cd.tree
+            tree, self.temp_index_tree = self.temp_index.merge(
+                base, ours, theirs, self.temp_index_tree)
         s = ''
         merge_conflict = False
         if not tree:
@@ -341,7 +345,9 @@ class StackTransaction(object):
         else:
             comm = None
             s = ' (unmodified)'
-        if not merge_conflict and cd.is_nochange():
+        if already_merged:
+            s = ' (merged)'
+        elif not merge_conflict and cd.is_nochange():
             s = ' (empty)'
         out.info('Pushed %s%s' % (pn, s))
         def update():
@@ -379,3 +385,21 @@ class StackTransaction(object):
         assert set(self.unapplied + self.hidden) == set(unapplied + hidden)
         self.unapplied = unapplied
         self.hidden = hidden
+
+    def check_merged(self, patches):
+        """Return a subset of patches already merged."""
+        merged = []
+        temp_index_tree = None
+        ours = self.stack.head.data.tree
+        for pn in reversed(patches):
+            # check whether patch changes can be reversed in the current tree
+            cd = self.patches[pn].data
+            base = cd.tree
+            theirs = cd.parent.data.tree
+            tree, temp_index_tree = \
+                    self.temp_index.merge(base, ours, theirs, temp_index_tree,
+                                          check_trivial = False)
+            if tree and tree != ours:
+                merged.append(pn)
+                ours = tree
+        return merged


-- 
Catalin

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [StGit PATCH] Add the --merged option to goto
  2009-03-23 16:33   ` Catalin Marinas
@ 2009-03-24 13:16     ` Karl Hasselström
  2009-03-24 15:40       ` Catalin Marinas
  0 siblings, 1 reply; 10+ messages in thread
From: Karl Hasselström @ 2009-03-24 13:16 UTC (permalink / raw)
  To: Catalin Marinas; +Cc: git

On 2009-03-23 16:33:04 +0000, Catalin Marinas wrote:

> 2009/3/23 Karl Hasselström <kha@treskal.com>:
>
> > On 2009-03-20 16:15:45 +0000, Catalin Marinas wrote:
> >> @@ -752,12 +752,13 @@ class Index(RunWithEnv):
> >>          assert current == None or isinstance(current, Tree)
> >>
> >>          # Take care of the really trivial cases.
> >> -        if base == ours:
> >> -            return (theirs, current)
> >> -        if base == theirs:
> >> -            return (ours, current)
> >> -        if ours == theirs:
> >> -            return (ours, current)
> >> +        if check_trivial:
> >> +            if base == ours:
> >> +                return (theirs, current)
> >> +            if base == theirs:
> >> +                return (ours, current)
> >> +            if ours == theirs:
> >> +                return (ours, current)
> >
> > Uh, what? What's the point of not doing this unconditionally?
>
> There are a few cases where my algorithm failed because the reverse
> applying of patches fell on one of those special cases (otherwise
> they wouldn't apply). The check_merged() function assumes that if a
> patch can be reversed in a given tree, it was already included in
> that tree.
>
> Let's assume that the tree corresponding to the top patch is T1. We
> have the following cases for reverse-applying a patch which fall
> under the trivial cases above (patch expressed as
> bottom_tree..top_tree):
>
> The empty patch cases should be ignored from such test (not done
> currently):
>
> T1..T1 => merge(T1, T1, T1) == T1
> T2..T2 => merge(T2, T1, T2) == T1
>
> The non-empty patch situations:
>
> T1..T2 => merge(T2, T1, T1) == T1
> T2..T1 => merge(T1, T1, T2) == T2
>
> The T1..T2 is pretty common and happens when the base of a patch
> wasn't modified. Reverse-applying such patch should not normally
> succeed but the merge() here uses one of those special cases. The
> merge() result is correct since we want two trees merged, T1 and T1,
> with a common base, T2, used a helper.
>
> The T2..T1 cases would succeed with both trivial checks and
> apply_treediff() and that's probably OK since if a patch generates
> the same tree when applied, the changes it makes were probably
> already included.
>
> Now I understand it better :-). Reading my explanation above, it
> seems that only the T1..T2 case matters and it can be taken care of
> in the check_merged() function. Checking whether the tree returned
> by merge() is different than "ours" should be enough for all the
> above cases.

Hmm. If the tip of the branch is T1, and we reverse-apply the patch
T1..T2, we get the merge (base T2, ours T1, theirs T1) ... yeah, I see
what you mean. The problem isn't that we give T1 as the result of this
merge -- that's actually the right thing to do -- the problem is that
you don't actually want a merge. What you want is patch application.
Maybe the apply_treediff method would do? See my other comment below.

> >> @@ -379,3 +385,25 @@ class StackTransaction(object):
> >>          assert set(self.unapplied + self.hidden) == set(unapplied + hidden)
> >>          self.unapplied = unapplied
> >>          self.hidden = hidden
> >> +
> >> +    def check_merged(self, patches):
> >> +        """Return a subset of patches already merged."""
> >> +        merged = []
> >> +        temp_index = self.__stack.repository.temp_index()
> >> +        temp_index_tree = None
> >
> > There's no need to create a new temp index here. The transaction
> > object already has one.
>
> I had the impression that an Index object would hold some state and
> didn't want to break it. It seems OK to use as long as I don't touch
> self.temp_index_tree. See below for an updated patch:

Yes, an Index object owns a git index file.

And no, not quite. temp_index_tree is set to the tree we know is
stored in temp_index right now (or None if we don't know). The idea is
that we'll often want to read a tree into the index that's already
there, and by keeping track of this we'll get better performance.
(This works very well in practice.) Apologies if there aren't comments
explaining this ... the merge method has some docs on the subject.

I think what you should do is something like what merge() does:

  if temp_index_tree != branch_tip:
      temp_index.read_tree(branch_tip)
      temp_index_tree = branch_tip
  try:
      temp_index.apply_treediff(patch_bottom, patch_top, quiet = True)
      temp_index_tree = temp_index.write_tree()
      return True
  except MergeException:
      return False

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [StGit PATCH] Add the --merged option to goto
  2009-03-24 13:16     ` Karl Hasselström
@ 2009-03-24 15:40       ` Catalin Marinas
  2009-03-25  9:05         ` Karl Hasselström
  0 siblings, 1 reply; 10+ messages in thread
From: Catalin Marinas @ 2009-03-24 15:40 UTC (permalink / raw)
  To: Karl Hasselström; +Cc: git

2009/3/24 Karl Hasselström <kha@treskal.com>:
> On 2009-03-23 16:33:04 +0000, Catalin Marinas wrote:
>> Now I understand it better :-). Reading my explanation above, it
>> seems that only the T1..T2 case matters and it can be taken care of
>> in the check_merged() function. Checking whether the tree returned
>> by merge() is different than "ours" should be enough for all the
>> above cases.
>
> Hmm. If the tip of the branch is T1, and we reverse-apply the patch
> T1..T2, we get the merge (base T2, ours T1, theirs T1) ... yeah, I see
> what you mean. The problem isn't that we give T1 as the result of this
> merge -- that's actually the right thing to do -- the problem is that
> you don't actually want a merge. What you want is patch application.
> Maybe the apply_treediff method would do?

Yes, see below for an updated patch.

>> I had the impression that an Index object would hold some state and
>> didn't want to break it. It seems OK to use as long as I don't touch
>> self.temp_index_tree. See below for an updated patch:
>
> Yes, an Index object owns a git index file.
>
> And no, not quite. temp_index_tree is set to the tree we know is
> stored in temp_index right now (or None if we don't know). The idea is
> that we'll often want to read a tree into the index that's already
> there, and by keeping track of this we'll get better performance.
> (This works very well in practice.) Apologies if there aren't comments
> explaining this ... the merge method has some docs on the subject.

I figured this out eventually. Anyway, with apply_treedif() there is
no need for the temp_index_tree. In the updated patch below, I don't
even call write_tree() as it isn't needed (to my understanding).
Whatever is in the index after the check_merged() function doesn't
correspond to any tree so it would need to be re-read.

BTW, I implemented push and pop and it now passes all the "push
--merged" tests. I had to correct the "already_merged" case in
Transaction.push_patch() to generate an empty patch.

    Add the --merged option to goto

    This patch adds support for checking which patches were already merged
    upstream. This checking is done by trying to reverse-apply the patches
    in the index before pushing them onto the stack.

    Signed-off-by: Catalin Marinas <catalin.marinas@gmail.com>

diff --git a/stgit/argparse.py b/stgit/argparse.py
index 85ee6e3..765579c 100644
--- a/stgit/argparse.py
+++ b/stgit/argparse.py
@@ -225,6 +225,10 @@ def keep_option():
                 short = 'Keep the local changes',
                 default = config.get('stgit.autokeep') == 'yes')]

+def merged_option():
+    return [opt('-m', '--merged', action = 'store_true',
+                short = 'Check for patches merged upstream')]
+
 class CompgenBase(object):
     def actions(self, var): return set()
     def words(self, var): return set()
diff --git a/stgit/commands/goto.py b/stgit/commands/goto.py
index 66f49df..839b75c 100644
--- a/stgit/commands/goto.py
+++ b/stgit/commands/goto.py
@@ -28,7 +28,7 @@ Push/pop patches to/from the stack until the one
given on the command
 line becomes current."""

 args = [argparse.other_applied_patches, argparse.unapplied_patches]
-options = argparse.keep_option()
+options = argparse.keep_option() + argparse.merged_option()

 directory = common.DirectoryHasRepositoryLib()

@@ -47,8 +47,14 @@ def func(parser, options, args):
         assert not trans.pop_patches(lambda pn: pn in to_pop)
     elif patch in trans.unapplied:
         try:
-            for pn in trans.unapplied[:trans.unapplied.index(patch)+1]:
-                trans.push_patch(pn, iw, allow_interactive = True)
+            to_push = trans.unapplied[:trans.unapplied.index(patch)+1]
+            if options.merged:
+                merged = set(trans.check_merged(to_push))
+            else:
+                merged = set()
+            for pn in to_push:
+                trans.push_patch(pn, iw, allow_interactive = True,
+                                 already_merged = pn in merged)
         except transaction.TransactionHalted:
             pass
     elif patch in trans.hidden:
diff --git a/stgit/lib/transaction.py b/stgit/lib/transaction.py
index b146648..e2d9b78 100644
--- a/stgit/lib/transaction.py
+++ b/stgit/lib/transaction.py
@@ -297,7 +297,8 @@ class StackTransaction(object):
                     out.info('Deleted %s%s' % (pn, s))
         return popped

-    def push_patch(self, pn, iw = None, allow_interactive = False):
+    def push_patch(self, pn, iw = None, allow_interactive = False,
+                   already_merged = False):
         """Attempt to push the named patch. If this results in conflicts,
         halts the transaction. If index+worktree are given, spill any
         conflicts to them."""
@@ -305,11 +306,15 @@ class StackTransaction(object):
         cd = orig_cd.set_committer(None)
         oldparent = cd.parent
         cd = cd.set_parent(self.top)
-        base = oldparent.data.tree
-        ours = cd.parent.data.tree
-        theirs = cd.tree
-        tree, self.temp_index_tree = self.temp_index.merge(
-            base, ours, theirs, self.temp_index_tree)
+        if already_merged:
+            # the resulting patch is empty
+            tree = cd.parent.data.tree
+        else:
+            base = oldparent.data.tree
+            ours = cd.parent.data.tree
+            theirs = cd.tree
+            tree, self.temp_index_tree = self.temp_index.merge(
+                base, ours, theirs, self.temp_index_tree)
         s = ''
         merge_conflict = False
         if not tree:
@@ -341,7 +346,9 @@ class StackTransaction(object):
         else:
             comm = None
             s = ' (unmodified)'
-        if not merge_conflict and cd.is_nochange():
+        if already_merged:
+            s = ' (merged)'
+        elif not merge_conflict and cd.is_nochange():
             s = ' (empty)'
         out.info('Pushed %s%s' % (pn, s))
         def update():
@@ -379,3 +386,23 @@ class StackTransaction(object):
         assert set(self.unapplied + self.hidden) == set(unapplied + hidden)
         self.unapplied = unapplied
         self.hidden = hidden
+
+    def check_merged(self, patches):
+        """Return a subset of patches already merged."""
+        merged = []
+        self.temp_index.read_tree(self.stack.head.data.tree)
+        # The self.temp_index is modified by apply_treediff() so force
+        # read_tree() the next time merge() is used.
+        self.temp_index_tree = None
+        for pn in reversed(patches):
+            # check whether patch changes can be reversed in the current index
+            cd = self.patches[pn].data
+            if cd.is_nochange():
+                continue
+            try:
+                self.temp_index.apply_treediff(cd.tree, cd.parent.data.tree,
+                                               quiet = True)
+                merged.append(pn)
+            except git.MergeException:
+                pass
+        return merged



Thanks.

-- 
Catalin

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [StGit PATCH] Add the --merged option to goto
  2009-03-24 15:40       ` Catalin Marinas
@ 2009-03-25  9:05         ` Karl Hasselström
  2009-03-25 10:24           ` Catalin Marinas
  0 siblings, 1 reply; 10+ messages in thread
From: Karl Hasselström @ 2009-03-25  9:05 UTC (permalink / raw)
  To: Catalin Marinas; +Cc: git

On 2009-03-24 15:40:10 +0000, Catalin Marinas wrote:

> Anyway, with apply_treedif() there is no need for the
> temp_index_tree. In the updated patch below, I don't even call
> write_tree() as it isn't needed (to my understanding).

Yes. You're not interested in the result of the patch application, you
just want to know if it succeeded or not.

> Whatever is in the index after the check_merged() function doesn't
> correspond to any tree so it would need to be re-read.

It does correspond to _some_ tree, but as we have no particular reason
to expect that it might be a tree that we're going to need again
immediately, we can set temp_index_tree to None to force re-reading,
rather than pay the cost of write_tree.

When pushing patches, on the other hand, we have good reasons to
believe that the next tree we'll need in the index is the same (or at
least very close to) the one produced in the last merge. Consider the
case of popping a few patches, changing the message on the top patch,
and then pushing the patches back, for example.

> +    def check_merged(self, patches):
> +        """Return a subset of patches already merged."""
> +        merged = []
> +        self.temp_index.read_tree(self.stack.head.data.tree)
> +        # The self.temp_index is modified by apply_treediff() so force
> +        # read_tree() the next time merge() is used.
> +        self.temp_index_tree = None
> +        for pn in reversed(patches):
> +            # check whether patch changes can be reversed in the current index
> +            cd = self.patches[pn].data
> +            if cd.is_nochange():
> +                continue
> +            try:
> +                self.temp_index.apply_treediff(cd.tree, cd.parent.data.tree,
> +                                               quiet = True)
> +                merged.append(pn)
> +            except git.MergeException:
> +                pass
> +        return merged

Some points here:

  1. Check if temp_index_tree already contains the tree you want
     instead of doing read_tree unconditionally. That is, change

       self.temp_index.read_tree(self.stack.head.data.tree)

     to

       if self.stack.head.data.tree != self.temp_index_tree:
           self.temp_index.read_tree(self.stack.head.data.tree)
           self.temp_index_tree = self.stack.head.data.tree

  2. If apply_treediff fails (raises MergeException), the index wasn't
     modified at all (but I guess you knew that, since your code
     relies on that fact), so there's no reason to set temp_index_tree
     to None in that case. So in order to not clear temp_index_tree
     unnecessarily in case none of the patches reverse-apply (a case I
     personally encounter frequently, since I leave the -m flag on all
     the time), move

       self.temp_index_tree = None

     to just after (or just before) "merged.append(pn)".

  3. Why are empty patches considered not merged?

Other than that, this logic looks fine.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [StGit PATCH] Add the --merged option to goto
  2009-03-25  9:05         ` Karl Hasselström
@ 2009-03-25 10:24           ` Catalin Marinas
  2009-03-26 11:15             ` Karl Hasselström
  0 siblings, 1 reply; 10+ messages in thread
From: Catalin Marinas @ 2009-03-25 10:24 UTC (permalink / raw)
  To: Karl Hasselström; +Cc: git

2009/3/25 Karl Hasselström <kha@treskal.com>:
> On 2009-03-24 15:40:10 +0000, Catalin Marinas wrote:
>> Whatever is in the index after the check_merged() function doesn't
>> correspond to any tree so it would need to be re-read.
>
> It does correspond to _some_ tree, but as we have no particular reason
> to expect that it might be a tree that we're going to need again
> immediately, we can set temp_index_tree to None to force re-reading,
> rather than pay the cost of write_tree.

Yes, that's what I meant by "doesn't correspond to any tree".

BTW, why don't we keep the tree information directly in the Index
object? Since this object is modified only via its own interface, it
can do all the checks and avoid the managing of temp_index_tree in the
Transaction object.

>> +    def check_merged(self, patches):
>> +        """Return a subset of patches already merged."""
>> +        merged = []
>> +        self.temp_index.read_tree(self.stack.head.data.tree)
>> +        # The self.temp_index is modified by apply_treediff() so force
>> +        # read_tree() the next time merge() is used.
>> +        self.temp_index_tree = None
>> +        for pn in reversed(patches):
>> +            # check whether patch changes can be reversed in the current index
>> +            cd = self.patches[pn].data
>> +            if cd.is_nochange():
>> +                continue
>> +            try:
>> +                self.temp_index.apply_treediff(cd.tree, cd.parent.data.tree,
>> +                                               quiet = True)
>> +                merged.append(pn)
>> +            except git.MergeException:
>> +                pass
>> +        return merged
>
> Some points here:
>
>  1. Check if temp_index_tree already contains the tree you want
>     instead of doing read_tree unconditionally. That is, change
>
>       self.temp_index.read_tree(self.stack.head.data.tree)
>
>     to
>
>       if self.stack.head.data.tree != self.temp_index_tree:
>           self.temp_index.read_tree(self.stack.head.data.tree)
>           self.temp_index_tree = self.stack.head.data.tree
>
>  2. If apply_treediff fails (raises MergeException), the index wasn't
>     modified at all (but I guess you knew that, since your code
>     relies on that fact), so there's no reason to set temp_index_tree
>     to None in that case. So in order to not clear temp_index_tree
>     unnecessarily in case none of the patches reverse-apply (a case I
>     personally encounter frequently, since I leave the -m flag on all
>     the time), move
>
>       self.temp_index_tree = None
>
>     to just after (or just before) "merged.append(pn)".

Yes. But it may be even better to do this in Index.
Index.apply_treediff() would set the tree to None and read_tree or
write_tree would set it to the corresponding tree.

>  3. Why are empty patches considered not merged?

They would be reported as empty anyway and in general you don't submit
empty patches for upstream merging.

-- 
Catalin

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [StGit PATCH] Add the --merged option to goto
  2009-03-25 10:24           ` Catalin Marinas
@ 2009-03-26 11:15             ` Karl Hasselström
  2009-03-30 16:01               ` Catalin Marinas
  0 siblings, 1 reply; 10+ messages in thread
From: Karl Hasselström @ 2009-03-26 11:15 UTC (permalink / raw)
  To: Catalin Marinas; +Cc: git

On 2009-03-25 10:24:13 +0000, Catalin Marinas wrote:

> BTW, why don't we keep the tree information directly in the Index
> object? Since this object is modified only via its own interface, it
> can do all the checks and avoid the managing of temp_index_tree in
> the Transaction object.

I guess that might be a good idea -- it should be doable without any
extra overhead for users that don't want it.

> Yes. But it may be even better to do this in Index.
> Index.apply_treediff() would set the tree to None and read_tree or
> write_tree would set it to the corresponding tree.

We'd have to cover the other index operations too. But yes, this is
probably a good idea.

> >  3. Why are empty patches considered not merged?
>
> They would be reported as empty anyway and in general you don't
> submit empty patches for upstream merging.

Ah, duh. I was forgetting what the "merged" detection was for in the
first place.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [StGit PATCH] Add the --merged option to goto
  2009-03-26 11:15             ` Karl Hasselström
@ 2009-03-30 16:01               ` Catalin Marinas
  2009-03-31  7:27                 ` Karl Hasselström
  0 siblings, 1 reply; 10+ messages in thread
From: Catalin Marinas @ 2009-03-30 16:01 UTC (permalink / raw)
  To: Karl Hasselström; +Cc: git

2009/3/26 Karl Hasselström <kha@treskal.com>:
> On 2009-03-25 10:24:13 +0000, Catalin Marinas wrote:
>
>> BTW, why don't we keep the tree information directly in the Index
>> object? Since this object is modified only via its own interface, it
>> can do all the checks and avoid the managing of temp_index_tree in
>> the Transaction object.
>
> I guess that might be a good idea -- it should be doable without any
> extra overhead for users that don't want it.

I tried but gave up quickly. The IndexAndWorktree class also dirties
the Index with the merge operations, so it is not worth the hassle.

That's the updated patch:

    Add the --merged option to goto

    This patch adds support for checking which patches were already merged
    upstream. This checking is done by trying to reverse-apply the patches
    in the index before pushing them onto the stack.

    Signed-off-by: Catalin Marinas <catalin.marinas@gmail.com>

diff --git a/stgit/argparse.py b/stgit/argparse.py
index 85ee6e3..765579c 100644
--- a/stgit/argparse.py
+++ b/stgit/argparse.py
@@ -225,6 +225,10 @@ def keep_option():
                 short = 'Keep the local changes',
                 default = config.get('stgit.autokeep') == 'yes')]

+def merged_option():
+    return [opt('-m', '--merged', action = 'store_true',
+                short = 'Check for patches merged upstream')]
+
 class CompgenBase(object):
     def actions(self, var): return set()
     def words(self, var): return set()
diff --git a/stgit/commands/goto.py b/stgit/commands/goto.py
index 66f49df..839b75c 100644
--- a/stgit/commands/goto.py
+++ b/stgit/commands/goto.py
@@ -28,7 +28,7 @@ Push/pop patches to/from the stack until the one
given on the command
 line becomes current."""

 args = [argparse.other_applied_patches, argparse.unapplied_patches]
-options = argparse.keep_option()
+options = argparse.keep_option() + argparse.merged_option()

 directory = common.DirectoryHasRepositoryLib()

@@ -47,8 +47,14 @@ def func(parser, options, args):
         assert not trans.pop_patches(lambda pn: pn in to_pop)
     elif patch in trans.unapplied:
         try:
-            for pn in trans.unapplied[:trans.unapplied.index(patch)+1]:
-                trans.push_patch(pn, iw, allow_interactive = True)
+            to_push = trans.unapplied[:trans.unapplied.index(patch)+1]
+            if options.merged:
+                merged = set(trans.check_merged(to_push))
+            else:
+                merged = set()
+            for pn in to_push:
+                trans.push_patch(pn, iw, allow_interactive = True,
+                                 already_merged = pn in merged)
         except transaction.TransactionHalted:
             pass
     elif patch in trans.hidden:
diff --git a/stgit/lib/transaction.py b/stgit/lib/transaction.py
index b146648..4148ff3 100644
--- a/stgit/lib/transaction.py
+++ b/stgit/lib/transaction.py
@@ -297,7 +297,8 @@ class StackTransaction(object):
                     out.info('Deleted %s%s' % (pn, s))
         return popped

-    def push_patch(self, pn, iw = None, allow_interactive = False):
+    def push_patch(self, pn, iw = None, allow_interactive = False,
+                   already_merged = False):
         """Attempt to push the named patch. If this results in conflicts,
         halts the transaction. If index+worktree are given, spill any
         conflicts to them."""
@@ -305,11 +306,15 @@ class StackTransaction(object):
         cd = orig_cd.set_committer(None)
         oldparent = cd.parent
         cd = cd.set_parent(self.top)
-        base = oldparent.data.tree
-        ours = cd.parent.data.tree
-        theirs = cd.tree
-        tree, self.temp_index_tree = self.temp_index.merge(
-            base, ours, theirs, self.temp_index_tree)
+        if already_merged:
+            # the resulting patch is empty
+            tree = cd.parent.data.tree
+        else:
+            base = oldparent.data.tree
+            ours = cd.parent.data.tree
+            theirs = cd.tree
+            tree, self.temp_index_tree = self.temp_index.merge(
+                base, ours, theirs, self.temp_index_tree)
         s = ''
         merge_conflict = False
         if not tree:
@@ -341,7 +346,9 @@ class StackTransaction(object):
         else:
             comm = None
             s = ' (unmodified)'
-        if not merge_conflict and cd.is_nochange():
+        if already_merged:
+            s = ' (merged)'
+        elif not merge_conflict and cd.is_nochange():
             s = ' (empty)'
         out.info('Pushed %s%s' % (pn, s))
         def update():
@@ -379,3 +386,25 @@ class StackTransaction(object):
         assert set(self.unapplied + self.hidden) == set(unapplied + hidden)
         self.unapplied = unapplied
         self.hidden = hidden
+
+    def check_merged(self, patches):
+        """Return a subset of patches already merged."""
+        merged = []
+        if self.temp_index_tree != self.stack.head.data.tree:
+            self.temp_index.read_tree(self.stack.head.data.tree)
+            self.temp_index_tree = self.stack.head.data.tree
+        for pn in reversed(patches):
+            # check whether patch changes can be reversed in the current index
+            cd = self.patches[pn].data
+            if cd.is_nochange():
+                continue
+            try:
+                self.temp_index.apply_treediff(cd.tree, cd.parent.data.tree,
+                                               quiet = True)
+                merged.append(pn)
+                # The self.temp_index was modified by apply_treediff() so
+                # force read_tree() the next time merge() is used.
+                self.temp_index_tree = None
+            except git.MergeException:
+                pass
+        return merged


-- 
Catalin

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [StGit PATCH] Add the --merged option to goto
  2009-03-30 16:01               ` Catalin Marinas
@ 2009-03-31  7:27                 ` Karl Hasselström
  0 siblings, 0 replies; 10+ messages in thread
From: Karl Hasselström @ 2009-03-31  7:27 UTC (permalink / raw)
  To: Catalin Marinas; +Cc: git

On 2009-03-30 17:01:12 +0100, Catalin Marinas wrote:

> 2009/3/26 Karl Hasselström <kha@treskal.com>:
>
> > On 2009-03-25 10:24:13 +0000, Catalin Marinas wrote:
> >
> > > BTW, why don't we keep the tree information directly in the
> > > Index object? Since this object is modified only via its own
> > > interface, it can do all the checks and avoid the managing of
> > > temp_index_tree in the Transaction object.
> >
> > I guess that might be a good idea -- it should be doable without
> > any extra overhead for users that don't want it.
>
> I tried but gave up quickly. The IndexAndWorktree class also dirties
> the Index with the merge operations, so it is not worth the hassle.

OK. (Though you should be able to set the tree to None for those
cases, since the meaning of None is simply that we don't promise
anything about what tree is currently in the index.)

And since I've run out of even remotely plausible things to complain
about,

Acked-by: Karl Hasselström <kha@treskal.com>

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2009-03-31  7:28 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-03-20 16:15 [StGit PATCH] Add the --merged option to goto Catalin Marinas
2009-03-23  8:45 ` Karl Hasselström
2009-03-23 16:33   ` Catalin Marinas
2009-03-24 13:16     ` Karl Hasselström
2009-03-24 15:40       ` Catalin Marinas
2009-03-25  9:05         ` Karl Hasselström
2009-03-25 10:24           ` Catalin Marinas
2009-03-26 11:15             ` Karl Hasselström
2009-03-30 16:01               ` Catalin Marinas
2009-03-31  7:27                 ` Karl Hasselström

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).