any good diff merging utility?

any good diff merging utility?

[Ivan Vadovic]

Hi,

I like to build kernels with a bunch of patches on top to test new stuff. The
problem is that it takes a lot of effort to fix all the failed hunks during
patching that really wouldn't have to be failed if only patch was a little more
inteligent and could merge several patches into one ( if possible) or if could
take into account already applied patches.

Well, are there any utilities to merge diffs? I couldn't find any on freshmeat.
So what are you using to stack many patches onto the kernel tree? Just manualy
modify the diff? I'll try to write something more automatic if nothing comes up.

Ivan Vadovic

Re: any good diff merging utility?

[Riley Williams]

Hi Ivan.

 > I like to build kernels with a bunch of patches on top to test
 > new stuff. The problem is that it takes a lot of effort to fix
 > all the failed hunks during patching that really wouldn't have
 > to be failed if only patch was a little more inteligent and
 > could merge several patches into one ( if possible) or if could
 > take into account already applied patches.

The basic problem here is that the "failed hunks" are usually there
because of conflicts between the two patches in question, and as a
result, they are not as easy to merge automagically as one might at
first assume.

 > Well, are there any utilities to merge diffs? I couldn't find
 > any on freshmeat. So what are you using to stack many patches
 > onto the kernel tree? Just manualy modify the diff? I'll try to
 > write something more automatic if nothing comes up.

I once came across a utility called "diff3" that was designed to take
a patch for one version of a package and create an equivalent patch
for another version of the same package, but I haven't been able to
find it again since my hard drive crashed.

Best wishes from Riley.

Re: any good diff merging utility?

[Ivan Vadovic]

>  > I like to build kernels with a bunch of patches on top to test
>  > new stuff. The problem is that it takes a lot of effort to fix
>  > all the failed hunks during patching that really wouldn't have
>  > to be failed if only patch was a little more inteligent and
>  > could merge several patches into one ( if possible) or if could
>  > take into account already applied patches.
> 
> The basic problem here is that the "failed hunks" are usually there
> because of conflicts between the two patches in question, and as a
> result, they are not as easy to merge automagically as one might at
> first assume.

Very often the case is that they indeed can be merged automagically.
For example two patches inserting few lines right after the #include
lines.

patch1:
@@ 10,1 10,2 @@
 #include <foo.h>
+#include <1.h>

patch2:
@@ 10,1 10,2 @@
 #include <foo.h>
+#include <2.h>

The patch will fail to patch :-). But there is no real conflict between
the patches.
 
>  > Well, are there any utilities to merge diffs? I couldn't find
>  > any on freshmeat. So what are you using to stack many patches
>  > onto the kernel tree? Just manualy modify the diff? I'll try to
>  > write something more automatic if nothing comes up.
> 
> I once came across a utility called "diff3" that was designed to take
> a patch for one version of a package and create an equivalent patch
> for another version of the same package, but I haven't been able to
> find it again since my hard drive crashed.

diff3 comes from gnu diffutils
<ftp://ftp.gnu.org/gnu/diffutils/diffutils-2.7.tar.gz>. But all it does
is comparing three FILES for differencies.

Ivan Vadovic

Re: any good diff merging utility?

[Jonathan Lundell]

At 2:34 AM +0200 2001-06-18, Ivan Vadovic wrote:
>Very often the case is that they indeed can be merged automagically.
>For example two patches inserting few lines right after the #include
>lines.
>
>patch1:
>@@ 10,1 10,2 @@
>  #include <foo.h>
>+#include <1.h>
>
>patch2:
>@@ 10,1 10,2 @@
>  #include <foo.h>
>+#include <2.h>
>
>The patch will fail to patch :-). But there is no real conflict between
>the patches.

Problem is, you can't tell automatically. Even if the diffs don't 
conflict physically, it's entirely possible that they conflict 
logically.
-- 
/Jonathan Lundell.

Re: any good diff merging utility?

[Keith Owens]

On Mon, 18 Jun 2001 01:45:47 +0200, 
Ivan Vadovic <pivo@pobox.sk> wrote:
>Well, are there any utilities to merge diffs? I couldn't find any on freshmeat.
>So what are you using to stack many patches onto the kernel tree? Just manualy
>modify the diff? I'll try to write something more automatic if nothing comes up.

Use any source repository tool that understands both change sets and
multiple branches.  Change sets treat all related patches as one
change, unlike CVS which treats each change to a file separately.  A
repository that recognises multiple branches and has decent support for
merging between branches makes life so much easier.  There are several
candidates, pick one that you feel comfortable with.

I use PRCS (http://www.xcf.berkeley.edu/~jmacd/prcs.html) to control my
local kernel source changes.  My linux-2.4 repository is 200Mb, that
contains all of Linus's patch sets (including -pre patches), all of
Alan Cox's patch sets, all my work on kdb (both i386 and ia64), XFS
changes, my 2.5 Makefile rewrite plus various other patches I have
worked on, both i386 and ia64, for a total of 307 patch sets over 5
major branches and lots of minor branches.  Not bad when a single 2.4
source tree is 125Mb.

PRCS merge between two branches identifies unchanged files, files
changed in branch 1 but not branch 2, files changed in branch 2 but not
branch 1, deleted files in either branch and files changed in both
branches.  Normally only the last category is a problem, with two sets
of patches to the same file.  PRCS uses diff3 to work out if the
patches overlap or not, if they do not overlap then there is usually no
problem, if they do overlap then PRCS creates a merged file with both
sets of patches and wraps conflict markers around them.  It is then up
to you to manually correct the conflict.

Using PRCS, I can upgrade to a new Linus or AC patch, merge the
previous kdb patch for that major branch and check if I need to issue a
new kdb patch.  All within 15 minutes.

There is also http://cvs.bofh.asn.au/mergetrees/ which does not use a
repository.

Re: any good diff merging utility?

[Paul Mackerras]

Ivan Vadovic writes:

> Well, are there any utilities to merge diffs? I couldn't find any on freshmeat.
> So what are you using to stack many patches onto the kernel tree? Just manualy
> modify the diff? I'll try to write something more automatic if nothing comes up.

Try dirdiff - ftp://ftp.samba.org/pub/paulus/dirdiff-1.2.tar.gz.  I
use it all the time for merging in changes between Linus' official
tree, my own development tree, and the PPC/Linux bitkeeper trees.

Dirdiff is a tcl/tk-based utility for graphically displaying the
difference between directory trees.  It can handle from 2 to 5 trees.
It displays a main window where it shows which files are different.
You can select a file and get it to show the diffs between that file
in any two of the directory trees.  This comes up in another window
in a format like a unified diff but with the background of the line
colored according to which file it comes from.  You can also copy
files between trees with a menu item - in fact you can select whole
groups of files to be copied.  And you can use it to generate patches
too. :)

Once you have the differences between two versions of a file
displayed, you can do a merge between the two versions.  Each line of
differences has a little check box beside it.  If you check the box it
means you want to make that change (right-click or shift-click selects
a whole group of boxes).  When you have checked all the boxes you want
you select an item from the merge menu to say which tree you want to
update.  The new version of the file comes up in an edit window and
you can check it, make any further changes you want, etc.  Then you
can either save the result or close the window (discarding the merge).

It's hard to explain in words everything about how it works and how
you use it.  It isn't really a utility to merge diffs but it is very
useful in tracking and merging changes between several large source
trees.  I find it particularly useful because I am usually interested
only in a subset of the files (i.e. particularly arch/ppc and
include/asm-ppc).  So when Linus releases a new pre-patch, I update my
"official Linus source" tree and do another dirdiff.  If there are
changes to files under fs/ for instance, I just select all of them and
copy them over to my tree without looking at the diffs.  If there are
changes in arch/i386 for instance, I look at the diff to see if I am
going to need to make a similar change in arch/ppc.

Regards,
Paul.

Re: any good diff merging utility?

[Geert Uytterhoeven]

On Mon, 18 Jun 2001, Ivan Vadovic wrote:
> >  > I like to build kernels with a bunch of patches on top to test
> >  > new stuff. The problem is that it takes a lot of effort to fix
> >  > all the failed hunks during patching that really wouldn't have
> >  > to be failed if only patch was a little more inteligent and
> >  > could merge several patches into one ( if possible) or if could
> >  > take into account already applied patches.
> > 
> > The basic problem here is that the "failed hunks" are usually there
> > because of conflicts between the two patches in question, and as a
> > result, they are not as easy to merge automagically as one might at
> > first assume.
> 
> Very often the case is that they indeed can be merged automagically.
> For example two patches inserting few lines right after the #include
> lines.
> 
> patch1:
> @@ 10,1 10,2 @@
>  #include <foo.h>
> +#include <1.h>
> 
> patch2:
> @@ 10,1 10,2 @@
>  #include <foo.h>
> +#include <2.h>
> 
> The patch will fail to patch :-). But there is no real conflict between
> the patches.
>  
> >  > Well, are there any utilities to merge diffs? I couldn't find
> >  > any on freshmeat. So what are you using to stack many patches
> >  > onto the kernel tree? Just manualy modify the diff? I'll try to
> >  > write something more automatic if nothing comes up.
> > 
> > I once came across a utility called "diff3" that was designed to take
> > a patch for one version of a package and create an equivalent patch
> > for another version of the same package, but I haven't been able to
> > find it again since my hard drive crashed.
> 
> diff3 comes from gnu diffutils
> <ftp://ftp.gnu.org/gnu/diffutils/diffutils-2.7.tar.gz>. But all it does
> is comparing three FILES for differencies.

Diffutils also contains `merge' which is more or less what you want: it merges
two files and a common ancestor into one file. So you can merge two diffs as
well, by merging the following three files:

  - original + diff1 applied
  - original
  - original + diff2 applied

If a conflict happens, merge will warn you and indicate it with special signs
in the merged file.

Merge is only for one file with different versions. I wrote a Perl script
`mergetree' that applies merge recursively to merge two directory trees and a
common ancestor. It also has an option to create hard links if the merge result
is identical to one of the merge input files. This saves diskspace if you use
cp -rl and patch and the `same' utility to have multiple revisions of the same
tree on one disk. If you want the script, mail me in private.

Revision control systems do the same, and many more things. But `mergetree'
proved to be very useful for me to keep my Linux/m68k in sync with Linus'/Alans
tree.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: any good diff merging utility?

[Riley Williams]

Hi Ivan.

 >>> I like to build kernels with a bunch of patches on top to test
 >>> new stuff. The problem is that it takes a lot of effort to fix
 >>> all the failed hunks during patching that really wouldn't have
 >>> to be failed if only patch was a little more inteligent and
 >>> could merge several patches into one ( if possible) or if could
 >>> take into account already applied patches.

 >> The basic problem here is that the "failed hunks" are usually there
 >> because of conflicts between the two patches in question, and as a
 >> result, they are not as easy to merge automagically as one might at
 >> first assume.

 > Very often the case is that they indeed can be merged automagically.
 > For example two patches inserting few lines right after the #include
 > lines.

 > patch1:
 > @@ 10,1 10,2 @@
 >  #include <foo.h>
 > +#include <1.h>

 > patch2:
 > @@ 10,1 10,2 @@
 >  #include <foo.h>
 > +#include <2.h>

 > The patch will fail to patch :-). But there is no real conflict
 > between the patches.

True, but how about the following (from one I had to merge recently):

 Q> patch1:
 Q> @@ 137,3 142,5
 Q>  for( ptr=head; *ptr; ptr=ptr->next ) {
 Q> +   ctr += ptr->qty;
 Q>     table[ctr] += ptr->qty;
 Q> +   total += table[ctr];
 Q>  }

 Q> patch2:
 Q> @@ 137,2 146,3
 Q>  for( ptr=head; *ptr; ptr=ptr->next ) {
 Q> +   total += table[ctr];
 Q>     table[ctr] += ptr->qty;

How would you merge those two patches?

 >>> Well, are there any utilities to merge diffs? I couldn't find
 >>> any on freshmeat. So what are you using to stack many patches
 >>> onto the kernel tree? Just manualy modify the diff? I'll try to
 >>> write something more automatic if nothing comes up.

 >> I once came across a utility called "diff3" that was designed to
 >> take a patch for one version of a package and create an
 >> equivalent patch for another version of the same package, but I
 >> haven't been able to find it again since my hard drive crashed.

 > diff3 comes from gnu diffutils
 > <ftp://ftp.gnu.org/gnu/diffutils/diffutils-2.7.tar.gz>. But all
 > it does is comparing three FILES for differencies.

If it does that, then it does all you need. Assume, for example, that
you have two patches to ORIGINAL.DOC that overlap each other, these
being PATCH1.DIFF and PATCH2.DIFF respectively...

 Q> gzip -9 < ORIGINAL.DOC > ORIGINAL.DOC.GZ
 Q> patch < PATCH1.DIFF
 Q> mv ORIGINAL.DOC ORIGINAL.DOC.V1
 Q> gunzip < ORIGINAL.DOC.GZ > ORIGINAL.DOC
 Q> patch < PATCH2.DIFF
 Q> mv ORIGINAL.DOC ORIGINAL.DOC.V2

At this point, you have copies of the patched versions of that file as
produced by the two patches separately. If my memory's right (I don't
have diff3 to hand) you then do...

 Q> diff3 ORIGINAL.DOC.V1 ORIGINAL.DOC.V2 REVISED.DOC

...and end up with REVISED.DOC being the result of taking ORIGINAL.DOC
and applying both PATCH1.DIFF and PATCH2.DIFF in parallel. You can
then do...

 Q> gunzip < ORIGINAL.DOC.GZ > ORIGINAL.DOC
 Q> diff -u ORIGINAL.DOC REVISED.DOC

...to get a consolidated diff that applies both patches.

Best wishes from Riley.