All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/2] fix union merge with binary files
@ 2021-06-10 12:51 Jeff King
  2021-06-10 12:57 ` [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION Jeff King
                   ` (2 more replies)
  0 siblings, 3 replies; 7+ messages in thread
From: Jeff King @ 2021-06-10 12:51 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano

This started as an attempt to silence a "gcc -O3" warning. But I was
curious if we could trigger the problem it complains about in practice
(spoiler: we can), so I wrote a test. And it seems there was an even
bigger bug lurking, where we'd generate bogus merge results. :)

This fixes both bugs.

  [1/2]: ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION
  [2/2]: ll_union_merge(): pass name labels to ll_xdl_merge()

 ll-merge.c            |  6 ++++--
 t/t6406-merge-attr.sh | 18 ++++++++++++++++++
 2 files changed, 22 insertions(+), 2 deletions(-)

-Peff

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION
  2021-06-10 12:51 [PATCH 0/2] fix union merge with binary files Jeff King
@ 2021-06-10 12:57 ` Jeff King
  2021-06-11  3:36   ` Junio C Hamano
  2021-06-10 12:58 ` [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge() Jeff King
  2021-06-10 15:19 ` [PATCH 0/2] fix union merge with binary files Elijah Newren
  2 siblings, 1 reply; 7+ messages in thread
From: Jeff King @ 2021-06-10 12:57 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano

Prior to commit a944af1d86 (merge: teach -Xours/-Xtheirs to binary
ll-merge driver, 2012-09-08), we always reported a conflict from
ll_binary_merge() by returning "1" (in the xdl_merge and ll_merge code,
this value is the number of conflict hunks). After that commit, we
report zero conflicts if the "variant" flag is set, under the assumption
that it is one of XDL_MERGE_FAVOR_OURS or XDL_MERGE_FAVOR_THEIRS.

But this gets confused by XDL_MERGE_FAVOR_UNION. We do not know how to
do a binary union merge, but erroneously report no conflicts anyway (and
just blindly use the "ours" content as the result).

Let's tighten our check to just the cases that a944af1d86 meant to
cover. This fixes the union case (which existed already back when that
commit was made), as well as future-proofing us against any other
variants that get added later.

Note that you can't trigger this from "git merge-file --union", as that
bails on binary files before even calling into the ll-merge machinery.
The test here uses the "union" merge attribute, which does erroneously
report a successful merge.

Signed-off-by: Jeff King <peff@peff.net>
---
 ll-merge.c            |  4 +++-
 t/t6406-merge-attr.sh | 17 +++++++++++++++++
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/ll-merge.c b/ll-merge.c
index 9a8a2c365c..145deb12fa 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -91,7 +91,9 @@ static int ll_binary_merge(const struct ll_merge_driver *drv_unused,
 	 * With -Xtheirs or -Xours, we have cleanly merged;
 	 * otherwise we got a conflict.
 	 */
-	return (opts->variant ? 0 : 1);
+	return opts->variant == XDL_MERGE_FAVOR_OURS ||
+	       opts->variant == XDL_MERGE_FAVOR_THEIRS ?
+	       0 : 1;
 }
 
 static int ll_xdl_merge(const struct ll_merge_driver *drv_unused,
diff --git a/t/t6406-merge-attr.sh b/t/t6406-merge-attr.sh
index d5a4ac2d81..c1c458d933 100755
--- a/t/t6406-merge-attr.sh
+++ b/t/t6406-merge-attr.sh
@@ -207,4 +207,21 @@ test_expect_success 'custom merge does not lock index' '
 	git merge main
 '
 
+test_expect_success 'binary files with union attribute' '
+	git checkout -b bin-main &&
+	printf "base\0" >bin.txt &&
+	echo "bin.txt merge=union" >.gitattributes &&
+	git add bin.txt .gitattributes &&
+	git commit -m base &&
+
+	printf "one\0" >bin.txt &&
+	git commit -am one &&
+
+	git checkout -b bin-side HEAD^ &&
+	printf "two\0" >bin.txt &&
+	git commit -am two &&
+
+	test_must_fail git merge bin-main
+'
+
 test_done
-- 
2.32.0.529.g079a794268


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge()
  2021-06-10 12:51 [PATCH 0/2] fix union merge with binary files Jeff King
  2021-06-10 12:57 ` [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION Jeff King
@ 2021-06-10 12:58 ` Jeff King
  2021-06-10 15:19   ` Elijah Newren
  2021-06-10 15:19 ` [PATCH 0/2] fix union merge with binary files Elijah Newren
  2 siblings, 1 reply; 7+ messages in thread
From: Jeff King @ 2021-06-10 12:58 UTC (permalink / raw)
  To: git; +Cc: Junio C Hamano

Since cd1d61c44f (make union merge an xdl merge favor, 2010-03-01), we
pass NULL to ll_xdl_merge() for the "name" labels of the ancestor, ours
and theirs buffers. We usually use these for annotating conflict markers
left in a file. For a union merge, these shouldn't matter; the point of
it is that we'd never leave conflict markers in the first place.

But there is one code path where we may dereference them: if the file
contents appear to be binary, ll_binary_merge() will give up and pass
them to warning() to generate a message for the user (that was true even
when cd1d61c44f was written, though the warning was in ll_xdl_merge()
back then).

That can result in a segfault, though on many systems (including glibc),
the printf routines will helpfully just say "(null)" instead. We can
extend our binary-union test in t6406 to check stderr, which catches the
problem on all systems.

This also fixes a warning from "gcc -O3". Unlike lower optimization
levels, it inlines enough to see that the NULL can make it to warning()
and complains:

  In function ‘ll_binary_merge’,
      inlined from ‘ll_xdl_merge’ at ll-merge.c:115:10,
      inlined from ‘ll_union_merge’ at ll-merge.c:151:9:
  ll-merge.c:74:4: warning: ‘%s’ directive argument is null [-Wformat-overflow=]
     74 |    warning("Cannot merge binary files: %s (%s vs. %s)",
        |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     75 |     path, name1, name2);
        |     ~~~~~~~~~~~~~~~~~~~

Signed-off-by: Jeff King <peff@peff.net>
---
 ll-merge.c            | 2 +-
 t/t6406-merge-attr.sh | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/ll-merge.c b/ll-merge.c
index 145deb12fa..0ee34d8a01 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -151,7 +151,7 @@ static int ll_union_merge(const struct ll_merge_driver *drv_unused,
 	o = *opts;
 	o.variant = XDL_MERGE_FAVOR_UNION;
 	return ll_xdl_merge(drv_unused, result, path_unused,
-			    orig, NULL, src1, NULL, src2, NULL,
+			    orig, orig_name, src1, name1, src2, name2,
 			    &o, marker_size);
 }
 
diff --git a/t/t6406-merge-attr.sh b/t/t6406-merge-attr.sh
index c1c458d933..8494645837 100755
--- a/t/t6406-merge-attr.sh
+++ b/t/t6406-merge-attr.sh
@@ -221,7 +221,8 @@ test_expect_success 'binary files with union attribute' '
 	printf "two\0" >bin.txt &&
 	git commit -am two &&
 
-	test_must_fail git merge bin-main
+	test_must_fail git merge bin-main 2>stderr &&
+	grep -i "warning.*cannot merge.*HEAD vs. bin-main" stderr
 '
 
 test_done
-- 
2.32.0.529.g079a794268

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge()
  2021-06-10 12:58 ` [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge() Jeff King
@ 2021-06-10 15:19   ` Elijah Newren
  2021-06-10 16:14     ` [PATCH 3/2] ll_union_merge(): rename path_unused parameter Jeff King
  0 siblings, 1 reply; 7+ messages in thread
From: Elijah Newren @ 2021-06-10 15:19 UTC (permalink / raw)
  To: Jeff King; +Cc: Git Mailing List, Junio C Hamano

On Thu, Jun 10, 2021 at 6:00 AM Jeff King <peff@peff.net> wrote:
>
> Since cd1d61c44f (make union merge an xdl merge favor, 2010-03-01), we
> pass NULL to ll_xdl_merge() for the "name" labels of the ancestor, ours
> and theirs buffers. We usually use these for annotating conflict markers
> left in a file. For a union merge, these shouldn't matter; the point of
> it is that we'd never leave conflict markers in the first place.
>
> But there is one code path where we may dereference them: if the file
> contents appear to be binary, ll_binary_merge() will give up and pass
> them to warning() to generate a message for the user (that was true even
> when cd1d61c44f was written, though the warning was in ll_xdl_merge()
> back then).
>
> That can result in a segfault, though on many systems (including glibc),
> the printf routines will helpfully just say "(null)" instead. We can
> extend our binary-union test in t6406 to check stderr, which catches the
> problem on all systems.

Nice catch (as is your 1/2 as well).

> This also fixes a warning from "gcc -O3". Unlike lower optimization
> levels, it inlines enough to see that the NULL can make it to warning()
> and complains:
>
>   In function ‘ll_binary_merge’,
>       inlined from ‘ll_xdl_merge’ at ll-merge.c:115:10,
>       inlined from ‘ll_union_merge’ at ll-merge.c:151:9:
>   ll-merge.c:74:4: warning: ‘%s’ directive argument is null [-Wformat-overflow=]
>      74 |    warning("Cannot merge binary files: %s (%s vs. %s)",
>         |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>      75 |     path, name1, name2);
>         |     ~~~~~~~~~~~~~~~~~~~

So the warning uses path as well as name1 and name2...

>
> Signed-off-by: Jeff King <peff@peff.net>
> ---
>  ll-merge.c            | 2 +-
>  t/t6406-merge-attr.sh | 3 ++-
>  2 files changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/ll-merge.c b/ll-merge.c
> index 145deb12fa..0ee34d8a01 100644
> --- a/ll-merge.c
> +++ b/ll-merge.c
> @@ -151,7 +151,7 @@ static int ll_union_merge(const struct ll_merge_driver *drv_unused,
>         o = *opts;
>         o.variant = XDL_MERGE_FAVOR_UNION;
>         return ll_xdl_merge(drv_unused, result, path_unused,

Should we also rename 'path_unused' to 'path', since it is actually used?

> -                           orig, NULL, src1, NULL, src2, NULL,
> +                           orig, orig_name, src1, name1, src2, name2,
>                             &o, marker_size);
>  }
>
> diff --git a/t/t6406-merge-attr.sh b/t/t6406-merge-attr.sh
> index c1c458d933..8494645837 100755
> --- a/t/t6406-merge-attr.sh
> +++ b/t/t6406-merge-attr.sh
> @@ -221,7 +221,8 @@ test_expect_success 'binary files with union attribute' '
>         printf "two\0" >bin.txt &&
>         git commit -am two &&
>
> -       test_must_fail git merge bin-main
> +       test_must_fail git merge bin-main 2>stderr &&
> +       grep -i "warning.*cannot merge.*HEAD vs. bin-main" stderr
>  '
>
>  test_done
> --
> 2.32.0.529.g079a794268

This patch has a minor textual conflict with my remerge-diff series,
but since I haven't submitted it yet, that's my problem rather than
yours...and it will be an easy fix anyway.

Anyway, good catches.  Other than maybe considering fixing the name of
'path_unused', this looks good to me.

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 0/2] fix union merge with binary files
  2021-06-10 12:51 [PATCH 0/2] fix union merge with binary files Jeff King
  2021-06-10 12:57 ` [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION Jeff King
  2021-06-10 12:58 ` [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge() Jeff King
@ 2021-06-10 15:19 ` Elijah Newren
  2 siblings, 0 replies; 7+ messages in thread
From: Elijah Newren @ 2021-06-10 15:19 UTC (permalink / raw)
  To: Jeff King; +Cc: Git Mailing List, Junio C Hamano

On Thu, Jun 10, 2021 at 5:54 AM Jeff King <peff@peff.net> wrote:
>
> This started as an attempt to silence a "gcc -O3" warning. But I was
> curious if we could trigger the problem it complains about in practice
> (spoiler: we can), so I wrote a test. And it seems there was an even
> bigger bug lurking, where we'd generate bogus merge results. :)
>
> This fixes both bugs.

Nice catches, and fixes.  I had a minor comment on 2/2, but with or
without fixing up the 'path_unused' variable name both patches are:

Reviewed-by: Elijah Newren <newren@gmail.com>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH 3/2] ll_union_merge(): rename path_unused parameter
  2021-06-10 15:19   ` Elijah Newren
@ 2021-06-10 16:14     ` Jeff King
  0 siblings, 0 replies; 7+ messages in thread
From: Jeff King @ 2021-06-10 16:14 UTC (permalink / raw)
  To: Elijah Newren; +Cc: Git Mailing List, Junio C Hamano

On Thu, Jun 10, 2021 at 08:19:02AM -0700, Elijah Newren wrote:

> > Signed-off-by: Jeff King <peff@peff.net>
> > ---
> >  ll-merge.c            | 2 +-
> >  t/t6406-merge-attr.sh | 3 ++-
> >  2 files changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/ll-merge.c b/ll-merge.c
> > index 145deb12fa..0ee34d8a01 100644
> > --- a/ll-merge.c
> > +++ b/ll-merge.c
> > @@ -151,7 +151,7 @@ static int ll_union_merge(const struct ll_merge_driver *drv_unused,
> >         o = *opts;
> >         o.variant = XDL_MERGE_FAVOR_UNION;
> >         return ll_xdl_merge(drv_unused, result, path_unused,
> 
> Should we also rename 'path_unused' to 'path', since it is actually used?

That seems reasonable, but I'd prefer to do it as a separate patch. So
here that is.

Ironically, orig_path was unused both before this patch (where we threw
it away and passed NULL instead) and after (where we pass it into the
xdl merge, but the union merge should ignore it completely). But I
prefer not to go too wild with these kind of annotations, as they can
easily become inaccurate or out of date. If we're passing it, then we
shouldn't make too many assumptions about what xdl_merge() is doing
under the hood.

So we could also rename drv_unused mentioned below, but I didn't here.

-- >8 --
Subject: [PATCH] ll_union_merge(): rename path_unused parameter

The "path" parameter to ll_union_merge() is named "path_unused", since
we don't ourselves use it. But we do pass it to ll_xdl_merge(), which
may look at it (it gets passed to ll_binary_merge(), which may pass it
to warning()). Let's rename it to correct this inaccuracy (both of the
other functions correctly do not call this "unused").

Note that we also pass drv_unused, but it truly is unused by the rest of
the stack (it only exists at all to provide a generic interface that
matches what ll_ext_merge() needs).

Reported-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
---
 ll-merge.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ll-merge.c b/ll-merge.c
index 0ee34d8a01..261657578c 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -138,7 +138,7 @@ static int ll_xdl_merge(const struct ll_merge_driver *drv_unused,
 
 static int ll_union_merge(const struct ll_merge_driver *drv_unused,
 			  mmbuffer_t *result,
-			  const char *path_unused,
+			  const char *path,
 			  mmfile_t *orig, const char *orig_name,
 			  mmfile_t *src1, const char *name1,
 			  mmfile_t *src2, const char *name2,
@@ -150,7 +150,7 @@ static int ll_union_merge(const struct ll_merge_driver *drv_unused,
 	assert(opts);
 	o = *opts;
 	o.variant = XDL_MERGE_FAVOR_UNION;
-	return ll_xdl_merge(drv_unused, result, path_unused,
+	return ll_xdl_merge(drv_unused, result, path,
 			    orig, orig_name, src1, name1, src2, name2,
 			    &o, marker_size);
 }
-- 
2.32.0.529.g079a794268


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION
  2021-06-10 12:57 ` [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION Jeff King
@ 2021-06-11  3:36   ` Junio C Hamano
  0 siblings, 0 replies; 7+ messages in thread
From: Junio C Hamano @ 2021-06-11  3:36 UTC (permalink / raw)
  To: Jeff King; +Cc: git

Jeff King <peff@peff.net> writes:

> Prior to commit a944af1d86 (merge: teach -Xours/-Xtheirs to binary
> ll-merge driver, 2012-09-08), we always reported a conflict from
> ll_binary_merge() by returning "1" (in the xdl_merge and ll_merge code,
> this value is the number of conflict hunks). After that commit, we
> report zero conflicts if the "variant" flag is set, under the assumption
> that it is one of XDL_MERGE_FAVOR_OURS or XDL_MERGE_FAVOR_THEIRS.
>
> But this gets confused by XDL_MERGE_FAVOR_UNION. We do not know how to
> do a binary union merge, but erroneously report no conflicts anyway (and
> just blindly use the "ours" content as the result).
>
> Let's tighten our check to just the cases that a944af1d86 meant to
> cover. This fixes the union case (which existed already back when that
> commit was made), as well as future-proofing us against any other
> variants that get added later.

Makes sense.

> Note that you can't trigger this from "git merge-file --union", as that
> bails on binary files before even calling into the ll-merge machinery.
> The test here uses the "union" merge attribute, which does erroneously
> report a successful merge.

Nice discovery.

Thanks.

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2021-06-11  3:36 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-10 12:51 [PATCH 0/2] fix union merge with binary files Jeff King
2021-06-10 12:57 ` [PATCH 1/2] ll_binary_merge(): handle XDL_MERGE_FAVOR_UNION Jeff King
2021-06-11  3:36   ` Junio C Hamano
2021-06-10 12:58 ` [PATCH 2/2] ll_union_merge(): pass name labels to ll_xdl_merge() Jeff King
2021-06-10 15:19   ` Elijah Newren
2021-06-10 16:14     ` [PATCH 3/2] ll_union_merge(): rename path_unused parameter Jeff King
2021-06-10 15:19 ` [PATCH 0/2] fix union merge with binary files Elijah Newren

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.