* [PATCH] merge-ort: avoid assuming all renames detected
@ 2022-01-15 2:09 Elijah Newren via GitGitGadget
2022-01-16 21:46 ` Junio C Hamano
` (2 more replies)
0 siblings, 3 replies; 12+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-15 2:09 UTC (permalink / raw)
To: git; +Cc: Taylor Blau, Elijah Newren, Elijah Newren
From: Elijah Newren <newren@gmail.com>
In commit 8b09a900a1 ("merge-ort: restart merge with cached renames to
reduce process entry cost", 2021-07-16), we noted that in the merge-ort
steps of
collect_merge_info()
detect_and_process_renames()
process_entries()
that process_entries() was expensive, and we could often make it cheaper
by changing this to
collect_merge_info()
detect_and_process_renames()
<cache all the renames, and restart>
collect_merge_info()
detect_and_process_renames()
process_entries()
because the second collect_merge_info() would be cheaper (we could avoid
traversing into some directories), the second
detect_and_process_renames() would be free since we had already detected
all renames, and then process_entries() has far fewer entries to handle.
However, this was built on the assumption that the first
detect_and_process_renames() actually detected all potential renames.
If someone has merge.renameLimit set to some small value, that
assumption is violated which manifests later with the following message:
$ git -c merge.renameLimit=1 rebase upstream
...
git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion
`renames->cached_pairs_valid_side == 0' failed.
Turn off this cache-renames-and-restart whenever we cannot detect all
renames, and add a testcase that would have caught this problem.
Reported-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
merge-ort: avoid assuming all renames detected
Fixes https://lore.kernel.org/git/YeHTIfEutLYM4TIU@nand.local/
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1194%2Fnewren%2Favoid-assertion-assuming-renames-found-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1194/newren/avoid-assertion-assuming-renames-found-v1
Pull-Request: https://github.com/git/git/pull/1194
merge-ort.c | 4 ++
t/t6429-merge-sequence-rename-caching.sh | 67 ++++++++++++++++++++++++
2 files changed, 71 insertions(+)
diff --git a/merge-ort.c b/merge-ort.c
index c3197970219..143d274f117 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -3060,6 +3060,10 @@ static int detect_and_process_renames(struct merge_options *opt,
trace2_region_enter("merge", "regular renames", opt->repo);
detection_run |= detect_regular_renames(opt, MERGE_SIDE1);
detection_run |= detect_regular_renames(opt, MERGE_SIDE2);
+ if (renames->needed_limit != 0) {
+ renames->cached_pairs_valid_side = 0;
+ renames->redo_after_renames = 0;
+ }
if (renames->redo_after_renames && detection_run) {
int i, side;
struct diff_filepair *p;
diff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh
index 035edc40b1e..f2bc8a7d2a2 100755
--- a/t/t6429-merge-sequence-rename-caching.sh
+++ b/t/t6429-merge-sequence-rename-caching.sh
@@ -697,4 +697,71 @@ test_expect_success 'caching renames only on upstream side, part 2' '
)
'
+#
+# The following testcase just creates two simple renames (slightly modified
+# on both sides but without conflicting changes), and a directory full of
+# files that are otherwise uninteresting. The setup is as follows:
+#
+# base: unrelated/<BUNCH OF FILES>
+# numbers
+# values
+# upstream: modify: numbers
+# modify: values
+# topic: add: unrelated/foo
+# modify: numbers
+# modify: values
+# rename: numbers -> sequence
+# rename: values -> progression
+#
+# This is a trivial rename case, but we're curious what happens with a very
+# low renameLimit interacting with the restart optimization trying to notice
+# that unrelated/ looks like a trivial merge candidate.
+#
+test_expect_success 'avoid assuming we detected renames' '
+ git init redo-weirdness &&
+ (
+ cd redo-weirdness &&
+
+ mkdir unrelated &&
+ for i in $(test_seq 1 10)
+ do
+ >unrelated/$i
+ done &&
+ test_seq 2 10 >numbers &&
+ test_seq 12 20 >values &&
+ git add numbers values unrelated/ &&
+ git commit -m orig &&
+
+ git branch upstream &&
+ git branch topic &&
+
+ git switch upstream &&
+ test_seq 1 10 >numbers &&
+ test_seq 11 20 >values &&
+ git add numbers &&
+ git commit -m "Some tweaks" &&
+
+ git switch topic &&
+
+ >unrelated/foo &&
+ test_seq 2 12 >numbers &&
+ test_seq 12 22 >values &&
+ git add numbers values unrelated/ &&
+ git mv numbers sequence &&
+ git mv values progression &&
+ git commit -m A &&
+
+ #
+ # Actual testing
+ #
+
+ git switch --detach topic^0 &&
+
+ test_must_fail git -c merge.renameLimit=1 rebase upstream &&
+
+ git ls-files -u >actual &&
+ ! test_file_is_empty actual
+ )
+'
+
test_done
base-commit: 1ffcbaa1a5f10c9f706314d77f88de20a4a498c2
--
gitgitgadget
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH] merge-ort: avoid assuming all renames detected
2022-01-15 2:09 [PATCH] merge-ort: avoid assuming all renames detected Elijah Newren via GitGitGadget
@ 2022-01-16 21:46 ` Junio C Hamano
2022-01-16 23:19 ` Junio C Hamano
2022-01-17 18:25 ` [PATCH v2] " Elijah Newren via GitGitGadget
2 siblings, 0 replies; 12+ messages in thread
From: Junio C Hamano @ 2022-01-16 21:46 UTC (permalink / raw)
To: Elijah Newren via GitGitGadget; +Cc: git, Taylor Blau, Elijah Newren
"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
> From: Elijah Newren <newren@gmail.com>
>
> In commit 8b09a900a1 ("merge-ort: restart merge with cached renames to
> reduce process entry cost", 2021-07-16), we noted that in the merge-ort
> steps of
> collect_merge_info()
> detect_and_process_renames()
> process_entries()
> that process_entries() was expensive, and we could often make it cheaper
> by changing this to
> collect_merge_info()
> detect_and_process_renames()
> <cache all the renames, and restart>
> collect_merge_info()
> detect_and_process_renames()
> process_entries()
> because the second collect_merge_info() would be cheaper (we could avoid
> traversing into some directories), the second
> detect_and_process_renames() would be free since we had already detected
> all renames, and then process_entries() has far fewer entries to handle.
>
> However, this was built on the assumption that the first
> detect_and_process_renames() actually detected all potential renames.
> If someone has merge.renameLimit set to some small value, that
> assumption is violated which manifests later with the following message:
>
> $ git -c merge.renameLimit=1 rebase upstream
> ...
> git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion
> `renames->cached_pairs_valid_side == 0' failed.
>
> Turn off this cache-renames-and-restart whenever we cannot detect all
> renames, and add a testcase that would have caught this problem.
>
> Reported-by: Taylor Blau <me@ttaylorr.com>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
Thanks. Will fork from maint-2.33 and park on 'seen' for now.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] merge-ort: avoid assuming all renames detected
2022-01-15 2:09 [PATCH] merge-ort: avoid assuming all renames detected Elijah Newren via GitGitGadget
2022-01-16 21:46 ` Junio C Hamano
@ 2022-01-16 23:19 ` Junio C Hamano
2022-01-17 18:25 ` [PATCH v2] " Elijah Newren via GitGitGadget
2 siblings, 0 replies; 12+ messages in thread
From: Junio C Hamano @ 2022-01-16 23:19 UTC (permalink / raw)
To: Elijah Newren via GitGitGadget; +Cc: git, Taylor Blau, Elijah Newren
"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
> diff --git a/merge-ort.c b/merge-ort.c
> index c3197970219..143d274f117 100644
> --- a/merge-ort.c
> +++ b/merge-ort.c
> @@ -3060,6 +3060,10 @@ static int detect_and_process_renames(struct merge_options *opt,
> trace2_region_enter("merge", "regular renames", opt->repo);
> detection_run |= detect_regular_renames(opt, MERGE_SIDE1);
> detection_run |= detect_regular_renames(opt, MERGE_SIDE2);
> + if (renames->needed_limit != 0) {
Don't compare with NULL or 0, i.e.
if (renames->needed_limit) {
> + renames->cached_pairs_valid_side = 0;
> + renames->redo_after_renames = 0;
> + }
> if (renames->redo_after_renames && detection_run) {
> int i, side;
> struct diff_filepair *p;
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v2] merge-ort: avoid assuming all renames detected
2022-01-15 2:09 [PATCH] merge-ort: avoid assuming all renames detected Elijah Newren via GitGitGadget
2022-01-16 21:46 ` Junio C Hamano
2022-01-16 23:19 ` Junio C Hamano
@ 2022-01-17 18:25 ` Elijah Newren via GitGitGadget
2022-01-17 19:33 ` Junio C Hamano
2022-06-30 9:53 ` SZEDER Gábor
2 siblings, 2 replies; 12+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-17 18:25 UTC (permalink / raw)
To: git; +Cc: Taylor Blau, Elijah Newren, Elijah Newren
From: Elijah Newren <newren@gmail.com>
In commit 8b09a900a1 ("merge-ort: restart merge with cached renames to
reduce process entry cost", 2021-07-16), we noted that in the merge-ort
steps of
collect_merge_info()
detect_and_process_renames()
process_entries()
that process_entries() was expensive, and we could often make it cheaper
by changing this to
collect_merge_info()
detect_and_process_renames()
<cache all the renames, and restart>
collect_merge_info()
detect_and_process_renames()
process_entries()
because the second collect_merge_info() would be cheaper (we could avoid
traversing into some directories), the second
detect_and_process_renames() would be free since we had already detected
all renames, and then process_entries() has far fewer entries to handle.
However, this was built on the assumption that the first
detect_and_process_renames() actually detected all potential renames.
If someone has merge.renameLimit set to some small value, that
assumption is violated which manifests later with the following message:
$ git -c merge.renameLimit=1 rebase upstream
...
git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion
`renames->cached_pairs_valid_side == 0' failed.
Turn off this cache-renames-and-restart whenever we cannot detect all
renames, and add a testcase that would have caught this problem.
Reported-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
merge-ort: avoid assuming all renames detected
Fixes https://lore.kernel.org/git/YeHTIfEutLYM4TIU@nand.local/
Changes since v1:
* Fixed a small style issue
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1194%2Fnewren%2Favoid-assertion-assuming-renames-found-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1194/newren/avoid-assertion-assuming-renames-found-v2
Pull-Request: https://github.com/git/git/pull/1194
Range-diff vs v1:
1: f1e9901ae67 ! 1: 239d3ba08c1 merge-ort: avoid assuming all renames detected
@@ merge-ort.c: static int detect_and_process_renames(struct merge_options *opt,
trace2_region_enter("merge", "regular renames", opt->repo);
detection_run |= detect_regular_renames(opt, MERGE_SIDE1);
detection_run |= detect_regular_renames(opt, MERGE_SIDE2);
-+ if (renames->needed_limit != 0) {
++ if (renames->needed_limit) {
+ renames->cached_pairs_valid_side = 0;
+ renames->redo_after_renames = 0;
+ }
merge-ort.c | 4 ++
t/t6429-merge-sequence-rename-caching.sh | 67 ++++++++++++++++++++++++
2 files changed, 71 insertions(+)
diff --git a/merge-ort.c b/merge-ort.c
index c3197970219..b0ff9a72879 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -3060,6 +3060,10 @@ static int detect_and_process_renames(struct merge_options *opt,
trace2_region_enter("merge", "regular renames", opt->repo);
detection_run |= detect_regular_renames(opt, MERGE_SIDE1);
detection_run |= detect_regular_renames(opt, MERGE_SIDE2);
+ if (renames->needed_limit) {
+ renames->cached_pairs_valid_side = 0;
+ renames->redo_after_renames = 0;
+ }
if (renames->redo_after_renames && detection_run) {
int i, side;
struct diff_filepair *p;
diff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh
index 035edc40b1e..f2bc8a7d2a2 100755
--- a/t/t6429-merge-sequence-rename-caching.sh
+++ b/t/t6429-merge-sequence-rename-caching.sh
@@ -697,4 +697,71 @@ test_expect_success 'caching renames only on upstream side, part 2' '
)
'
+#
+# The following testcase just creates two simple renames (slightly modified
+# on both sides but without conflicting changes), and a directory full of
+# files that are otherwise uninteresting. The setup is as follows:
+#
+# base: unrelated/<BUNCH OF FILES>
+# numbers
+# values
+# upstream: modify: numbers
+# modify: values
+# topic: add: unrelated/foo
+# modify: numbers
+# modify: values
+# rename: numbers -> sequence
+# rename: values -> progression
+#
+# This is a trivial rename case, but we're curious what happens with a very
+# low renameLimit interacting with the restart optimization trying to notice
+# that unrelated/ looks like a trivial merge candidate.
+#
+test_expect_success 'avoid assuming we detected renames' '
+ git init redo-weirdness &&
+ (
+ cd redo-weirdness &&
+
+ mkdir unrelated &&
+ for i in $(test_seq 1 10)
+ do
+ >unrelated/$i
+ done &&
+ test_seq 2 10 >numbers &&
+ test_seq 12 20 >values &&
+ git add numbers values unrelated/ &&
+ git commit -m orig &&
+
+ git branch upstream &&
+ git branch topic &&
+
+ git switch upstream &&
+ test_seq 1 10 >numbers &&
+ test_seq 11 20 >values &&
+ git add numbers &&
+ git commit -m "Some tweaks" &&
+
+ git switch topic &&
+
+ >unrelated/foo &&
+ test_seq 2 12 >numbers &&
+ test_seq 12 22 >values &&
+ git add numbers values unrelated/ &&
+ git mv numbers sequence &&
+ git mv values progression &&
+ git commit -m A &&
+
+ #
+ # Actual testing
+ #
+
+ git switch --detach topic^0 &&
+
+ test_must_fail git -c merge.renameLimit=1 rebase upstream &&
+
+ git ls-files -u >actual &&
+ ! test_file_is_empty actual
+ )
+'
+
test_done
base-commit: 1ffcbaa1a5f10c9f706314d77f88de20a4a498c2
--
gitgitgadget
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH v2] merge-ort: avoid assuming all renames detected
2022-01-17 18:25 ` [PATCH v2] " Elijah Newren via GitGitGadget
@ 2022-01-17 19:33 ` Junio C Hamano
2022-01-17 21:21 ` Elijah Newren
2022-06-30 9:53 ` SZEDER Gábor
1 sibling, 1 reply; 12+ messages in thread
From: Junio C Hamano @ 2022-01-17 19:33 UTC (permalink / raw)
To: Taylor Blau, Elijah Newren via GitGitGadget; +Cc: git, Elijah Newren
"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
> From: Elijah Newren <newren@gmail.com>
>
> In commit 8b09a900a1 ("merge-ort: restart merge with cached renames to
> reduce process entry cost", 2021-07-16), we noted that in the merge-ort
> steps of
> collect_merge_info()
> detect_and_process_renames()
> process_entries()
> that process_entries() was expensive, and we could often make it cheaper
> by changing this to
> collect_merge_info()
> detect_and_process_renames()
> <cache all the renames, and restart>
> collect_merge_info()
> detect_and_process_renames()
> process_entries()
> because the second collect_merge_info() would be cheaper (we could avoid
> traversing into some directories), the second
> detect_and_process_renames() would be free since we had already detected
> all renames, and then process_entries() has far fewer entries to handle.
>
> However, this was built on the assumption that the first
> detect_and_process_renames() actually detected all potential renames.
> If someone has merge.renameLimit set to some small value, that
> assumption is violated which manifests later with the following message:
>
> $ git -c merge.renameLimit=1 rebase upstream
> ...
> git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion
> `renames->cached_pairs_valid_side == 0' failed.
>
> Turn off this cache-renames-and-restart whenever we cannot detect all
> renames, and add a testcase that would have caught this problem.
>
> Reported-by: Taylor Blau <me@ttaylorr.com>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
Thanks. An Ack?
> merge-ort: avoid assuming all renames detected
>
> Fixes https://lore.kernel.org/git/YeHTIfEutLYM4TIU@nand.local/
>
> Changes since v1:
>
> * Fixed a small style issue
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1194%2Fnewren%2Favoid-assertion-assuming-renames-found-v2
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1194/newren/avoid-assertion-assuming-renames-found-v2
> Pull-Request: https://github.com/git/git/pull/1194
>
> Range-diff vs v1:
>
> 1: f1e9901ae67 ! 1: 239d3ba08c1 merge-ort: avoid assuming all renames detected
> @@ merge-ort.c: static int detect_and_process_renames(struct merge_options *opt,
> trace2_region_enter("merge", "regular renames", opt->repo);
> detection_run |= detect_regular_renames(opt, MERGE_SIDE1);
> detection_run |= detect_regular_renames(opt, MERGE_SIDE2);
> -+ if (renames->needed_limit != 0) {
> ++ if (renames->needed_limit) {
> + renames->cached_pairs_valid_side = 0;
> + renames->redo_after_renames = 0;
> + }
>
>
> merge-ort.c | 4 ++
> t/t6429-merge-sequence-rename-caching.sh | 67 ++++++++++++++++++++++++
> 2 files changed, 71 insertions(+)
>
> diff --git a/merge-ort.c b/merge-ort.c
> index c3197970219..b0ff9a72879 100644
> --- a/merge-ort.c
> +++ b/merge-ort.c
> @@ -3060,6 +3060,10 @@ static int detect_and_process_renames(struct merge_options *opt,
> trace2_region_enter("merge", "regular renames", opt->repo);
> detection_run |= detect_regular_renames(opt, MERGE_SIDE1);
> detection_run |= detect_regular_renames(opt, MERGE_SIDE2);
> + if (renames->needed_limit) {
> + renames->cached_pairs_valid_side = 0;
> + renames->redo_after_renames = 0;
> + }
> if (renames->redo_after_renames && detection_run) {
> int i, side;
> struct diff_filepair *p;
> diff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh
> index 035edc40b1e..f2bc8a7d2a2 100755
> --- a/t/t6429-merge-sequence-rename-caching.sh
> +++ b/t/t6429-merge-sequence-rename-caching.sh
> @@ -697,4 +697,71 @@ test_expect_success 'caching renames only on upstream side, part 2' '
> )
> '
>
> +#
> +# The following testcase just creates two simple renames (slightly modified
> +# on both sides but without conflicting changes), and a directory full of
> +# files that are otherwise uninteresting. The setup is as follows:
> +#
> +# base: unrelated/<BUNCH OF FILES>
> +# numbers
> +# values
> +# upstream: modify: numbers
> +# modify: values
> +# topic: add: unrelated/foo
> +# modify: numbers
> +# modify: values
> +# rename: numbers -> sequence
> +# rename: values -> progression
> +#
> +# This is a trivial rename case, but we're curious what happens with a very
> +# low renameLimit interacting with the restart optimization trying to notice
> +# that unrelated/ looks like a trivial merge candidate.
> +#
> +test_expect_success 'avoid assuming we detected renames' '
> + git init redo-weirdness &&
> + (
> + cd redo-weirdness &&
> +
> + mkdir unrelated &&
> + for i in $(test_seq 1 10)
> + do
> + >unrelated/$i
> + done &&
> + test_seq 2 10 >numbers &&
> + test_seq 12 20 >values &&
> + git add numbers values unrelated/ &&
> + git commit -m orig &&
> +
> + git branch upstream &&
> + git branch topic &&
> +
> + git switch upstream &&
> + test_seq 1 10 >numbers &&
> + test_seq 11 20 >values &&
> + git add numbers &&
> + git commit -m "Some tweaks" &&
> +
> + git switch topic &&
> +
> + >unrelated/foo &&
> + test_seq 2 12 >numbers &&
> + test_seq 12 22 >values &&
> + git add numbers values unrelated/ &&
> + git mv numbers sequence &&
> + git mv values progression &&
> + git commit -m A &&
> +
> + #
> + # Actual testing
> + #
> +
> + git switch --detach topic^0 &&
> +
> + test_must_fail git -c merge.renameLimit=1 rebase upstream &&
> +
> + git ls-files -u >actual &&
> + ! test_file_is_empty actual
> + )
> +'
> +
> test_done
>
> base-commit: 1ffcbaa1a5f10c9f706314d77f88de20a4a498c2
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2] merge-ort: avoid assuming all renames detected
2022-01-17 19:33 ` Junio C Hamano
@ 2022-01-17 21:21 ` Elijah Newren
2022-01-17 22:07 ` Taylor Blau
2022-01-17 22:16 ` Junio C Hamano
0 siblings, 2 replies; 12+ messages in thread
From: Elijah Newren @ 2022-01-17 21:21 UTC (permalink / raw)
To: Junio C Hamano
Cc: Taylor Blau, Elijah Newren via GitGitGadget, Git Mailing List
On Mon, Jan 17, 2022 at 11:33 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > In commit 8b09a900a1 ("merge-ort: restart merge with cached renames to
> > reduce process entry cost", 2021-07-16), we noted that in the merge-ort
> > steps of
> > collect_merge_info()
> > detect_and_process_renames()
> > process_entries()
> > that process_entries() was expensive, and we could often make it cheaper
> > by changing this to
> > collect_merge_info()
> > detect_and_process_renames()
> > <cache all the renames, and restart>
> > collect_merge_info()
> > detect_and_process_renames()
> > process_entries()
> > because the second collect_merge_info() would be cheaper (we could avoid
> > traversing into some directories), the second
> > detect_and_process_renames() would be free since we had already detected
> > all renames, and then process_entries() has far fewer entries to handle.
> >
> > However, this was built on the assumption that the first
> > detect_and_process_renames() actually detected all potential renames.
> > If someone has merge.renameLimit set to some small value, that
> > assumption is violated which manifests later with the following message:
> >
> > $ git -c merge.renameLimit=1 rebase upstream
> > ...
> > git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion
> > `renames->cached_pairs_valid_side == 0' failed.
> >
> > Turn off this cache-renames-and-restart whenever we cannot detect all
> > renames, and add a testcase that would have caught this problem.
> >
> > Reported-by: Taylor Blau <me@ttaylorr.com>
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
>
> Thanks. An Ack?
Taylor told me the code change fixed his case, and that he'd review my
full patch with the testcase when I posted it. Let's wait to hear
from him.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2] merge-ort: avoid assuming all renames detected
2022-01-17 21:21 ` Elijah Newren
@ 2022-01-17 22:07 ` Taylor Blau
2022-01-17 22:23 ` Junio C Hamano
2022-01-17 22:16 ` Junio C Hamano
1 sibling, 1 reply; 12+ messages in thread
From: Taylor Blau @ 2022-01-17 22:07 UTC (permalink / raw)
To: Elijah Newren
Cc: Junio C Hamano, Taylor Blau, Elijah Newren via GitGitGadget,
Git Mailing List
On Mon, Jan 17, 2022 at 01:21:11PM -0800, Elijah Newren wrote:
> On Mon, Jan 17, 2022 at 11:33 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> > Thanks. An Ack?
>
> Taylor told me the code change fixed his case, and that he'd review my
> full patch with the testcase when I posted it. Let's wait to hear
> from him.
Ack. I can't vouch for the ort-specific details, but I trust Elijah's
judgement (obviously). Running a version of Git with this patch applied
fixes the issue I originally reported.
Thanks, Elijah!
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2] merge-ort: avoid assuming all renames detected
2022-01-17 21:21 ` Elijah Newren
2022-01-17 22:07 ` Taylor Blau
@ 2022-01-17 22:16 ` Junio C Hamano
1 sibling, 0 replies; 12+ messages in thread
From: Junio C Hamano @ 2022-01-17 22:16 UTC (permalink / raw)
To: Elijah Newren
Cc: Taylor Blau, Elijah Newren via GitGitGadget, Git Mailing List
Elijah Newren <newren@gmail.com> writes:
> On Mon, Jan 17, 2022 at 11:33 AM Junio C Hamano <gitster@pobox.com> wrote:
>>
>> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>>
>> > From: Elijah Newren <newren@gmail.com>
>> >
>> > In commit 8b09a900a1 ("merge-ort: restart merge with cached renames to
>> > reduce process entry cost", 2021-07-16), we noted that in the merge-ort
>> > steps of
>> > collect_merge_info()
>> > detect_and_process_renames()
>> > process_entries()
>> > that process_entries() was expensive, and we could often make it cheaper
>> > by changing this to
>> > collect_merge_info()
>> > detect_and_process_renames()
>> > <cache all the renames, and restart>
>> > collect_merge_info()
>> > detect_and_process_renames()
>> > process_entries()
>> > because the second collect_merge_info() would be cheaper (we could avoid
>> > traversing into some directories), the second
>> > detect_and_process_renames() would be free since we had already detected
>> > all renames, and then process_entries() has far fewer entries to handle.
>> >
>> > However, this was built on the assumption that the first
>> > detect_and_process_renames() actually detected all potential renames.
>> > If someone has merge.renameLimit set to some small value, that
>> > assumption is violated which manifests later with the following message:
>> >
>> > $ git -c merge.renameLimit=1 rebase upstream
>> > ...
>> > git: merge-ort.c:546: clear_or_reinit_internal_opts: Assertion
>> > `renames->cached_pairs_valid_side == 0' failed.
>> >
>> > Turn off this cache-renames-and-restart whenever we cannot detect all
>> > renames, and add a testcase that would have caught this problem.
>> >
>> > Reported-by: Taylor Blau <me@ttaylorr.com>
>> > Signed-off-by: Elijah Newren <newren@gmail.com>
>> > ---
>>
>> Thanks. An Ack?
>
> Taylor told me the code change fixed his case, and that he'd review my
> full patch with the testcase when I posted it. Let's wait to hear
> from him.
Yes, I am waiting (notice who is on To: and not Cc: on the message
you are responding to ;-).
Thanks.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2] merge-ort: avoid assuming all renames detected
2022-01-17 22:07 ` Taylor Blau
@ 2022-01-17 22:23 ` Junio C Hamano
0 siblings, 0 replies; 12+ messages in thread
From: Junio C Hamano @ 2022-01-17 22:23 UTC (permalink / raw)
To: Taylor Blau
Cc: Elijah Newren, Elijah Newren via GitGitGadget, Git Mailing List
Taylor Blau <me@ttaylorr.com> writes:
> On Mon, Jan 17, 2022 at 01:21:11PM -0800, Elijah Newren wrote:
>> On Mon, Jan 17, 2022 at 11:33 AM Junio C Hamano <gitster@pobox.com> wrote:
>>
>> > Thanks. An Ack?
>>
>> Taylor told me the code change fixed his case, and that he'd review my
>> full patch with the testcase when I posted it. Let's wait to hear
>> from him.
>
> Ack. I can't vouch for the ort-specific details, but I trust Elijah's
> judgement (obviously). Running a version of Git with this patch applied
> fixes the issue I originally reported.
>
> Thanks, Elijah!
Thanks. Will queue with your Tested-by, then.
Thank you very much, both of you.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2] merge-ort: avoid assuming all renames detected
2022-01-17 18:25 ` [PATCH v2] " Elijah Newren via GitGitGadget
2022-01-17 19:33 ` Junio C Hamano
@ 2022-06-30 9:53 ` SZEDER Gábor
2022-07-01 2:30 ` Elijah Newren
1 sibling, 1 reply; 12+ messages in thread
From: SZEDER Gábor @ 2022-06-30 9:53 UTC (permalink / raw)
To: Elijah Newren via GitGitGadget; +Cc: git, Taylor Blau, Elijah Newren
On Mon, Jan 17, 2022 at 06:25:55PM +0000, Elijah Newren via GitGitGadget wrote:
> diff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh
> index 035edc40b1e..f2bc8a7d2a2 100755
> --- a/t/t6429-merge-sequence-rename-caching.sh
> +++ b/t/t6429-merge-sequence-rename-caching.sh
> @@ -697,4 +697,71 @@ test_expect_success 'caching renames only on upstream side, part 2' '
> )
> '
>
> +#
> +# The following testcase just creates two simple renames (slightly modified
> +# on both sides but without conflicting changes), and a directory full of
> +# files that are otherwise uninteresting. The setup is as follows:
> +#
> +# base: unrelated/<BUNCH OF FILES>
> +# numbers
> +# values
> +# upstream: modify: numbers
> +# modify: values
> +# topic: add: unrelated/foo
> +# modify: numbers
> +# modify: values
> +# rename: numbers -> sequence
> +# rename: values -> progression
> +#
> +# This is a trivial rename case, but we're curious what happens with a very
> +# low renameLimit interacting with the restart optimization trying to notice
> +# that unrelated/ looks like a trivial merge candidate.
> +#
> +test_expect_success 'avoid assuming we detected renames' '
> + git init redo-weirdness &&
> + (
> + cd redo-weirdness &&
> +
> + mkdir unrelated &&
> + for i in $(test_seq 1 10)
> + do
> + >unrelated/$i
> + done &&
> + test_seq 2 10 >numbers &&
> + test_seq 12 20 >values &&
> + git add numbers values unrelated/ &&
> + git commit -m orig &&
> +
> + git branch upstream &&
> + git branch topic &&
> +
> + git switch upstream &&
> + test_seq 1 10 >numbers &&
> + test_seq 11 20 >values &&
> + git add numbers &&
> + git commit -m "Some tweaks" &&
> +
> + git switch topic &&
> +
> + >unrelated/foo &&
> + test_seq 2 12 >numbers &&
> + test_seq 12 22 >values &&
> + git add numbers values unrelated/ &&
> + git mv numbers sequence &&
> + git mv values progression &&
> + git commit -m A &&
> +
> + #
> + # Actual testing
> + #
> +
> + git switch --detach topic^0 &&
> +
> + test_must_fail git -c merge.renameLimit=1 rebase upstream &&
> +
> + git ls-files -u >actual &&
> + ! test_file_is_empty actual
There is no 'test_file_is_empty' function, but because of the ! at the
beginning of the line it didn't fail the test.
The minimal fix would be to use 'test_file_not_empty' instead, but I
wonder whether we should use 'test_line_count = 2' instead for a tad
tighter check.
> + )
> +'
> +
> test_done
>
> base-commit: 1ffcbaa1a5f10c9f706314d77f88de20a4a498c2
> --
> gitgitgadget
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2] merge-ort: avoid assuming all renames detected
2022-06-30 9:53 ` SZEDER Gábor
@ 2022-07-01 2:30 ` Elijah Newren
2022-07-01 5:21 ` Elijah Newren
0 siblings, 1 reply; 12+ messages in thread
From: Elijah Newren @ 2022-07-01 2:30 UTC (permalink / raw)
To: SZEDER Gábor
Cc: Elijah Newren via GitGitGadget, Git Mailing List, Taylor Blau
On Thu, Jun 30, 2022 at 2:54 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
>
> On Mon, Jan 17, 2022 at 06:25:55PM +0000, Elijah Newren via GitGitGadget wrote:
> > diff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh
> > index 035edc40b1e..f2bc8a7d2a2 100755
> > --- a/t/t6429-merge-sequence-rename-caching.sh
> > +++ b/t/t6429-merge-sequence-rename-caching.sh
> > @@ -697,4 +697,71 @@ test_expect_success 'caching renames only on upstream side, part 2' '
> > )
> > '
> >
> > +#
> > +# The following testcase just creates two simple renames (slightly modified
> > +# on both sides but without conflicting changes), and a directory full of
> > +# files that are otherwise uninteresting. The setup is as follows:
> > +#
> > +# base: unrelated/<BUNCH OF FILES>
> > +# numbers
> > +# values
> > +# upstream: modify: numbers
> > +# modify: values
> > +# topic: add: unrelated/foo
> > +# modify: numbers
> > +# modify: values
> > +# rename: numbers -> sequence
> > +# rename: values -> progression
> > +#
> > +# This is a trivial rename case, but we're curious what happens with a very
> > +# low renameLimit interacting with the restart optimization trying to notice
> > +# that unrelated/ looks like a trivial merge candidate.
> > +#
> > +test_expect_success 'avoid assuming we detected renames' '
> > + git init redo-weirdness &&
> > + (
> > + cd redo-weirdness &&
> > +
> > + mkdir unrelated &&
> > + for i in $(test_seq 1 10)
> > + do
> > + >unrelated/$i
> > + done &&
> > + test_seq 2 10 >numbers &&
> > + test_seq 12 20 >values &&
> > + git add numbers values unrelated/ &&
> > + git commit -m orig &&
> > +
> > + git branch upstream &&
> > + git branch topic &&
> > +
> > + git switch upstream &&
> > + test_seq 1 10 >numbers &&
> > + test_seq 11 20 >values &&
> > + git add numbers &&
> > + git commit -m "Some tweaks" &&
> > +
> > + git switch topic &&
> > +
> > + >unrelated/foo &&
> > + test_seq 2 12 >numbers &&
> > + test_seq 12 22 >values &&
> > + git add numbers values unrelated/ &&
> > + git mv numbers sequence &&
> > + git mv values progression &&
> > + git commit -m A &&
> > +
> > + #
> > + # Actual testing
> > + #
> > +
> > + git switch --detach topic^0 &&
> > +
> > + test_must_fail git -c merge.renameLimit=1 rebase upstream &&
> > +
> > + git ls-files -u >actual &&
> > + ! test_file_is_empty actual
>
> There is no 'test_file_is_empty' function, but because of the ! at the
> beginning of the line it didn't fail the test.
Oops, looks like I meant test_must_be_empty.
> The minimal fix would be to use 'test_file_not_empty' instead, but I
> wonder whether we should use 'test_line_count = 2' instead for a tad
> tighter check.
Makes sense; since this merged about half a year ago, I'll submit a
new patch to fix this. Thanks for catching and pointing it out!
>
> > + )
> > +'
> > +
> > test_done
> >
> > base-commit: 1ffcbaa1a5f10c9f706314d77f88de20a4a498c2
> > --
> > gitgitgadget
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2] merge-ort: avoid assuming all renames detected
2022-07-01 2:30 ` Elijah Newren
@ 2022-07-01 5:21 ` Elijah Newren
0 siblings, 0 replies; 12+ messages in thread
From: Elijah Newren @ 2022-07-01 5:21 UTC (permalink / raw)
To: SZEDER Gábor
Cc: Elijah Newren via GitGitGadget, Git Mailing List, Taylor Blau
On Thu, Jun 30, 2022 at 7:30 PM Elijah Newren <newren@gmail.com> wrote:
>
> On Thu, Jun 30, 2022 at 2:54 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
> >
> > On Mon, Jan 17, 2022 at 06:25:55PM +0000, Elijah Newren via GitGitGadget wrote:
> > > diff --git a/t/t6429-merge-sequence-rename-caching.sh b/t/t6429-merge-sequence-rename-caching.sh
> > > index 035edc40b1e..f2bc8a7d2a2 100755
> > > --- a/t/t6429-merge-sequence-rename-caching.sh
> > > +++ b/t/t6429-merge-sequence-rename-caching.sh
[...]
> > > + git ls-files -u >actual &&
> > > + ! test_file_is_empty actual
> >
> > There is no 'test_file_is_empty' function, but because of the ! at the
> > beginning of the line it didn't fail the test.
>
> Oops, looks like I meant test_must_be_empty.
>
> > The minimal fix would be to use 'test_file_not_empty' instead, but I
> > wonder whether we should use 'test_line_count = 2' instead for a tad
> > tighter check.
>
> Makes sense; since this merged about half a year ago, I'll submit a
> new patch to fix this. Thanks for catching and pointing it out!
Submitted over here:
https://lore.kernel.org/git/pull.1276.git.1656652799863.gitgitgadget@gmail.com/
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2022-07-01 5:21 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-15 2:09 [PATCH] merge-ort: avoid assuming all renames detected Elijah Newren via GitGitGadget
2022-01-16 21:46 ` Junio C Hamano
2022-01-16 23:19 ` Junio C Hamano
2022-01-17 18:25 ` [PATCH v2] " Elijah Newren via GitGitGadget
2022-01-17 19:33 ` Junio C Hamano
2022-01-17 21:21 ` Elijah Newren
2022-01-17 22:07 ` Taylor Blau
2022-01-17 22:23 ` Junio C Hamano
2022-01-17 22:16 ` Junio C Hamano
2022-06-30 9:53 ` SZEDER Gábor
2022-07-01 2:30 ` Elijah Newren
2022-07-01 5:21 ` Elijah Newren
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.