Re: [PATCH v2 0/8] Optimization batch 9: avoid detecting irrelevant renames

From: Elijah Newren <newren@gmail.com>
To: Elijah Newren via GitGitGadget <gitgitgadget@gmail.com>
Cc: "Git Mailing List" <git@vger.kernel.org>,
	"Derrick Stolee" <dstolee@microsoft.com>,
	"Jonathan Tan" <jonathantanmy@google.com>,
	"Taylor Blau" <me@ttaylorr.com>,
	"Junio C Hamano" <gitster@pobox.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: Re: [PATCH v2 0/8] Optimization batch 9: avoid detecting irrelevant renames
Date: Mon, 15 Mar 2021 10:10:50 -0700	[thread overview]
Message-ID: <CABPp-BGEJ+Qvb4C0bgnMEoxVZfYj1NMrtYF-opLG2gcwJ3-v_A@mail.gmail.com> (raw)
In-Reply-To: <pull.845.v2.git.1615248599.gitgitgadget@gmail.com>

On Mon, Mar 8, 2021 at 4:10 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> This series depends textually on ort-perf-batch-8, but semantically it's
> almost completely unrelated and can be reviewed without any familiarity with
> any of the previous patch series.
>
> There are no changes since v1; it's just a resend just over a week later to
> bump it so it isn't lost.
>
> === Basic Optimization idea ===
>
> This series determines paths which meet special cases where detection of
> renames is irrelevant, where the irrelevance is due to the fact that the
> merge machinery will arrive at the same result regardless of whether a
> rename is detected for any of those paths. This series represents
> "Optimization #2" from my Git Merge 2020 talk[1], though this series has
> some improvements on the optimization relative to what I had at that time.
>
> The basic idea here is:
>
> We only need expensive rename detection on the subset of files changed on
> both sides of history (for the most part).

I know this series was already reviewed and even a subsequent series
was reviewed, but I thought I'd insert a bit of history trivia:

I first submitted this optimization in late 2017 as an RFC at
https://lore.kernel.org/git/20171110222156.23221-9-newren@gmail.com/.
Unfortunately I had only handled the "for the most part" piece, not
the other special cases.  I knew not handling those other cases caused
bugs, but didn't readily find a solution for them at the time.  I kept
mulling it over periodically despite being switched onto other
projects; eventually I weaseled my way into being able to work on it
again and with some effort was able to work out the other necessary
conditions and audit the code to verify I had all the cases covered.
Those "other cases" were much more easily expressed in the context of
merge-ort's data structures than merge-recursive's, in part because
merge-ort's data structures were picked to help me solve this
optimization problem and the various known failing testcases that I
wanted to fix.