Re: [PATCH v1 0/2] add additional config settings for merge

From: Elijah Newren <newren@gmail.com>
To: Ben Peart <Ben.Peart@microsoft.com>
Cc: "git@vger.kernel.org" <git@vger.kernel.org>,
	"peff@peff.net" <peff@peff.net>,
	"gitster@pobox.com" <gitster@pobox.com>,
	"pclouds@gmail.com" <pclouds@gmail.com>,
	"vmiklos@frugalware.org" <vmiklos@frugalware.org>,
	Kevin Willford <kewillf@microsoft.com>,
	"Johannes.Schindelin@gmx.de" <Johannes.Schindelin@gmx.de>
Subject: Re: [PATCH v1 0/2] add additional config settings for merge
Date: Fri, 20 Apr 2018 10:34:48 -0700	[thread overview]
Message-ID: <CABPp-BEwwn+NwOEtWOKOdUKxoXfq6YwWeoH6OwkPjSwVtTm5=Q@mail.gmail.com> (raw)
In-Reply-To: <20180420133632.17580-1-benpeart@microsoft.com>

On Fri, Apr 20, 2018 at 6:36 AM, Ben Peart <Ben.Peart@microsoft.com> wrote:
> This enables the user to set a couple of additional options for merge.
>
> 1. merge.aggressive - this is to try to resolve a few more trivial
>    merge cases.  It is documented in read-tree and is not something you
>    can pass into merge itself.
>
> 2. merge.renames - this is to save git from having to go through the entire
>    3 trees to see if there were any renames that happened.
>
> For the work item repro that I have been using this drops the merge time
> from ~1 hour to ~5 minutes and the unmerged entries goes down from
> ~40,000 to 1.

Ooh, this is *very* interesting.  Is there any chance I could also get
you to test performing the same merge with the version of git at
https://github.com/newren/git/tree/big-repo-small-cherry-pick and
report on your timings?

The 'big-repo-small-cherry-pick' name could be improved, but that
branch has a number of performance fixes for really poor rename
detection performance during merges.  From your description, I'm
pretty sure it'll apply to your case.  For my specific testcase,  I
got a speedup factor of 30.  Someone else on the list saw a factor of
24[1].  Results are highly dependent on the specific repo, but it's
certainly possible that it gets much of your factor of 12 speedup that
you saw with these new config settings you added.

However, what makes this case even more interesting to me is that my
branch may not be quite as effective as your workarounds.  There are
other other performance issues in merge that I am aware of, but for
which I haven't had the time to write the patches yet (I've been
waiting for the directory rename detection stuff to land and settle
down before working more on the performance aspects).  I do not know
how big a factor those other performance issues are, but your
workarounds (namely the aggressive setting) may get around some of
those other issues as well, so I'm very interested to see how my
current branch compares to the speedups you got with these settings.

Thanks,
Elijah

[1] https://public-inbox.org/git/alpine.DEB.2.00.1711211303290.20686@ds9.cixit.se/