Re: [PATCH v2 0/7] Speed up mirror-fetches with many refs

From: Junio C Hamano <gitster@pobox.com>
To: Patrick Steinhardt <ps@pks.im>
Cc: git@vger.kernel.org, "Jeff King" <peff@peff.net>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
	"Derrick Stolee" <stolee@gmail.com>,
	"René Scharfe" <l.s.r@web.de>
Subject: Re: [PATCH v2 0/7] Speed up mirror-fetches with many refs
Date: Tue, 24 Aug 2021 15:48:19 -0700	[thread overview]
Message-ID: <xmqqfsuylarw.fsf@gitster.g> (raw)
In-Reply-To: cover.1629800774.git.ps@pks.im

Patrick Steinhardt <ps@pks.im> writes:

> this is the second version of my patch series to speed up mirror-fetches
> with many refs. This topic applies on top of Junio's 9d5700f60b (Merge
> branch 'ps/connectivity-optim' into jch, 2021-08-23).

It is a horrible commit to base anything on.  You are taking your
patches hostage to all of these other topics.

    9d5700f60b Merge branch 'ps/connectivity-optim' into jch
    7ad315de2f Merge branch 'js/log-protocol-version' into jch
    1726f748f5 Merge branch 'en/ort-becomes-the-default' into jch
    23aeecb099 Merge branch 'en/merge-strategy-docs' into jch
    568277d458 Merge branch 'en/pull-conflicting-options' into jch
    2b316bb006 ### match next
    4efa9ea0b6 Merge branch 'ps/fetch-pack-load-refs-optim' into jch
    b305842ee8 Merge branch 'jt/push-negotiation-fixes' into jch
    83b45616f1 Merge branch 'es/trace2-log-parent-process-name' into jch
    be89aa8c38 Merge branch 'hn/refs-test-cleanup' into jch
    256d56ed32 Merge branch 'en/ort-perf-batch-15' into jch
    7477fbf53a Merge branch 'js/expand-runtime-prefix' into jch
    b1453dfd30 Merge branch 'ab/bundle-doc' into jch
    1b66e8e89d Merge branch 'zh/ref-filter-raw-data' into jch
    1fbf27ddcd Merge branch 'ab/pack-stdin-packs-fix' into jch
    dcf57bfebb Merge branch 'ab/http-drop-old-curl' into jch
    93041f7c57 Merge branch 'ds/add-with-sparse-index' into jch
    814a016195 Merge branch 'jc/bisect-sans-show-branch' into jch

A better way to handle a situation like this is to limit your
dependencies more explicitly.  If you look at what I did to the last
round of this topic, you'll see that there is a merge of the
'ps/connectivity-optim' topic into v2.33 followed by application of
the patches, like this:

    1d576ca7b2 fetch: avoid second connectivity check if we already have all objects
    6768595f10 fetch: refactor fetch refs to be more extendable
    a615d7cf87 fetch-pack: optimize loading of refs via commit graph
    bfd04fc24c connected: refactor iterator to return next object ID directly
    1a387c9f3a fetch: avoid unpacking headers in object existence check
    f1a4367ec4 fetch: speed up lookup of want refs via commit-graph
    3628199d4d Merge branch 'ps/connectivity-optim' into ps/fetch-optim

What I did to your last round was to merge 'ps/connectivity-optim'
on top of v2.33 and then queue them.  You can do the same for this
round (you can tell people "apply these on top of the result of
merging topic X, Y and Z on tag V").

    df52ef2c3a fetch: avoid second connectivity check if we already have all objects
    c1721680e4 fetch: merge fetching and consuming refs
    5470cbe1be fetch: refactor fetch refs to be more extendable
    016a510428 fetch-pack: optimize loading of refs via commit graph
    f6c7e63cc7 connected: refactor iterator to return next object ID directly
    17c8e90df3 fetch: avoid unpacking headers in object existence check
    a54c245004 fetch: speed up lookup of want refs via commit-graph
    3628199d4d Merge branch 'ps/connectivity-optim' into ps/fetch-optim

I had to adjust [4/7] while applying them on top of the same
3628199d4d I created for queuing the previous round, and it would be
appreciated if you can double-check the result.

Thanks.