Re: [GSOC] How to improve the performance of git cat-file --batch

From: ZheNing Hu <adlternative@gmail.com>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Cc: Git List <git@vger.kernel.org>,
	Junio C Hamano <gitster@pobox.com>,
	Christian Couder <christian.couder@gmail.com>,
	Hariom verma <hariom18599@gmail.com>
Subject: Re: [GSOC] How to improve the performance of git cat-file --batch
Date: Sun, 25 Jul 2021 20:05:12 +0800	[thread overview]
Message-ID: <CAOLTT8Sbusr8=iJbG1qXcSerivZqP5xm-GS8R7TqDMh7QXSZaA@mail.gmail.com> (raw)
In-Reply-To: <87im0zs8wn.fsf@evledraar.gmail.com>

Ævar Arnfjörð Bjarmason <avarab@gmail.com> 于2021年7月25日周日 上午5:23写道：
>
>
> On Sat, Jul 24 2021, ZheNing Hu wrote:
>
> > After reusing ref-filter logic for cat-file --batch, the function of
> > cat-file --batch has been enhanced,
> > but the performance of cat-file is severely degraded. So we need a
> > better solution to solve it.
> > its last version is here:
> > https://lore.kernel.org/git/pull.993.v2.git.1626363626.gitgitgadget@gmail.com/
> >
> > Use google doc to show some of my recent ideas:
> > https://docs.google.com/document/d/1hyM-ecMd8C_TJ3OsOn46nMr8ZxCUxeK7HKj-bRkq3sk/edit?usp=sharing
> >
> > Anyone is welcome to comment and suggest better solutions!
>
> Having looked at that Google Doc it seems to just discuss "Plan A", or
> there some multi-doc magic and there's a "Plan B", or...? I'm not very
> familiar with Google Docs.
>

There may be several plan that have not been written to it for the time being.

> Anyway, I'd encourage you to just send this as an E-Mail to the mailing
> list.
>

Ok, I can migrate them here.

> Having skimmed it I'm a bit confused about this in reference to
> performance generally. I haven't looked into the case you're discussing,
> but as I noted in
> https://lore.kernel.org/git/87im1p6x34.fsf@evledraar.gmail.com/ the
> profiling clearly shows that the main problem is that you've added
> object lookups we skipped before.
>

Yeah, you showed me last time that lookup_object() took up a lot of time.

> I think any plan to improve performance should start with a perf test,
> see which of your patches have performance regressions, if some don't
> and are otherwise OK perhaps those can go in earlier.
>

Makes sense. Last time the test performance of commits is here:
https://lore.kernel.org/git/CAOLTT8RR4+tUuT2yc2PDL9NwCburW8bM_Sht6nhKJ_fYV8fGsQ@mail.gmail.com/

The main commit with performance degradation is "[GSOC] cat-file:
reuse ref-filter logic",
we need to focus on it.

> Then we'll have the smallest possible set of changes that's correct, but
> introduces performance regressions, and can look at what those are doing
> to slow things down...

Ok.

Thanks.
--
ZheNing Hu