Re: Proposal for a regular upstream performance testing

[Peter Maydell <peter.maydell@linaro.org>]

On Thu, 26 Nov 2020 at 08:13, Lukáš Doktor <ldoktor@redhat.com> wrote:
> The goal of this initiative is to detect system-wide performance
> regressions as well as improvements early, ideally pin-point the
> individual commits and notify people that they should fix things.
> All in upstream and ideally with least human interaction possible.

So, my general view on this is that automated testing of performance
is nice, but unless it also comes with people willing to do the
work of identifying and fixing the causes of performance regressions
there's a risk that it degrades into another automated email or set
of graphs that nobody pays much attention to (outside of the obvious
"oops this commit dropped us by 50%" mistakes). As with fuzz testing,
I'm a bit wary of setting up an automated system that just pushes
work onto humans who already have full plates. Is RedHat also
planning to set performance requirements and assign engineers to
keep us from falling below them ?

thanks
-- PMM