Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] & [TECH TOPIC] Improve regression tracking

[Carlos O'Donell <carlos@redhat.com>]

On 07/05/2017 10:06 AM, Greg KH wrote:
> I don't mean to poo-poo the idea, but please realize that around 75% of
> the kernel is hardware/arch support, so that means that 75% of the
> changes/fixes deal with hardware things (yes, change is in direct
> correlation to size of the codebase in the tree, strange but true).

We should distinguish between the reviewer reviewing the regression test
and running the regression test. As long as the submitter ran the 
regression test on their hardware, and it passed, the reviewer need only
review the test for logical consistency and correctness?

Lack of test infrastructure was a serious problem for us in glibc. We are
relying on namespaces for more complex network and filesystem testing.
Without namespaces we would have needed a much more complex setup that
might never have seen developer adoption. When I attended LPC 2016 I
prioritized listening in on namespaces discussions to make sure nothing
was changing that might break our testing framework.

This conversation is going to lead down the path of driver HAL or
emulation in order to provide regression testing for code above the actual
hardware, and that's another hard problem, but one need not go there.

Starting with real hardware tests can have benefit.

In glibc we test SSE, AVX, AVX512, TSX etc. but if you don't have the
extensions you get a bunch of UNSUPPORTED tests. While upstream kernel 
may have a more limited set of available hardware per-person, the
collective set of developers has hardware to cover all configurations,
and they should run the regression tests for hardware they care about
... and *must* do so if they submit a patch to fix a bug! :-)

-- 
Cheers,
Carlos.