Re: [kernelci] Testing with distros

["Tomeu Vizoso" <tomeu.vizoso@collabora.com>]

On 06/28/2018 01:05 AM, Kevin Hilman wrote:
> "Guillaume Tucker" <guillaume.tucker@gmail.com> writes:
> 
>> One subject that comes up regularly is about testing distributions
>> against upstream kernels in KernelCI.  We've been historically using a
>> bare buildroot user-space to do boot testing and run very low-level test
>> cases.  We're now starting to produce Debian root file systems for more
>> advanced test plans and extend the coverage of kernel-user interfaces
>> (DRM, V4L2...).
>>
>> It seems like there would also be some value in having tests that
>> capture distro user-space breakages due to kernel changes.  This may
>> mean building upstream kernels with a distro config, booting with a
>> fixed stable user-space from that distro and detecting new failures.
>>
>> The next part to be defined would be what tests to run with these
>> distros, as even booting means choosing which services to start etc...
>> The point being to increase kernel test coverage without ending up
>> testing the user-space itself as this would seem outside of the scope of
>> KernelCI.
>>
>> Does this sound like a path worth exploring?
> 
> It definitely sounds like a path worth exploring, but personally, I'd
> rather get our kernel-focused testing (and reporting!) figured out using
> our debian baseline before tackling distros.

I would prefer if distros ran mainline kernels in their own CI, then 
reported the bugs upstream after triage. And ideally, added tests against 
the kernel ABI to kernelci.

Regards,

Tomeu

> In particular, I think a kselftest and/or LTP testplan should probably
> be next on the list.
> 
> Kevin
> 
> 
>