* Testing with distros
@ 2018-06-22 12:34 Guillaume Tucker
2018-06-22 18:35 ` getting started? micah.parrish
2018-06-27 23:05 ` [kernelci] Testing with distros Kevin Hilman
0 siblings, 2 replies; 8+ messages in thread
From: Guillaume Tucker @ 2018-06-22 12:34 UTC (permalink / raw)
To: kernelci
Hi,
One subject that comes up regularly is about testing distributions
against upstream kernels in KernelCI. We've been historically using a
bare buildroot user-space to do boot testing and run very low-level test
cases. We're now starting to produce Debian root file systems for more
advanced test plans and extend the coverage of kernel-user interfaces
(DRM, V4L2...).
It seems like there would also be some value in having tests that
capture distro user-space breakages due to kernel changes. This may
mean building upstream kernels with a distro config, booting with a
fixed stable user-space from that distro and detecting new failures.
The next part to be defined would be what tests to run with these
distros, as even booting means choosing which services to start etc...
The point being to increase kernel test coverage without ending up
testing the user-space itself as this would seem outside of the scope of
KernelCI.
Does this sound like a path worth exploring?
Best wishes,
Guillaume
^ permalink raw reply [flat|nested] 8+ messages in thread
* getting started?
2018-06-22 12:34 Testing with distros Guillaume Tucker
@ 2018-06-22 18:35 ` micah.parrish
2018-06-28 13:11 ` [kernelci] " Matt Hart
2018-06-27 23:05 ` [kernelci] Testing with distros Kevin Hilman
1 sibling, 1 reply; 8+ messages in thread
From: micah.parrish @ 2018-06-22 18:35 UTC (permalink / raw)
To: kernelci
I really like what you've done with kernelci.org, and I'm interested in
running it in a semi-isolated x86 test lab. Do you have a recipe I can
follow for doing this? It seems like there are a lot of moving parts,
but I don't quite see how they all fit together yet.
Regards,
Micah Parrish
Linux OS Engineer
Hewlett-Packard Enterprise
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [kernelci] Testing with distros
2018-06-22 12:34 Testing with distros Guillaume Tucker
2018-06-22 18:35 ` getting started? micah.parrish
@ 2018-06-27 23:05 ` Kevin Hilman
2018-06-28 13:13 ` Matt Hart
2018-07-02 8:07 ` Tomeu Vizoso
1 sibling, 2 replies; 8+ messages in thread
From: Kevin Hilman @ 2018-06-27 23:05 UTC (permalink / raw)
To: Guillaume Tucker; +Cc: kernelci
"Guillaume Tucker" <guillaume.tucker@gmail.com> writes:
> One subject that comes up regularly is about testing distributions
> against upstream kernels in KernelCI. We've been historically using a
> bare buildroot user-space to do boot testing and run very low-level test
> cases. We're now starting to produce Debian root file systems for more
> advanced test plans and extend the coverage of kernel-user interfaces
> (DRM, V4L2...).
>
> It seems like there would also be some value in having tests that
> capture distro user-space breakages due to kernel changes. This may
> mean building upstream kernels with a distro config, booting with a
> fixed stable user-space from that distro and detecting new failures.
>
> The next part to be defined would be what tests to run with these
> distros, as even booting means choosing which services to start etc...
> The point being to increase kernel test coverage without ending up
> testing the user-space itself as this would seem outside of the scope of
> KernelCI.
>
> Does this sound like a path worth exploring?
It definitely sounds like a path worth exploring, but personally, I'd
rather get our kernel-focused testing (and reporting!) figured out using
our debian baseline before tackling distros.
In particular, I think a kselftest and/or LTP testplan should probably
be next on the list.
Kevin
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [kernelci] getting started?
2018-06-22 18:35 ` getting started? micah.parrish
@ 2018-06-28 13:11 ` Matt Hart
2018-06-28 17:53 ` Micah Parrish
0 siblings, 1 reply; 8+ messages in thread
From: Matt Hart @ 2018-06-28 13:11 UTC (permalink / raw)
To: kernelci
[-- Attachment #1: Type: text/plain, Size: 713 bytes --]
Hi Micah
There are indeed a lot of moving parts. You could run your own web UI and
API fairly easily but there's no guides to setup the build system and the
boot testing (LAVA) lab.
What are you trying to achieve? Could it be with KernelCI rather than your
own setup?
Matt
On 22 June 2018 at 19:35, Micah Parrish <micah.parrish@hpe.com> wrote:
> I really like what you've done with kernelci.org, and I'm interested in
> running it in a semi-isolated x86 test lab. Do you have a recipe I can
> follow for doing this? It seems like there are a lot of moving parts, but
> I don't quite see how they all fit together yet.
>
> Regards,
>
> Micah Parrish
> Linux OS Engineer
> Hewlett-Packard Enterprise
>
>
>
>
[-- Attachment #2: Type: text/html, Size: 1178 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [kernelci] Testing with distros
2018-06-27 23:05 ` [kernelci] Testing with distros Kevin Hilman
@ 2018-06-28 13:13 ` Matt Hart
2018-07-02 8:07 ` Tomeu Vizoso
1 sibling, 0 replies; 8+ messages in thread
From: Matt Hart @ 2018-06-28 13:13 UTC (permalink / raw)
To: kernelci; +Cc: Guillaume Tucker
[-- Attachment #1: Type: text/plain, Size: 1613 bytes --]
On 28 June 2018 at 00:05, Kevin Hilman <khilman@baylibre.com> wrote:
> "Guillaume Tucker" <guillaume.tucker@gmail.com> writes:
>
> > One subject that comes up regularly is about testing distributions
> > against upstream kernels in KernelCI. We've been historically using a
> > bare buildroot user-space to do boot testing and run very low-level test
> > cases. We're now starting to produce Debian root file systems for more
> > advanced test plans and extend the coverage of kernel-user interfaces
> > (DRM, V4L2...).
> >
> > It seems like there would also be some value in having tests that
> > capture distro user-space breakages due to kernel changes. This may
> > mean building upstream kernels with a distro config, booting with a
> > fixed stable user-space from that distro and detecting new failures.
> >
> > The next part to be defined would be what tests to run with these
> > distros, as even booting means choosing which services to start etc...
> > The point being to increase kernel test coverage without ending up
> > testing the user-space itself as this would seem outside of the scope of
> > KernelCI.
> >
> > Does this sound like a path worth exploring?
>
> It definitely sounds like a path worth exploring, but personally, I'd
> rather get our kernel-focused testing (and reporting!) figured out using
> our debian baseline before tackling distros.
>
I'd put distro testing as a long term interest. I like it, but there's a
lot of other things that should come first.
> In particular, I think a kselftest and/or LTP testplan should probably
> be next on the list.
>
> Kevin
>
>
>
>
[-- Attachment #2: Type: text/html, Size: 2376 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [kernelci] getting started?
2018-06-28 13:11 ` [kernelci] " Matt Hart
@ 2018-06-28 17:53 ` Micah Parrish
2018-07-26 22:22 ` Kevin Hilman
0 siblings, 1 reply; 8+ messages in thread
From: Micah Parrish @ 2018-06-28 17:53 UTC (permalink / raw)
To: kernelci
I'm trying to redeploy a mix of prototypes and production systems as a
very low touch "top of tree kernel test" farm. This give us early
warning when an kernel change has affected our HW or FW. Kernelci.org
does almost exactly what I'd like to do, but I need to keep an instance
in-house because of prototype confidentiality requirements. If that
works out I'm open to putting some production servers in the main pool
somehow.
Is there a diagram of how all the parts fit together?
-Micah
On 06/28/2018 07:11 AM, Matt Hart wrote:
> Hi Micah
>
> There are indeed a lot of moving parts. You could run your own web UI
> and API fairly easily but there's no guides to setup the build system
> and the boot testing (LAVA) lab.
> What are you trying to achieve? Could it be with KernelCI rather than
> your own setup?
>
> Matt
>
> On 22 June 2018 at 19:35, Micah Parrish <micah.parrish@hpe.com
> <mailto:micah.parrish@hpe.com>> wrote:
>
> I really like what you've done with kernelci.org
> <http://kernelci.org>, and I'm interested in running it in a
> semi-isolated x86 test lab. Do you have a recipe I can follow for
> doing this? It seems like there are a lot of moving parts, but I
> don't quite see how they all fit together yet.
>
> Regards,
>
> Micah Parrish
> Linux OS Engineer
> Hewlett-Packard Enterprise
>
>
>
>
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [kernelci] Testing with distros
2018-06-27 23:05 ` [kernelci] Testing with distros Kevin Hilman
2018-06-28 13:13 ` Matt Hart
@ 2018-07-02 8:07 ` Tomeu Vizoso
1 sibling, 0 replies; 8+ messages in thread
From: Tomeu Vizoso @ 2018-07-02 8:07 UTC (permalink / raw)
To: kernelci, Guillaume Tucker
On 06/28/2018 01:05 AM, Kevin Hilman wrote:
> "Guillaume Tucker" <guillaume.tucker@gmail.com> writes:
>
>> One subject that comes up regularly is about testing distributions
>> against upstream kernels in KernelCI. We've been historically using a
>> bare buildroot user-space to do boot testing and run very low-level test
>> cases. We're now starting to produce Debian root file systems for more
>> advanced test plans and extend the coverage of kernel-user interfaces
>> (DRM, V4L2...).
>>
>> It seems like there would also be some value in having tests that
>> capture distro user-space breakages due to kernel changes. This may
>> mean building upstream kernels with a distro config, booting with a
>> fixed stable user-space from that distro and detecting new failures.
>>
>> The next part to be defined would be what tests to run with these
>> distros, as even booting means choosing which services to start etc...
>> The point being to increase kernel test coverage without ending up
>> testing the user-space itself as this would seem outside of the scope of
>> KernelCI.
>>
>> Does this sound like a path worth exploring?
>
> It definitely sounds like a path worth exploring, but personally, I'd
> rather get our kernel-focused testing (and reporting!) figured out using
> our debian baseline before tackling distros.
I would prefer if distros ran mainline kernels in their own CI, then
reported the bugs upstream after triage. And ideally, added tests against
the kernel ABI to kernelci.
Regards,
Tomeu
> In particular, I think a kselftest and/or LTP testplan should probably
> be next on the list.
>
> Kevin
>
>
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [kernelci] getting started?
2018-06-28 17:53 ` Micah Parrish
@ 2018-07-26 22:22 ` Kevin Hilman
0 siblings, 0 replies; 8+ messages in thread
From: Kevin Hilman @ 2018-07-26 22:22 UTC (permalink / raw)
To: Micah Parrish; +Cc: kernelci
"Micah Parrish" <micah.parrish@hpe.com> writes:
> I'm trying to redeploy a mix of prototypes and production systems as a
> very low touch "top of tree kernel test" farm. This give us early
> warning when an kernel change has affected our HW or FW. Kernelci.org
> does almost exactly what I'd like to do, but I need to keep an
> instance in-house because of prototype confidentiality requirements.
> If that works out I'm open to putting some production servers in the
> main pool somehow.
>
> Is there a diagram of how all the parts fit together?
The long delay in responding meant that no, there was no diagram. Our
docs are pretty minimal, but we've begun an attempt to improve that
on the wiki for the kernelci-doc project: https://github.com/kernelci/kernelci-doc/wiki
The sub-page for setting up a local instance[1] would be a good start,
and to compliment that, I just whipped up this draw.io diagram[1] that
tries to show graphically how all the kernelCI pieces fit together in a
full CI loop. Most items have links to the various github projects where
the code lives.
I admit that it's very primitive, but it should at least help you get
started seeing how all the parts fit together.
Hope that helps,
Kevin
[1] https://github.com/kernelci/kernelci-doc/wiki/Setting-up-a-local-development-instance
[2] https://drive.google.com/file/d/1x2SXt24ZNzuk3SfBzsQpw_RnLOcF3vWd/view?usp=sharing
> -Micah
>
>
> On 06/28/2018 07:11 AM, Matt Hart wrote:
>> Hi Micah
>>
>> There are indeed a lot of moving parts. You could run your own web
>> UI and API fairly easily but there's no guides to setup the build
>> system and the boot testing (LAVA) lab.
>> What are you trying to achieve? Could it be with KernelCI rather
>> than your own setup?
>>
>> Matt
>>
>> On 22 June 2018 at 19:35, Micah Parrish <micah.parrish@hpe.com
>> <mailto:micah.parrish@hpe.com>> wrote:
>>
>> I really like what you've done with kernelci.org
>> <http://kernelci.org>, and I'm interested in running it in a
>> semi-isolated x86 test lab. Do you have a recipe I can follow for
>> doing this? It seems like there are a lot of moving parts, but I
>> don't quite see how they all fit together yet.
>>
>> Regards,
>>
>> Micah Parrish
>> Linux OS Engineer
>> Hewlett-Packard Enterprise
>>
>>
>>
>>
>>
>
>
>
>
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2018-07-26 22:22 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-22 12:34 Testing with distros Guillaume Tucker
2018-06-22 18:35 ` getting started? micah.parrish
2018-06-28 13:11 ` [kernelci] " Matt Hart
2018-06-28 17:53 ` Micah Parrish
2018-07-26 22:22 ` Kevin Hilman
2018-06-27 23:05 ` [kernelci] Testing with distros Kevin Hilman
2018-06-28 13:13 ` Matt Hart
2018-07-02 8:07 ` Tomeu Vizoso
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.