Re: [PATCH 1/3] kci-gitlab: Introducing GitLab-CI Pipeline for Kernel Testing

[Nicolas Dufresne <nicolas.dufresne@collabora.com>]

Hi Tim,

just replying below to one of your comment which I happen to be involved in, but
I'll let others reply for the more specific comments.

Le jeudi 29 février 2024 à 02:44 +0000, Bird, Tim a écrit :
> > -----Original Message-----
> > From: Helen Koike <helen.koike@collabora.com>
> > 
> ....
> 
> > Hey all,
> > 
> > You can check the validation of this patchset on:
> >         https://gitlab.collabora.com/koike/linux/-/pipelines/87035
> > 
> > I would appreciate your feedback on this work, what do you think?
> > 
> > If you would rate from 0 to 5, where:
> > 
> > [ ] 0. I don't think this is useful at all, and I doubt it will ever be. It doesn't seem worthwhile.
> > [ ] 1. I don't find it useful in its current form.
> > [ ] 2. It might be useful to others, but not for me.
> > [ ] 3. It has potential, but it's not yet something I can incorporate into my workflow.
> > [ ] 4. This is useful, but it needs some adjustments before I can include it in my workflow.
> > [ ] 5. This is really useful! I'm eager to start using it right away. Why didn't you send this earlier? :)
> > 
> > Which rating would you select?
> 
> For me, this is a "5".  I don't currently use gitlab, but I might install it just to test this out.
> 
> It looks like there are a large number of YAML files which define the integration between the
> test scripts and gitlab.  Also, there are a number of shell scripts to perform some of the setup
> and tests.  Do you have any idea how difficult it would be to use the shell scripts outside of
> the CI system (e.g. manually)?  That is, are there dependencies between the CI system
> and the shell scripts?

You are effectively the second person I'm aware to provide similar feedback. We
agreed to conduct an effort to remove the gitlab specifics from the script.
Avoid using gitlab CI shell environment in favour of command line arguments.
Also ensure scripts have a "-h" option. This should ease local reproduction and
allow for other CI integration. After all, the Linux kernel is a large community
and gitlab is just one option for managing CI. It is a big system, so we rarely
"just install it" ourself. DRM and Linux Media community are using the
Freedesktop instance, sharing resources and cost within that instance. In Linux
Media we are developing out of tree scripts with similar purpose, but we also
believe a common set of tool, directly in the kernel tree would be a better long
term solution.

> 
> I think the convention, of putting this kind of stuff under a "ci" directory, makes sense.
> And it appears that the sub-dir structure allows for other CI systems to add their
> own config and files also.

CI scripts have the particularity of being very granular, which is very unlike
what a human user would prefer. But when CI fails, you really want to know which
small step failed, which can sometimes be hidden by more en-globing scripts. We
also care a lot about parallelism, since we have hundreds of runners available
to execute these tests.

Short answer, I also like that this is under a CI directory, its makes ensure
the purpose and intention of this work is clear.

regards,
Nicolas