Re: Plan for hybrid testing

[shuah <shuah@kernel.org>]

Hi Brendan,

On 9/13/19 3:02 PM, Brendan Higgins wrote:
> Hey Knut and Shuah,
> 
> Following up on our offline discussion on Wednesday night:

Awesome. Thanks for doing this.

> 
> We decided that it would make sense for Knut to try to implement Hybrid
> Testing (testing that crosses the kernel userspace boundary) that he
> introduced here[1] on top of the existing KUnit infrastructure.
> 
> We discussed several possible things in the kernel that Knut could test
> with the new Hybrid Testing feature as an initial example. Those were
> (in reverse order of expected difficulty):
> 
> 1. RDS (Reliable Datagram Sockets) - We decided that, although this was
>     one of the more complicated subsystems to work with, it was probably
>     the best candidate for Knut to start with because it was in desperate
>     need of better testing, much of the testing would require crossing
>     the kernel userspace boundary to be effective, and Knut has access to
>     RDS (since he works at Oracle).
> 
> 2. KMOD - Probably much simpler than RDS, and the maintainer, Luis
>     Chamberlain (CC'ed) would like to see better testing here, but
>     probably still not as good as RDS because it is in less dire need of
>     testing, collaboration on this would be more difficult, and Luis is
>     currently on an extended vacation. Luis and I had already been
>     discussing testing KMOD here[2].
> 
> 3. IP over USB - Least desirable option, but still possible. More
>     complicated than KMOD, and not as easy to collaborate on as RDS.
> 
> I don't really think we discussed how this would work. I remember that I
> mentioned that it would be easier if I sent out a patch that
> centralizes where KUnit tests are dispatched from in the kernel; I will
> try to get an RFC for that out, probably sometime next week. That should
> provide a pretty straightforward place for Knut to move his work on top
> of.
> 

That will be awesome.

> The next question is what the userspace component of this should look
> like. To me it seems like we should probably have the kselftest test
> runner manage when the test gets run, and collecting and reporting the
> result of the test, but I think Knut has thought more about this than I,
> and Shuah is the kselftest maintainer, so I am guessing this will
> probably mostly be a discussion between the two of you.
> 

Yes. This is what I have in mind.

> So I think we have a couple of TODOs between us:
> 
> Brendan:
>   - Need to send out patch that provides a single place where all tests
>     are dispatched from.
> 
> Knut:
>   - Start splitting out the hybrid test stuff from the rest of the RFC
>     you sent previously.
> 
> Knut and Shuah:
>   - Start figuring out what the userspace component of this will look
>     like.
> 

Once Knut decides which one of the above options he chooses and sends me
RFC patches, we can start discussing the details based on the RFC.

thanks,
-- Shuah