How to test device driver?

How to test device driver?

[Hyeonggon Yoo]

Hello, there are lots of drivers in linux.

My questions are:

  - how can we test driver? is it just using that device for a long
    time as a user?

  - what to do if there's a bug (like syzbot report), fixed it,
    but if we don't have that device?

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to test device driver?

[jim.cromie]

On Mon, May 17, 2021 at 9:28 AM Hyeonggon Yoo <42.hyeyoo@gmail.com> wrote:
>
> Hello, there are lots of drivers in linux.
>
> My questions are:
>
>   - how can we test driver? is it just using that device for a long
>     time as a user?
>
testing is hard.  Having the device is highly recommended.
ISTM essential if adding a Tested-by: tag
Do you have a particular device in mind ?


>   - what to do if there's a bug (like syzbot report), fixed it,
>     but if we don't have that device?
>

You could reply to that report email, with your patch included.
0day/lkp robot watches many email lists, and might test your patch
automatically, for you on different architectures

at least Ive seen it happening on kbuild, kbuild-all lists


> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to test device driver?

[Hyeonggon Yoo]

On Mon, May 17, 2021 at 12:48:38PM -0600, jim.cromie@gmail.com wrote:
> On Mon, May 17, 2021 at 9:28 AM Hyeonggon Yoo <42.hyeyoo@gmail.com> wrote:
> >
> > Hello, there are lots of drivers in linux.
> >
> > My questions are:
> >
> >   - how can we test driver? is it just using that device for a long
> >     time as a user?
> >
> testing is hard.  Having the device is highly recommended.
> ISTM essential if adding a Tested-by: tag
> Do you have a particular device in mind ?

I recently was tracking syzbot report that is occured in
sound driver. I don't have the device, but was curious about
what the test method is.

> You could reply to that report email, with your patch included.
> 0day/lkp robot watches many email lists, and might test your patch
> automatically, for you on different architectures
>
> at least Ive seen it happening on kbuild, kbuild-all lists

well, build tests are useful but it is not enough, we should run the
code on actual device. is it possible to make the bot test the device
(which I don't have) even if there is no syzbot reproducer?

I think I need to learn about bots more..

Thanks,
Hyeongogn Yoo


_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to test device driver?

[jim.cromie]

On Mon, May 17, 2021 at 5:54 PM Hyeonggon Yoo <42.hyeyoo@gmail.com> wrote:
>
> On Mon, May 17, 2021 at 12:48:38PM -0600, jim.cromie@gmail.com wrote:
> > On Mon, May 17, 2021 at 9:28 AM Hyeonggon Yoo <42.hyeyoo@gmail.com> wrote:
> > >
> > > Hello, there are lots of drivers in linux.
> > >
> > > My questions are:
> > >
> > >   - how can we test driver? is it just using that device for a long
> > >     time as a user?
> > >
> > testing is hard.  Having the device is highly recommended.
> > ISTM essential if adding a Tested-by: tag
> > Do you have a particular device in mind ?
>
> I recently was tracking syzbot report that is occured in
> sound driver. I don't have the device, but was curious about
> what the test method is.

well, it surely depends on the bug - and whether its reproducible.

if you could defend the patch as "obviously correct"
you could reasonably send it without extensive testing,
case by case Id guess.

>
> > You could reply to that report email, with your patch included.
> > 0day/lkp robot watches many email lists, and might test your patch
> > automatically, for you on different architectures
> >
> > at least Ive seen it happening on kbuild, kbuild-all lists
>
> well, build tests are useful but it is not enough, we should run the
> code on actual device. is it possible to make the bot test the device
> (which I don't have) even if there is no syzbot reproducer?
>

its possible / practical to disqualify patches robotically,
but to bless them takes a priest.
Robots win by prequalifying submissions.

> I think I need to learn about bots more..
>

Now think about how its a real-life turing test.

0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
_______________________________________________
kbuild-all mailing list -- kbuild-all@lists.01.org
To unsubscribe send an email to kbuild-all-leave@lists.01.org

theres also a wiki, which helped me make sense of things
a bit better than the torrent of reports

> Thanks,
> Hyeongogn Yoo
>

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to test device driver?

[Valdis Klētnieks]

[-- Attachment #1.1: Type: text/plain, Size: 871 bytes --]

On Tue, 18 May 2021 08:53:58 +0900, Hyeonggon Yoo said:

> I recently was tracking syzbot report that is occured in
> sound driver. I don't have the device, but was curious about
> what the test method is.

Depends what the bug was.

> well, build tests are useful but it is not enough, we should run the
> code on actual device. is it possible to make the bot test the device
> (which I don't have) even if there is no syzbot reproducer?

Bugs that are obvious logic errors (use after free, locking issues, failure to
build because a previous commit didn't cover all cases, and so on)
can often be either tested or proven correct without the actual device
being available.  I've lost count how many times I've reported that some
device driver or other simply doesn't even *compile* because an API was
changed but incomplete - stuff that a 'make allmodconfig' will find.

[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to test device driver?

[Jeffrey Walton]

On Mon, May 17, 2021 at 11:28 AM Hyeonggon Yoo <42.hyeyoo@gmail.com> wrote:
> ...
>   - what to do if there's a bug (like syzbot report), fixed it,
>     but if we don't have that device?

You could also ping the mailing list and ask if someone has the
hardware available for testing.

I make most of my machines available over SSH and I am happy to lend
them to free software developers.

I find the problem with testing is the developer. Some package will
have a bug on a particular platform. I offer them access to the
machine to reproduce it, but they don't take the offer. I find GNU
developers are notorious for it.

Jeff

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies