[Linux-kernel-mentees] On test-media script integration for vidtv

[Linux-kernel-mentees] On test-media script integration for vidtv

[Daniel W. S. Almeida]

Hi Hans & Mauro.

Well I now have some more time to work on vidtv so I'd like to start off 
with the test-media script integration as mentioned by Hans some time ago.

Let's recap:



> For regression testing of vidtv during the daily build it would be great if
> the contrib/test/test-media script can be enhanced to include vidtv.
> 
> This is run during the daily build with a kernel that has lockdep and many
> other checks enabled, so it is very helpful to verify that no regressions
> happened.
> 
> Note that this script relies on the /dev/mediaX devices to run the tests. I
> noticed that vidtv doesn't appear to create a /dev/mediaX device, even though
> CONFIG_MEDIA_CONTROLLER_DVB=y. This is definitely something that would be good
> to support in vidtv.
> 
> Regards,
> 
> Hans


So I have been looking at the documentation for a bit, i.e.

> https://www.kernel.org/doc/html/v4.14/media/uapi/mediactl/media-controller-intro.html

What I understood from that is that entities and interfaces under 
/dev/media/ are created to model actual hardware, but vidtv is a virtual 
driver. Can this be clarified?

Is this separate from the files created by the dvb core as part of the 
driver initialization?
e.g. (for a single adapter):

/dev/adapter0/frontend0
/dev/adapter0/demux0
/dev/adapter0/dvr0

Btw I noticed some drivers call dvb_create_media_graph() as part of 
their initialization, e.g. au0828-dvb.c, em28xx-dvb.c, cx231xx-dvb.c, 
but some (most) do not, why is this?


> Daniel, if you look at the test-media script, then you'll see that it has
> separate sections for each virtual driver. It's probably best to look at
> the vim2m driver tests since that's the easiest.
> 
> It loads the module, then it starts v4l2-compliance to test the API. This
> utility basically tries all V4L2 APIs and checks that the driver conforms to
> the spec.
> 
> Note that you see options like '-m platform:vim2m' that selects which /dev/media
> device to use based on the name and v4l2-compliance (or v4l2-ctl with the -z option)
> then walks the topology of the media device and tests all interfaces it finds.
> 
> Hence my question about media controller support in vidtv: this should be
> supported there as well since it allows you to write these test sequences without
> having to know which /dev/fooX device should be used.
> 
> After the compliance test is run the script tests unbind/bind (always a nasty test)
> and checks for memory leaks (if enabled in the kernel).
> 
> Much of this test sequence can be copied for vidtv, but you need something else for
> the compliance test. It would for now be enough to have some quick check with the
> existing dvb utils from v4l-utils to see that the basics work.
> 
> IMHO I think a dtv-compliance utility along the lines of v4l2-compliance and
> cec-compliance should be created.
> 
> I'm actually wondering whether the dtv compliance tests shouldn't be part of
> v4l2-compliance (which would have to be renamed to media-compliance in that case)
> since there are hybrid drivers supporting both in the same media topology.
> 
> This would make compliance tests possible where analog/digital TV mode handling
> is tested (i.e. if analog TV is in use, then trying to use digital TV would return
> EBUSY and vice versa).
> 
> It would require some work in v4l2-compliance.cpp to make this possible, but I
> can do that myself.
> 
> Compliance tests have proven to be a great method of testing for regressions and
> testing drivers in general.
> 
> Note that it takes a lot of time to create good compliance tests, but you just start
> with some simple tests and expand it over time.
> 
> Regards,
> 
> Hans


Well, I checked the test-media script and it seems to be easy to add 
support for vidtv there, as Hans said most of the sequence there can be 
copied from the other drivers. However..

 >It would for now be enough to have some quick check with the existing 
dvb utils from v4l-utils to see that the basics work.

Can we discuss this in further depth please? Maybe a few examples?

What I currently do is:

1. zap the stream & open it in dvbinspector, manually check for mistakes
2. manually check dmesg for errors
3. maybe play the stream using vlc or Kaffeine

-- thanks
-- Daniel
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] On test-media script integration for vidtv

[Hans Verkuil]

On 09/11/2020 12:54, Daniel W. S. Almeida wrote:
> Hi Hans & Mauro.
> 
> Well I now have some more time to work on vidtv so I'd like to start off 
> with the test-media script integration as mentioned by Hans some time ago.
> 
> Let's recap:
> 
> 
> 
>> For regression testing of vidtv during the daily build it would be great if
>> the contrib/test/test-media script can be enhanced to include vidtv.
>>
>> This is run during the daily build with a kernel that has lockdep and many
>> other checks enabled, so it is very helpful to verify that no regressions
>> happened.
>>
>> Note that this script relies on the /dev/mediaX devices to run the tests. I
>> noticed that vidtv doesn't appear to create a /dev/mediaX device, even though
>> CONFIG_MEDIA_CONTROLLER_DVB=y. This is definitely something that would be good
>> to support in vidtv.
>>
>> Regards,
>>
>> Hans
> 
> 
> So I have been looking at the documentation for a bit, i.e.
> 
>> https://www.kernel.org/doc/html/v4.14/media/uapi/mediactl/media-controller-intro.html
> 
> What I understood from that is that entities and interfaces under 
> /dev/media/ are created to model actual hardware, but vidtv is a virtual 
> driver. Can this be clarified?

For virtual drivers it models the virtual hardware. Userspace doesn't know
that it is actually all emulated, it just looks like real hardware.

> 
> Is this separate from the files created by the dvb core as part of the 
> driver initialization?
> e.g. (for a single adapter):
> 
> /dev/adapter0/frontend0
> /dev/adapter0/demux0
> /dev/adapter0/dvr0

Yes, it is separate. If CONFIG_MEDIA_CONTROLLER_DVB=y then an additional
device node /dev/media0 is created.

> 
> Btw I noticed some drivers call dvb_create_media_graph() as part of 
> their initialization, e.g. au0828-dvb.c, em28xx-dvb.c, cx231xx-dvb.c, 
> but some (most) do not, why is this?

I think that's because only a subset of DVB drivers support the media controller.

> 
> 
>> Daniel, if you look at the test-media script, then you'll see that it has
>> separate sections for each virtual driver. It's probably best to look at
>> the vim2m driver tests since that's the easiest.
>>
>> It loads the module, then it starts v4l2-compliance to test the API. This
>> utility basically tries all V4L2 APIs and checks that the driver conforms to
>> the spec.
>>
>> Note that you see options like '-m platform:vim2m' that selects which /dev/media
>> device to use based on the name and v4l2-compliance (or v4l2-ctl with the -z option)
>> then walks the topology of the media device and tests all interfaces it finds.
>>
>> Hence my question about media controller support in vidtv: this should be
>> supported there as well since it allows you to write these test sequences without
>> having to know which /dev/fooX device should be used.
>>
>> After the compliance test is run the script tests unbind/bind (always a nasty test)
>> and checks for memory leaks (if enabled in the kernel).
>>
>> Much of this test sequence can be copied for vidtv, but you need something else for
>> the compliance test. It would for now be enough to have some quick check with the
>> existing dvb utils from v4l-utils to see that the basics work.
>>
>> IMHO I think a dtv-compliance utility along the lines of v4l2-compliance and
>> cec-compliance should be created.
>>
>> I'm actually wondering whether the dtv compliance tests shouldn't be part of
>> v4l2-compliance (which would have to be renamed to media-compliance in that case)
>> since there are hybrid drivers supporting both in the same media topology.
>>
>> This would make compliance tests possible where analog/digital TV mode handling
>> is tested (i.e. if analog TV is in use, then trying to use digital TV would return
>> EBUSY and vice versa).
>>
>> It would require some work in v4l2-compliance.cpp to make this possible, but I
>> can do that myself.
>>
>> Compliance tests have proven to be a great method of testing for regressions and
>> testing drivers in general.
>>
>> Note that it takes a lot of time to create good compliance tests, but you just start
>> with some simple tests and expand it over time.
>>
>> Regards,
>>
>> Hans
> 
> 
> Well, I checked the test-media script and it seems to be easy to add 
> support for vidtv there, as Hans said most of the sequence there can be 
> copied from the other drivers. However..
> 
>  >It would for now be enough to have some quick check with the existing 
> dvb utils from v4l-utils to see that the basics work.
> 
> Can we discuss this in further depth please? Maybe a few examples?
> 
> What I currently do is:
> 
> 1. zap the stream & open it in dvbinspector, manually check for mistakes
> 2. manually check dmesg for errors
> 3. maybe play the stream using vlc or Kaffeine

I'm not a DVB expert, but I was thinking of testing if it can discover
channels, and perhaps just a test to stream data without checking if it is
actually valid video.

The problem is that I am not all that familiar with the DVB utilities so I
don't know if anything can be automated.

An alternative is to start adding dtv support to v4l2-compliance. That utility
already has media controller support, but it needs a bit of work to comfortably
integrate DTV support as well. I can help with that if you want to go in that
direction. v4l2-compliance would need a bit of refactoring, but it is nice to
have the tests there since that makes it easy to support hybrid devices with a
single compliance utility.

Regards,

	Hans
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Re: [Linux-kernel-mentees] On test-media script integration for vidtv

[Daniel W. S. Almeida]

Hi Hans & Mauro.

So I sent a few patches today implementing some of what was discussed
here. I added support for modprobe/rmmod, bind/unbind and kmemleak
results at the test-media script by basically adapting the
implementation for vim2m.

The initial compliance test part is missing, of course.

> I'm not a DVB expert, but I was thinking of testing if it can discover
> channels, and perhaps just a test to stream data without checking if
> it is
> actually valid video.
> 
> The problem is that I am not all that familiar with the DVB utilities
> so I
> don't know if anything can be automated.

Mauro, can you chime in on the above please? 

Some of the last changes for vidtv broke some very minor details in the
driver. This meant that some of the tables would not get picked up by
userspace software, even though the driver itself did not crash.

I wonder whether this sort of failure can be picked up in some automated way?

> An alternative is to start adding dtv support to v4l2-compliance. That utility
> already has media controller support, but it needs a bit of work to comfortably
> integrate DTV support as well. I can help with that if you want to go
> in that
> direction. v4l2-compliance would need a bit of refactoring, but it is
> nice to
> have the tests there since that makes it easy to support hybrid
> devices with a
> single compliance utility.

I am still getting acquainted with this tool, but otherwise I am fine
with that.

Also, as let's recap this from a past email from Mauro:

> I suspect that, before that (or together with such tooling), we need
> to properly implement the frontend ioctl, validating the per delivery
> system parameters, as, right now, it just accepts anything from
> userspace. 

Mauro, can you give me some tips on how to get started? I assume there's
some technical documentation detailing what would be considered valid
values for the different delivery systems in use today? 

Also, where should I add the checks? Is it at
"vidtv_demod_set_frontend()" ?

-- Daniel
_______________________________________________
Linux-kernel-mentees mailing list
Linux-kernel-mentees@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees