Re: DRM Format Modifiers in v4l2

[Hans Verkuil <hverkuil@xs4all.nl>]

On 08/21/2017 06:01 PM, Daniel Vetter wrote:
> On Mon, Aug 21, 2017 at 5:52 PM, Brian Starkey <brian.starkey@arm.com> wrote:
>> Hi all,
>>
>> I couldn't find this topic talked about elsewhere, but apologies if
>> it's a duplicate - I'll be glad to be steered in the direction of a
>> thread.
>>
>> We'd like to support DRM format modifiers in v4l2 in order to share
>> the description of different (mostly proprietary) buffer formats
>> between e.g. a v4l2 device and a DRM device.
>>
>> DRM format modifiers are defined in include/uapi/drm/drm_fourcc.h and
>> are a vendor-namespaced 64-bit value used to describe various
>> vendor-specific buffer layouts. They are combined with a (DRM) FourCC
>> code to give a complete description of the data contained in a buffer.
>>
>> The same modifier definition is used in the Khronos EGL extension
>> EGL_EXT_image_dma_buf_import_modifiers, and is supported in the
>> Wayland linux-dmabuf protocol.
>>
>>
>> This buffer information could of course be described in the
>> vendor-specific part of V4L2_PIX_FMT_*, but this would duplicate the
>> information already defined in drm_fourcc.h. Additionally, there
>> would be quite a format explosion where a device supports a dozen or
>> more formats, all of which can use one or more different
>> layouts/compression schemes.
>>
>> So, I'm wondering if anyone has views on how/whether this could be
>> incorporated?
>>
>> I spoke briefly about this to Laurent at LPC last year, and he
>> suggested v4l2_control as one approach.
>>
>> I also wondered if could be added in v4l2_pix_format_mplane - looks
>> like there's 8 bytes left before it exceeds the 200 bytes, or could go
>> in the reserved portion of v4l2_plane_pix_format.
>>
>> Thanks for any thoughts,
> 
> One problem is that the modifers sometimes reference the DRM fourcc
> codes. v4l has a different (and incompatible set) of fourcc codes,
> whereas all the protocols and specs (you can add DRI3.1 for Xorg to
> that list btw) use both drm fourcc and drm modifiers.
> 
> This might or might not make this proposal unworkable, but it's
> something I'd at least review carefully.
> 
> Otherwise I think it'd be great if we could have one namespace for all
> modifiers, that's pretty much why we have them. Please also note that
> for drm_fourcc.h we don't require an in-kernel user for a new modifier
> since a bunch of them might need to be allocated just for
> userspace-to-userspace buffer sharing (e.g. in EGL/vk). One example
> for this would be compressed surfaces with fast-clearing, which is
> planned for i915 (but current hw can't scan it out). And we really
> want to have one namespace for everything.

Who sets these modifiers? Kernel or userspace? Or can it be set by both?
I assume any userspace code that sets/reads this is code specific for that
hardware?

I think Laurent's suggestion of using a 64 bit V4L2 control for this makes
the most sense.

Especially if you can assume that whoever sets this knows the hardware.

I think this only makes sense if you pass buffers from one HW device to another.

Because you cannot expect generic video capture code to be able to interpret
all the zillion different combinations of modifiers.

Regards,

	Hans