Re: DRM Format Modifiers in v4l2

From: Daniel Vetter <daniel@ffwll.ch>
To: Nicolas Dufresne <nicolas@ndufresne.ca>
Cc: Brian Starkey <brian.starkey@arm.com>,
	Hans Verkuil <hverkuil@xs4all.nl>,
	"linux-media@vger.kernel.org" <linux-media@vger.kernel.org>,
	jonathan.chai@arm.com,
	Laurent Pinchart <laurent.pinchart@ideasonboard.com>,
	dri-devel <dri-devel@lists.freedesktop.org>
Subject: Re: DRM Format Modifiers in v4l2
Date: Mon, 28 Aug 2017 22:49:07 +0200	[thread overview]
Message-ID: <CAKMK7uGaQ+9cZ2PyLkwC06Qpch3AK+Tkr4SZFZVLfUqUFKyygQ@mail.gmail.com> (raw)
In-Reply-To: <1503943642.3316.7.camel@ndufresne.ca>

On Mon, Aug 28, 2017 at 8:07 PM, Nicolas Dufresne <nicolas@ndufresne.ca> wrote:
> Le jeudi 24 août 2017 à 13:26 +0100, Brian Starkey a écrit :
>> > What I mean was: an application can use the modifier to give buffers from
>> > one device to another without needing to understand it.
>> >
>> > But a generic video capture application that processes the video itself
>> > cannot be expected to know about the modifiers. It's a custom HW specific
>> > format that you only use between two HW devices or with software written
>> > for that hardware.
>> >
>>
>> Yes, makes sense.
>>
>> > >
>> > > However, in DRM the API lets you get the supported formats for each
>> > > modifier as-well-as the modifier list itself. I'm not sure how exactly
>> > > to provide that in a control.
>> >
>> > We have support for a 'menu' of 64 bit integers: V4L2_CTRL_TYPE_INTEGER_MENU.
>> > You use VIDIOC_QUERYMENU to enumerate the available modifiers.
>> >
>> > So enumerating these modifiers would work out-of-the-box.
>>
>> Right. So I guess the supported set of formats could be somehow
>> enumerated in the menu item string. In DRM the pairs are (modifier +
>> bitmask) where bits represent formats in the supported formats list
>> (commit db1689aa61bd in drm-next). Printing a hex representation of
>> the bitmask would be functional but I concede not very pretty.
>
> The problem is that the list of modifiers depends on the format
> selected. Having to call S_FMT to obtain this list is quite
> inefficient.
>
> Also, be aware that DRM_FORMAT_MOD_SAMSUNG_64_32_TILE modifier has been
> implemented in V4L2 with a direct format (V4L2_PIX_FMT_NV12MT). I think
> an other one made it the same way recently, something from Mediatek if
> I remember. Though, unlike the Intel one, the same modifier does not
> have various result depending on the hardware revision.

Note on the intel modifers: On most recent platforms (iirc gen9) the
modifier is well defined and always describes the same byte layout. We
simply didn't want to rewrite our entire software stack for all the
old gunk platforms, hence the language. I guess we could/should
describe the layout in detail, but atm we're the only ones using it.

On your topic of v4l2 encoding the drm fourcc+modifier combo into a
special v4l fourcc: That's exactly the mismatch I was thinking of.
There's other examples of v4l2 fourcc being more specific than their
drm counters (e.g. specific way the different planes are laid out).
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch