Re: [RFC] Request API and V4L2 capabilities

[Tomasz Figa <tfiga@chromium.org>]

On Wed, Aug 22, 2018 at 10:21 PM Paul Kocialkowski
<paul.kocialkowski@bootlin.com> wrote:
>
> Hi,
>
> [...]
>
> On Wed, 2018-08-15 at 14:51 +0200, Maxime Jourdan wrote:
> > Hi Paul, I think we need to go deeper than just exposing the supported
> > profiles/levels and also include a way to query the CAPTURE pixel
> > formats that are supported for each profile.
> >
> > Maybe HEVC Main produces yuv420p but HEVC Main10 gives you
> > yuv420p10le. Maybe H.264 HiP produces NV12 but H.264 Hi422P produces
> > YUYV while also supporting down-sampling to NV12.
>
> Well, I think we're looking at this backwards. Userspace certainly known
> what destination format is relevant for the video, so it shouldn't have
> to query the driver about it except to check that the format is indeed
> supported.

Typically the profile itself only defines the sub-sampling and sample
size, but not the exact set of formats supported by the hardware. VP9
profile 0 is expected to decode into YUV 4:2:0 8-bit, but whether that
would be NV12, YUV420, NV21 or maybe some hw-specific tiled format
(like NV12MT), is entirely up to the hardware/driver. Userspace will
definitely need to know if the decoder can decode the video into a
format, which it can later use (display).

>
> > I don't know the specifics of each platform and the only example I can
> > think of is the Amlogic HEVC decoder that can produce NV12 for Main,
> > but only outputs a compressed proprietary format for Main10.
> >
> > I unfortunately don't have an idea about how to implement that, but
> > I'll think about it.
>
> On the first generations of Allwinner platforms, we also have a vendor-
> specific format as output, that we expose with a dedicated format.
> There's no particular interfacing issue with that. Only that userspace
> has to be aware of the format and how to deal with it.
>

Typically a decode operates as a part of a pipeline with other
components. If your display doesn't let you display format X on an
overlay plane, you may want to use a post-processor hardware to
convert it to format Y. Or maybe use GPU to blit the video into the
primary plane? Or maybe you need to do both, because format X is a
decoder-specific tiled format and the GPU can't deal with it and all
the overlay planes are occupied with some other surfaces? Or maybe
it's just cheaper to do software decode rather than the conversion+GPU
blit? This is something that normally needs to be queried before the
video playback is initialized.

Best regards,
Tomasz