Re: [RFC PATCH v3] media: docs-rst: Document m2m stateless video decoder interface

[Hans Verkuil <hverkuil@xs4all.nl>]

On 10/19/18 10:09, Alexandre Courbot wrote:
> Thanks everyone for the feedback on v2! I have not replied to all the
> individual emails but hope this v3 will address some of the problems
> raised and become a continuation point for the topics still in
> discussion (probably during the ELCE Media Summit).
> 
> This patch documents the protocol that user-space should follow when
> communicating with stateless video decoders. It is based on the
> following references:
> 
> * The current protocol used by Chromium (converted from config store to
>   request API)
> 
> * The submitted Cedrus VPU driver
> 
> As such, some things may not be entirely consistent with the current
> state of drivers, so it would be great if all stakeholders could point
> out these inconsistencies. :)
> 
> This patch is supposed to be applied on top of the Request API V18 as
> well as the memory-to-memory video decoder interface series by Tomasz
> Figa.
> 
> Changes since v2:
> 
> * Specify that the frame header controls should be set prior to
>   enumerating the CAPTURE queue, instead of the profile which as Paul
>   and Tomasz pointed out is not enough to know which raw formats will be
>   usable.
> * Change V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM to
>   V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS.
> * Various rewording and rephrasing
> 
> Two points being currently discussed have not been changed in this
> revision due to lack of better idea. Of course this is open to change:
> 
> * The restriction of having to send full frames for each input buffer is
>   kept as-is. As Hans pointed, we currently have a hard limit of 32
>   buffers per queue, and it may be non-trivial to lift. Also some codecs
>   (at least Venus AFAIK) do have this restriction in hardware, so unless
>   we want to do some buffer-rearranging in-kernel, it is probably better
>   to keep the default behavior as-is. Finally, relaxing the rule should
>   be easy enough if we add one extra control to query whether the
>   hardware can work with slice units, as opposed to frame units.

Makes sense, as long as the restriction can be lifted in the future.

> * The other hot topic is the use of capture buffer indexes in order to
>   reference frames. I understand the concerns, but I doesn't seem like
>   we have come with a better proposal so far - and since capture buffers
>   are essentially well, frames, using their buffer index to directly
>   reference them doesn't sound too inappropriate to me. There is also
>   the restriction that drivers must return capture buffers in queue
>   order. Do we have any concrete example where this scenario would not
>   work?

I'll start a separate discussion thread for this to avoid polluting the
review of this documentation.

Regards,

	Hans