Re: [PATCH] v4l2-ctrl: add control for thumnails

[Hans Verkuil <hverkuil-cisco@xs4all.nl>]

On 04/06/2020 11:02, Stanimir Varbanov wrote:
> Hi Hans,
> 
> On 5/27/20 12:53 AM, Stanimir Varbanov wrote:
>> Hi Hans,
>>
>> On 5/26/20 3:04 PM, Hans Verkuil wrote:
>>> On 26/05/2020 10:54, Stanimir Varbanov wrote:
>>>> Add v4l2 control for decoder thumbnail.
>>>>
>>>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>>>> ---
>>>>  Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst | 7 +++++++
>>>>  drivers/media/v4l2-core/v4l2-ctrls.c                      | 2 ++
>>>>  include/uapi/linux/v4l2-controls.h                        | 2 ++
>>>>  3 files changed, 11 insertions(+)
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>> index d0d506a444b1..e838e410651b 100644
>>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>> @@ -3726,6 +3726,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>>>      disables generating SPS and PPS at every IDR. Setting it to one enables
>>>>      generating SPS and PPS at every IDR.
>>>>  
>>>> +``V4L2_CID_MPEG_VIDEO_DECODER_THUMBNAIL (button)``
>>>> +    Instructs the decoder to produce immediate output. The decoder should
>>>> +    consume first input buffer for progressive stream (or first two buffers
>>>> +    for interlace). Decoder should not allocate more output buffers that it
>>>> +    is required to consume one input frame. Usually the decoder input
>>>> +    buffers will contain only I/IDR frames but it is not mandatory.
>>>
>>> This is very vague. It doesn't explain why the control is called 'THUMBNAIL',
>>> but more importantly it doesn't explain how this relates to normal decoding.
>>
>> If in the normal decode the capture queue buffers are 5, in the
>> thumbnail mode the number of buffers will be only 1 (if the bitstream is
>> progressive) and this will guarantee low memory usage. The other
>> difference is that the decoder will produce decoded frames (without
>> errors) only for I/IDR (sync frames).

Isn't this really a "DECODE_SYNC_FRAMES_ONLY" control? That's what it does,
right? Skip any B/P frames and only decode sync frames.

That this is useful for creating thumbnails is just a specific use-case.

Regards,

	Hans

>>
>>>
>>> I.e. if you are decoding and 'press' this control, what happens then?
>>
>> Might be the button type wasn't great idea. In fact the control should
>> be set before streamon so that the driver returns min_capture_bufs 1.
>>
>>>
>>> What exactly is the use-case?
>>
>> It could be used to generate thumbnails of all video clips in a folder
>> or when you open a Gallery application on your phone.
>>
> 
> What is your opinion on that control? I could consider to make it Venus
> custom control but from the use-case it looks other drivers also can
> benefit of it.
> 
> I tried to make more generic one [1] but it looks it will be too difficult.
>