From: Hans Verkuil <hverkuil@xs4all.nl>
To: Boris Brezillon <boris.brezillon@collabora.com>
Cc: Mauro Carvalho Chehab <mchehab@kernel.org>,
Hans Verkuil <hans.verkuil@cisco.com>,
Laurent Pinchart <laurent.pinchart@ideasonboard.com>,
Sakari Ailus <sakari.ailus@iki.fi>,
linux-media@vger.kernel.org, Tomasz Figa <tfiga@chromium.org>,
Nicolas Dufresne <nicolas@ndufresne.ca>,
kernel@collabora.com,
Paul Kocialkowski <paul.kocialkowski@bootlin.com>,
Maxime Ripard <maxime.ripard@bootlin.com>,
Ezequiel Garcia <ezequiel@collabora.com>,
Jonas Karlman <jonas@kwiboo.se>,
Jernej Skrabec <jernej.skrabec@siol.net>,
Alexandre Courbot <acourbot@chromium.org>,
Thierry Reding <thierry.reding@gmail.com>
Subject: Re: [PATCH v3 2/3] media: uapi: h264: Add the concept of decoding mode
Date: Mon, 22 Jul 2019 21:00:16 +0200 [thread overview]
Message-ID: <d84dcc42-f0b6-9e6d-e1a9-0cdebdc12597@xs4all.nl> (raw)
In-Reply-To: <20190722195432.09667355@collabora.com>
On 7/22/19 7:54 PM, Boris Brezillon wrote:
> Hi Hans,
>
> On Mon, 22 Jul 2019 17:29:21 +0200
> Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
>> On 7/3/19 2:28 PM, Boris Brezillon wrote:
>>> Some stateless decoders don't support per-slice decoding (or at least
>>> not in a way that would make them efficient or easy to use).
>>> Let's expose a menu to control and expose the supported decoding modes.
>>> Drivers are allowed to support only one decoding but they can support
>>> both too.
>>>
>>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
>>> Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
>>> ---
>>> Changes in v3:
>>> * s/per-{slice,frame} decoding/{slice,frame}-based decoding/
>>> * Add Paul's R-b
>>>
>>> Changes in v2:
>>> * Allow decoding multiple slices in per-slice decoding mode
>>> * Minor doc improvement/fixes
>>> ---
>>> .../media/uapi/v4l/ext-ctrls-codec.rst | 47 ++++++++++++++++++-
>>> drivers/media/v4l2-core/v4l2-ctrls.c | 9 ++++
>>> include/media/h264-ctrls.h | 13 +++++
>>> 3 files changed, 68 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
>>> index 3ae1367806cf..47ba2d057a92 100644
>>> --- a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
>>> +++ b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
>>> @@ -1748,6 +1748,14 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
>>> * - __u32
>>> - ``size``
>>> -
>>> + * - __u32
>>> + - ``start_byte_offset``
>>> + - Where the slice payload starts in the output buffer. Useful when
>>> + operating in frame-based decoding mode and decoding multi-slice
>>> + content. In this case, the output buffer will contain more than one
>>> + slice and some codecs need to know where each slice starts. Note that
>>> + this offsets points to the beginning of the slice which is supposed to
>>
>> offsets -> offset
>>
>>> + contain an ANNEX B start code
>>
>> Add . at the end of the sentence.
>>
>> I think this is a bit awkward. How about:
>>
>> "Note that the slice at this offset shall start with an ANNEX B start code."
>
> Definitely better.
>
>>
>> I'm assuming it has to actually start with an ANNEX B code? Or should it
>> just 'contain' an ANNEX B code?
>
> It has to start with an ANNEX B code.
>
>>
>> When in sliced-based decoding mode, what should be used here? I assume that in
>> that case start_byte_offset would be 0, and that the slice shall still begin
>> with an ANNEX B start code?
>
> The first slice should have start_byte_offset set to 0 and should start
> with an ANNEX B start code, but even in slice-based decoding mode, the
> driver can be passed several slices in the same buffer, in which case,
> the second slice will have start_byte_offset > 0.
>
>
>>
>>> * - __u32
>>> - ``header_bit_size``
>>> -
>>> @@ -1931,7 +1939,10 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
>>> -
>>> * - __u16
>>> - ``num_slices``
>>> - - Number of slices needed to decode the current frame
>>> + - Number of slices needed to decode the current frame/field. When
>>> + operating in slice-based decoding mode (see
>>> + :c:type:`v4l2_mpeg_video_h264_decoding_mode`), this field
>>> + should always be set to one
>>
>> Add . at the end of the sentence.
>>
>>> * - __u16
>>> - ``nal_ref_idc``
>>> - NAL reference ID value coming from the NAL Unit header
>>> @@ -2022,6 +2033,40 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
>>> - 0x00000004
>>> - The DPB entry is a long term reference frame
>>>
>>> +``V4L2_CID_MPEG_VIDEO_H264_DECODING_MODE (enum)``
>>> + Specifies the decoding mode to use. Currently exposes slice-based and
>>> + frame-based decoding but new modes might be added later on.
>>> +
>>> + .. note::
>>> +
>>> + This menu control is not yet part of the public kernel API and
>>> + it is expected to change.
>>> +
>>> +.. c:type:: v4l2_mpeg_video_h264_decoding_mode
>>> +
>>> +.. cssclass:: longtable
>>> +
>>> +.. flat-table::
>>> + :header-rows: 0
>>> + :stub-columns: 0
>>> + :widths: 1 1 2
>>> +
>>> + * - ``V4L2_MPEG_VIDEO_H264_SLICE_BASED_DECODING``
>>> + - 0
>>> + - The decoding is done at the slice granularity.
>>> + v4l2_ctrl_h264_decode_params->num_slices can be set to anything between
>>> + 1 and then number of slices that remain to fully decode the
>>
>> then -> the
>>
>>> + frame/field.
>>> + The output buffer should contain
>>> + v4l2_ctrl_h264_decode_params->num_slices slices.
>>> + * - ``V4L2_MPEG_VIDEO_H264_FRAME_BASED_DECODING``
>>> + - 1
>>> + - The decoding is done at the frame granularity.
>>> + v4l2_ctrl_h264_decode_params->num_slices should be set to the number of
>>> + slices forming a frame.
>>> + The output buffer should contain all slices needed to decode the
>>> + frame/field.
>>> +
>>> .. _v4l2-mpeg-mpeg2:
>>>
>>> ``V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS (struct)``
>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
>>> index 471ff5c91f43..70d994be27e1 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
>>> @@ -394,6 +394,11 @@ const char * const *v4l2_ctrl_get_menu(u32 id)
>>> "Explicit",
>>> NULL,
>>> };
>>> + static const char * const h264_decoding_mode[] = {
>>> + "Slice-based",
>>> + "Frame-based",
>>
>> based -> Based
>>
>>> + NULL,
>>> + };
>>> static const char * const mpeg_mpeg2_level[] = {
>>> "Low",
>>> "Main",
>>> @@ -625,6 +630,8 @@ const char * const *v4l2_ctrl_get_menu(u32 id)
>>> return h264_fp_arrangement_type;
>>> case V4L2_CID_MPEG_VIDEO_H264_FMO_MAP_TYPE:
>>> return h264_fmo_map_type;
>>> + case V4L2_CID_MPEG_VIDEO_H264_DECODING_MODE:
>>> + return h264_decoding_mode;
>>> case V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL:
>>> return mpeg_mpeg2_level;
>>> case V4L2_CID_MPEG_VIDEO_MPEG2_PROFILE:
>>> @@ -844,6 +851,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>>> case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix";
>>> case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: return "H264 Slice Parameters";
>>> case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: return "H264 Decode Parameters";
>>> + case V4L2_CID_MPEG_VIDEO_H264_DECODING_MODE: return "H264 Decoding Mode";
>>> case V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL: return "MPEG2 Level";
>>> case V4L2_CID_MPEG_VIDEO_MPEG2_PROFILE: return "MPEG2 Profile";
>>> case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value";
>>> @@ -1212,6 +1220,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>>> case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_IDC:
>>> case V4L2_CID_MPEG_VIDEO_H264_SEI_FP_ARRANGEMENT_TYPE:
>>> case V4L2_CID_MPEG_VIDEO_H264_FMO_MAP_TYPE:
>>> + case V4L2_CID_MPEG_VIDEO_H264_DECODING_MODE:
>>> case V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL:
>>> case V4L2_CID_MPEG_VIDEO_MPEG2_PROFILE:
>>> case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL:
>>> diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h
>>> index e1404d78d6ff..206fd5ada620 100644
>>> --- a/include/media/h264-ctrls.h
>>> +++ b/include/media/h264-ctrls.h
>>> @@ -26,6 +26,7 @@
>>> #define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+1002)
>>> #define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (V4L2_CID_MPEG_BASE+1003)
>>> #define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (V4L2_CID_MPEG_BASE+1004)
>>> +#define V4L2_CID_MPEG_VIDEO_H264_DECODING_MODE (V4L2_CID_MPEG_BASE+1005)
>>>
>>> /* enum v4l2_ctrl_type type values */
>>> #define V4L2_CTRL_TYPE_H264_SPS 0x0110
>>> @@ -33,6 +34,12 @@
>>> #define V4L2_CTRL_TYPE_H264_SCALING_MATRIX 0x0112
>>> #define V4L2_CTRL_TYPE_H264_SLICE_PARAMS 0x0113
>>> #define V4L2_CTRL_TYPE_H264_DECODE_PARAMS 0x0114
>>> +#define V4L2_CTRL_TYPE_H264_DECODING_MODE 0x0115
>>> +
>>> +enum v4l2_mpeg_video_h264_decoding_mode {
>>> + V4L2_MPEG_VIDEO_H264_SLICE_BASED_DECODING,
>>> + V4L2_MPEG_VIDEO_H264_FRAME_BASED_DECODING,
>>> +};
>>>
>>> #define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01
>>> #define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02
>>> @@ -111,6 +118,8 @@ struct v4l2_h264_pred_weight_table {
>>> struct v4l2_h264_weight_factors weight_factors[2];
>>> };
>>>
>>> +#define V4L2_H264_MAX_SLICES_PER_FRAME 16
>>
>> Are there arrays in these compound control structs where this define can be used?
>
> No, slices_params is a separate control, but I initialize
> slices_params_ctrl_cfg.dims[0] to this value.
>
>> Is this define standards-based or a restriction of V4L2?
>
> It's defined by the standard.
OK, can you add a comment before the V4L2_H264_MAX_SLICES_PER_FRAME define
mentioning this? Perhaps with a reference to the standard as well.
>
> Will fix the other typos you reported.
>
> Thanks for the review.
My pleasure,
Hans
next prev parent reply other threads:[~2019-07-22 19:00 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-03 12:28 [PATCH v3 0/3] media: uapi: h264: First batch of adjusments Boris Brezillon
2019-07-03 12:28 ` [PATCH v3 1/3] media: uapi: h264: Clarify our expectations regarding NAL header format Boris Brezillon
2019-07-05 16:40 ` Ezequiel Garcia
2019-07-05 17:16 ` Boris Brezillon
2019-07-25 6:42 ` Boris Brezillon
2019-07-25 19:36 ` Paul Kocialkowski
2019-07-26 2:39 ` Ezequiel Garcia
2019-07-26 6:28 ` Boris Brezillon
2019-07-26 7:30 ` Boris Brezillon
2019-07-26 8:53 ` Hans Verkuil
2019-07-27 9:27 ` Paul Kocialkowski
2019-07-27 9:46 ` Boris Brezillon
2019-07-29 13:25 ` Paul Kocialkowski
2019-07-29 14:19 ` Boris Brezillon
2019-07-27 12:52 ` Ezequiel Garcia
2019-07-27 13:49 ` Boris Brezillon
2019-07-29 13:33 ` Paul Kocialkowski
2019-07-25 19:26 ` Paul Kocialkowski
2019-07-03 12:28 ` [PATCH v3 2/3] media: uapi: h264: Add the concept of decoding mode Boris Brezillon
2019-07-22 15:29 ` Hans Verkuil
2019-07-22 17:54 ` Boris Brezillon
2019-07-22 19:00 ` Hans Verkuil [this message]
2019-07-25 19:20 ` Paul Kocialkowski
2019-07-03 12:28 ` [PATCH v3 3/3] media: uapi: h264: Get rid of the p0/b0/b1 ref-lists Boris Brezillon
2019-07-03 17:18 ` Nicolas Dufresne
2019-07-05 15:24 ` Ezequiel Garcia
2019-07-24 3:39 ` Tomasz Figa
2019-07-24 5:46 ` Boris Brezillon
2019-07-25 19:38 ` Paul Kocialkowski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d84dcc42-f0b6-9e6d-e1a9-0cdebdc12597@xs4all.nl \
--to=hverkuil@xs4all.nl \
--cc=acourbot@chromium.org \
--cc=boris.brezillon@collabora.com \
--cc=ezequiel@collabora.com \
--cc=hans.verkuil@cisco.com \
--cc=jernej.skrabec@siol.net \
--cc=jonas@kwiboo.se \
--cc=kernel@collabora.com \
--cc=laurent.pinchart@ideasonboard.com \
--cc=linux-media@vger.kernel.org \
--cc=maxime.ripard@bootlin.com \
--cc=mchehab@kernel.org \
--cc=nicolas@ndufresne.ca \
--cc=paul.kocialkowski@bootlin.com \
--cc=sakari.ailus@iki.fi \
--cc=tfiga@chromium.org \
--cc=thierry.reding@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).