* [PATCH v2 0/4] Add random IR and AUD encoder controls
@ 2020-12-06 10:27 Stanimir Varbanov
2020-12-06 10:27 ` [PATCH v2 1/4] media: v4l2-ctrls: Add intra-refresh period control Stanimir Varbanov
` (3 more replies)
0 siblings, 4 replies; 8+ messages in thread
From: Stanimir Varbanov @ 2020-12-06 10:27 UTC (permalink / raw)
To: linux-media, linux-kernel, linux-arm-msm
Cc: Ezequiel Garcia, Nicolas Dufresne, Maheshwar Ajja, Hans Verkuil,
Stanimir Varbanov
Hi,
The changes in the second version of the series are:
* change the sematics of the random intra-refresh control to
intra-refresh period, see 1/1 for more info.
* incorporate review comments for AUD NALU control (Hans)
regards,
Stan
Stanimir Varbanov (4):
media: v4l2-ctrls: Add intra-refresh period control
venus: venc: Add support for intra-refresh period
media: v4l2-ctrls: Add control for AUD generation
venus: venc: Add support for AUD NALU control
.../media/v4l/ext-ctrls-codec.rst | 16 +++++++
drivers/media/platform/qcom/venus/core.h | 2 +
drivers/media/platform/qcom/venus/venc.c | 42 +++++++++++++++++++
.../media/platform/qcom/venus/venc_ctrls.c | 15 ++++++-
drivers/media/v4l2-core/v4l2-ctrls.c | 4 ++
include/uapi/linux/v4l2-controls.h | 2 +
6 files changed, 80 insertions(+), 1 deletion(-)
--
2.17.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 1/4] media: v4l2-ctrls: Add intra-refresh period control
2020-12-06 10:27 [PATCH v2 0/4] Add random IR and AUD encoder controls Stanimir Varbanov
@ 2020-12-06 10:27 ` Stanimir Varbanov
2021-01-12 10:05 ` Hans Verkuil
2020-12-06 10:27 ` [PATCH v2 2/4] venus: venc: Add support for intra-refresh period Stanimir Varbanov
` (2 subsequent siblings)
3 siblings, 1 reply; 8+ messages in thread
From: Stanimir Varbanov @ 2020-12-06 10:27 UTC (permalink / raw)
To: linux-media, linux-kernel, linux-arm-msm
Cc: Ezequiel Garcia, Nicolas Dufresne, Maheshwar Ajja, Hans Verkuil,
Stanimir Varbanov
Add a control to set intra-refresh period.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
.../userspace-api/media/v4l/ext-ctrls-codec.rst | 11 +++++++++++
drivers/media/v4l2-core/v4l2-ctrls.c | 2 ++
include/uapi/linux/v4l2-controls.h | 1 +
3 files changed, 14 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 454ecd9a0f83..d65d7c1381b7 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -1104,6 +1104,17 @@ enum v4l2_mpeg_video_h264_entropy_mode -
macroblocks is refreshed until the cycle completes and starts from
the top of the frame. Applicable to H264, H263 and MPEG4 encoder.
+``V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD (integer)``
+ Intra macroblock refresh period. This sets the period to refresh
+ the whole frame. With other words, this defines the number of frames
+ for which the whole frame will be intra-refreshed. An example:
+ setting period to 1 means that the whole frame will be refreshed,
+ setting period to 2 means that the half of macroblocks will be
+ intra-refreshed on frameX and the other half of macroblocks
+ will be refreshed in frameX + 1 and so on. Setting period to zero
+ means no period is specified.
+ Applicable to H264 and HEVC encoders.
+
``V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE (boolean)``
Frame level rate control enable. If this control is disabled then
the quantization parameter for each frame type is constant and set
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index 5cbe0ffbf501..ac44848d2d6e 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -869,6 +869,7 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_MPEG_VIDEO_DECODER_SLICE_INTERFACE: return "Decoder Slice Interface";
case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER: return "MPEG4 Loop Filter Enable";
case V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB: return "Number of Intra Refresh MBs";
+ case V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD: return "Intra Refresh Period";
case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE: return "Frame Level Rate Control Enable";
case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE: return "H264 MB Level Rate Control";
case V4L2_CID_MPEG_VIDEO_HEADER_MODE: return "Sequence Header Mode";
@@ -1260,6 +1261,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
break;
case V4L2_CID_MPEG_VIDEO_MV_H_SEARCH_RANGE:
case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:
+ case V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD:
*type = V4L2_CTRL_TYPE_INTEGER;
break;
case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 823b214aac0c..54b9072ac49d 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -422,6 +422,7 @@ enum v4l2_mpeg_video_multi_slice_mode {
#define V4L2_CID_MPEG_VIDEO_MV_H_SEARCH_RANGE (V4L2_CID_CODEC_BASE+227)
#define V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE (V4L2_CID_CODEC_BASE+228)
#define V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME (V4L2_CID_CODEC_BASE+229)
+#define V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD (V4L2_CID_CODEC_BASE+230)
/* CIDs for the MPEG-2 Part 2 (H.262) codec */
#define V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL (V4L2_CID_CODEC_BASE+270)
--
2.17.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 2/4] venus: venc: Add support for intra-refresh period
2020-12-06 10:27 [PATCH v2 0/4] Add random IR and AUD encoder controls Stanimir Varbanov
2020-12-06 10:27 ` [PATCH v2 1/4] media: v4l2-ctrls: Add intra-refresh period control Stanimir Varbanov
@ 2020-12-06 10:27 ` Stanimir Varbanov
2020-12-06 10:27 ` [PATCH v2 3/4] media: v4l2-ctrls: Add control for AUD generation Stanimir Varbanov
2020-12-06 10:27 ` [PATCH v2 4/4] venus: venc: Add support for AUD NALU control Stanimir Varbanov
3 siblings, 0 replies; 8+ messages in thread
From: Stanimir Varbanov @ 2020-12-06 10:27 UTC (permalink / raw)
To: linux-media, linux-kernel, linux-arm-msm
Cc: Ezequiel Garcia, Nicolas Dufresne, Maheshwar Ajja, Hans Verkuil,
Stanimir Varbanov
Add support for intra-refresh period v4l2 control.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/core.h | 1 +
drivers/media/platform/qcom/venus/venc.c | 28 +++++++++++++++++++
.../media/platform/qcom/venus/venc_ctrls.c | 9 +++++-
3 files changed, 37 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index f03ed427accd..097fca39eb79 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -242,6 +242,7 @@ struct venc_controls {
u32 multi_slice_max_mb;
u32 header_mode;
+ u32 intra_refresh_period;
struct {
u32 h264;
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 1c61602c5de1..615d90612ebc 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -536,6 +536,7 @@ static int venc_set_properties(struct venus_inst *inst)
struct hfi_idr_period idrp;
struct hfi_quantization quant;
struct hfi_quantization_range quant_range;
+ struct hfi_intra_refresh intra_refresh = {};
u32 ptype, rate_control, bitrate;
u32 profile, level;
int ret;
@@ -717,6 +718,33 @@ static int venc_set_properties(struct venus_inst *inst)
if (ret)
return ret;
+ if ((inst->fmt_cap->pixfmt == V4L2_PIX_FMT_H264 ||
+ inst->fmt_cap->pixfmt == V4L2_PIX_FMT_HEVC) &&
+ (rate_control == HFI_RATE_CONTROL_CBR_VFR ||
+ rate_control == HFI_RATE_CONTROL_CBR_CFR)) {
+ intra_refresh.mode = HFI_INTRA_REFRESH_NONE;
+ intra_refresh.cir_mbs = 0;
+
+ if (ctr->intra_refresh_period) {
+ u32 mbs;
+
+ mbs = ALIGN(inst->width, 16) * ALIGN(inst->height, 16);
+ mbs /= 16 * 16;
+ if (mbs % ctr->intra_refresh_period)
+ mbs++;
+ mbs /= ctr->intra_refresh_period;
+
+ intra_refresh.mode = HFI_INTRA_REFRESH_RANDOM;
+ intra_refresh.cir_mbs = mbs;
+ }
+
+ ptype = HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH;
+
+ ret = hfi_session_set_property(inst, ptype, &intra_refresh);
+ if (ret)
+ return ret;
+ }
+
return 0;
}
diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c b/drivers/media/platform/qcom/venus/venc_ctrls.c
index cf860e6446c0..801026fab89d 100644
--- a/drivers/media/platform/qcom/venus/venc_ctrls.c
+++ b/drivers/media/platform/qcom/venus/venc_ctrls.c
@@ -208,6 +208,9 @@ static int venc_op_s_ctrl(struct v4l2_ctrl *ctrl)
case V4L2_CID_MPEG_VIDEO_FRAME_SKIP_MODE:
ctr->frame_skip_mode = ctrl->val;
break;
+ case V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD:
+ ctr->intra_refresh_period = ctrl->val;
+ break;
default:
return -EINVAL;
}
@@ -223,7 +226,7 @@ int venc_ctrl_init(struct venus_inst *inst)
{
int ret;
- ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 33);
+ ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 34);
if (ret)
return ret;
@@ -374,6 +377,10 @@ int venc_ctrl_init(struct venus_inst *inst)
(1 << V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT)),
V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED);
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD, 0,
+ ((4096 * 2304) >> 8), 1, 0);
+
ret = inst->ctrl_handler.error;
if (ret)
goto err;
--
2.17.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 3/4] media: v4l2-ctrls: Add control for AUD generation
2020-12-06 10:27 [PATCH v2 0/4] Add random IR and AUD encoder controls Stanimir Varbanov
2020-12-06 10:27 ` [PATCH v2 1/4] media: v4l2-ctrls: Add intra-refresh period control Stanimir Varbanov
2020-12-06 10:27 ` [PATCH v2 2/4] venus: venc: Add support for intra-refresh period Stanimir Varbanov
@ 2020-12-06 10:27 ` Stanimir Varbanov
2021-01-12 10:06 ` Hans Verkuil
2020-12-06 10:27 ` [PATCH v2 4/4] venus: venc: Add support for AUD NALU control Stanimir Varbanov
3 siblings, 1 reply; 8+ messages in thread
From: Stanimir Varbanov @ 2020-12-06 10:27 UTC (permalink / raw)
To: linux-media, linux-kernel, linux-arm-msm
Cc: Ezequiel Garcia, Nicolas Dufresne, Maheshwar Ajja, Hans Verkuil,
Stanimir Varbanov
Add a control to enable inserting of AUD NALU into encoded
bitstream.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst | 5 +++++
drivers/media/v4l2-core/v4l2-ctrls.c | 2 ++
include/uapi/linux/v4l2-controls.h | 1 +
3 files changed, 8 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index d65d7c1381b7..32eb233ee089 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -679,6 +679,11 @@ enum v4l2_mpeg_video_frame_skip_mode -
otherwise the decoder expects a single frame in per buffer.
Applicable to the decoder, all codecs.
+``V4L2_CID_MPEG_VIDEO_AU_DELIMITER (boolean)``
+ If enabled then, AUD (Access Unit Delimiter) NALUs will be generated.
+ That could be useful to find the start of a frame without having to
+ fully parse each NALU. Applicable to the H264 and HEVC encoders.
+
``V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE (boolean)``
Enable writing sample aspect ratio in the Video Usability
Information. Applicable to the H264 encoder.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index ac44848d2d6e..ea2de8b51e29 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -875,6 +875,7 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_MPEG_VIDEO_HEADER_MODE: return "Sequence Header Mode";
case V4L2_CID_MPEG_VIDEO_MAX_REF_PIC: return "Max Number of Reference Pics";
case V4L2_CID_MPEG_VIDEO_FRAME_SKIP_MODE: return "Frame Skip Mode";
+ case V4L2_CID_MPEG_VIDEO_AU_DELIMITER: return "Generate Access Unit Delimiters";
case V4L2_CID_MPEG_VIDEO_H263_I_FRAME_QP: return "H263 I-Frame QP Value";
case V4L2_CID_MPEG_VIDEO_H263_P_FRAME_QP: return "H263 P-Frame QP Value";
case V4L2_CID_MPEG_VIDEO_H263_B_FRAME_QP: return "H263 B-Frame QP Value";
@@ -1232,6 +1233,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE:
case V4L2_CID_MPEG_VIDEO_MPEG4_QPEL:
case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:
+ case V4L2_CID_MPEG_VIDEO_AU_DELIMITER:
case V4L2_CID_WIDE_DYNAMIC_RANGE:
case V4L2_CID_IMAGE_STABILIZATION:
case V4L2_CID_RDS_RECEPTION:
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 54b9072ac49d..1fb1a3ba3985 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -423,6 +423,7 @@ enum v4l2_mpeg_video_multi_slice_mode {
#define V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE (V4L2_CID_CODEC_BASE+228)
#define V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME (V4L2_CID_CODEC_BASE+229)
#define V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD (V4L2_CID_CODEC_BASE+230)
+#define V4L2_CID_MPEG_VIDEO_AU_DELIMITER (V4L2_CID_CODEC_BASE+231)
/* CIDs for the MPEG-2 Part 2 (H.262) codec */
#define V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL (V4L2_CID_CODEC_BASE+270)
--
2.17.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v2 4/4] venus: venc: Add support for AUD NALU control
2020-12-06 10:27 [PATCH v2 0/4] Add random IR and AUD encoder controls Stanimir Varbanov
` (2 preceding siblings ...)
2020-12-06 10:27 ` [PATCH v2 3/4] media: v4l2-ctrls: Add control for AUD generation Stanimir Varbanov
@ 2020-12-06 10:27 ` Stanimir Varbanov
3 siblings, 0 replies; 8+ messages in thread
From: Stanimir Varbanov @ 2020-12-06 10:27 UTC (permalink / raw)
To: linux-media, linux-kernel, linux-arm-msm
Cc: Ezequiel Garcia, Nicolas Dufresne, Maheshwar Ajja, Hans Verkuil,
Stanimir Varbanov
Add support for Access Unit Delimiter control into encoder.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/core.h | 1 +
drivers/media/platform/qcom/venus/venc.c | 14 ++++++++++++++
drivers/media/platform/qcom/venus/venc_ctrls.c | 8 +++++++-
3 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 097fca39eb79..f105a34ded5a 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -243,6 +243,7 @@ struct venc_controls {
u32 header_mode;
u32 intra_refresh_period;
+ bool aud_enable;
struct {
u32 h264;
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 615d90612ebc..99e0a82a31e0 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -745,6 +745,20 @@ static int venc_set_properties(struct venus_inst *inst)
return ret;
}
+ if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_H264 ||
+ inst->fmt_cap->pixfmt == V4L2_PIX_FMT_HEVC) {
+ struct hfi_enable en = {};
+
+ ptype = HFI_PROPERTY_PARAM_VENC_H264_GENERATE_AUDNAL;
+
+ if (ctr->aud_enable)
+ en.enable = 1;
+
+ ret = hfi_session_set_property(inst, ptype, &en);
+ if (ret)
+ return ret;
+ }
+
return 0;
}
diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c b/drivers/media/platform/qcom/venus/venc_ctrls.c
index 801026fab89d..a1aa56c74bc5 100644
--- a/drivers/media/platform/qcom/venus/venc_ctrls.c
+++ b/drivers/media/platform/qcom/venus/venc_ctrls.c
@@ -211,6 +211,9 @@ static int venc_op_s_ctrl(struct v4l2_ctrl *ctrl)
case V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD:
ctr->intra_refresh_period = ctrl->val;
break;
+ case V4L2_CID_MPEG_VIDEO_AU_DELIMITER:
+ ctr->aud_enable = ctrl->val;
+ break;
default:
return -EINVAL;
}
@@ -226,7 +229,7 @@ int venc_ctrl_init(struct venus_inst *inst)
{
int ret;
- ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 34);
+ ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 35);
if (ret)
return ret;
@@ -381,6 +384,9 @@ int venc_ctrl_init(struct venus_inst *inst)
V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD, 0,
((4096 * 2304) >> 8), 1, 0);
+ v4l2_ctrl_new_std(&inst->ctrl_handler, &venc_ctrl_ops,
+ V4L2_CID_MPEG_VIDEO_AU_DELIMITER, 0, 1, 1, 0);
+
ret = inst->ctrl_handler.error;
if (ret)
goto err;
--
2.17.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/4] media: v4l2-ctrls: Add intra-refresh period control
2020-12-06 10:27 ` [PATCH v2 1/4] media: v4l2-ctrls: Add intra-refresh period control Stanimir Varbanov
@ 2021-01-12 10:05 ` Hans Verkuil
2021-01-18 9:20 ` Stanimir Varbanov
0 siblings, 1 reply; 8+ messages in thread
From: Hans Verkuil @ 2021-01-12 10:05 UTC (permalink / raw)
To: Stanimir Varbanov, linux-media, linux-kernel, linux-arm-msm
Cc: Ezequiel Garcia, Nicolas Dufresne, Maheshwar Ajja
On 06/12/2020 11:27, Stanimir Varbanov wrote:
> Add a control to set intra-refresh period.
>
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 11 +++++++++++
> drivers/media/v4l2-core/v4l2-ctrls.c | 2 ++
> include/uapi/linux/v4l2-controls.h | 1 +
> 3 files changed, 14 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index 454ecd9a0f83..d65d7c1381b7 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -1104,6 +1104,17 @@ enum v4l2_mpeg_video_h264_entropy_mode -
> macroblocks is refreshed until the cycle completes and starts from
> the top of the frame. Applicable to H264, H263 and MPEG4 encoder.
>
> +``V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD (integer)``
> + Intra macroblock refresh period. This sets the period to refresh
> + the whole frame. With other words, this defines the number of frames
With -> In
> + for which the whole frame will be intra-refreshed. An example:
> + setting period to 1 means that the whole frame will be refreshed,
> + setting period to 2 means that the half of macroblocks will be
> + intra-refreshed on frameX and the other half of macroblocks
> + will be refreshed in frameX + 1 and so on. Setting period to zero
> + means no period is specified.
> + Applicable to H264 and HEVC encoders.
I'm confused. Isn't this the same as V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB?
Except that here you don't give the number of macroblocks but instead the number
of frames it will take to fully refresh a frame and leave it to the driver to
calculate the number of macroblocks?
If I am right, then you need to clearly document the relationship between the
two controls, and what happens if you set them both.
It seems the venus driver already supports V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB,
so why add this control as well?
Regards,
Hans
> +
> ``V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE (boolean)``
> Frame level rate control enable. If this control is disabled then
> the quantization parameter for each frame type is constant and set
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index 5cbe0ffbf501..ac44848d2d6e 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -869,6 +869,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_DECODER_SLICE_INTERFACE: return "Decoder Slice Interface";
> case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER: return "MPEG4 Loop Filter Enable";
> case V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB: return "Number of Intra Refresh MBs";
> + case V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD: return "Intra Refresh Period";
> case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE: return "Frame Level Rate Control Enable";
> case V4L2_CID_MPEG_VIDEO_MB_RC_ENABLE: return "H264 MB Level Rate Control";
> case V4L2_CID_MPEG_VIDEO_HEADER_MODE: return "Sequence Header Mode";
> @@ -1260,6 +1261,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> break;
> case V4L2_CID_MPEG_VIDEO_MV_H_SEARCH_RANGE:
> case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:
> + case V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD:
> *type = V4L2_CTRL_TYPE_INTEGER;
> break;
> case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 823b214aac0c..54b9072ac49d 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -422,6 +422,7 @@ enum v4l2_mpeg_video_multi_slice_mode {
> #define V4L2_CID_MPEG_VIDEO_MV_H_SEARCH_RANGE (V4L2_CID_CODEC_BASE+227)
> #define V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE (V4L2_CID_CODEC_BASE+228)
> #define V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME (V4L2_CID_CODEC_BASE+229)
> +#define V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD (V4L2_CID_CODEC_BASE+230)
>
> /* CIDs for the MPEG-2 Part 2 (H.262) codec */
> #define V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL (V4L2_CID_CODEC_BASE+270)
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 3/4] media: v4l2-ctrls: Add control for AUD generation
2020-12-06 10:27 ` [PATCH v2 3/4] media: v4l2-ctrls: Add control for AUD generation Stanimir Varbanov
@ 2021-01-12 10:06 ` Hans Verkuil
0 siblings, 0 replies; 8+ messages in thread
From: Hans Verkuil @ 2021-01-12 10:06 UTC (permalink / raw)
To: Stanimir Varbanov, linux-media, linux-kernel, linux-arm-msm
Cc: Ezequiel Garcia, Nicolas Dufresne, Maheshwar Ajja
On 06/12/2020 11:27, Stanimir Varbanov wrote:
> Add a control to enable inserting of AUD NALU into encoded
> bitstream.
>
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Looks good!
Hans
> ---
> Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst | 5 +++++
> drivers/media/v4l2-core/v4l2-ctrls.c | 2 ++
> include/uapi/linux/v4l2-controls.h | 1 +
> 3 files changed, 8 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index d65d7c1381b7..32eb233ee089 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -679,6 +679,11 @@ enum v4l2_mpeg_video_frame_skip_mode -
> otherwise the decoder expects a single frame in per buffer.
> Applicable to the decoder, all codecs.
>
> +``V4L2_CID_MPEG_VIDEO_AU_DELIMITER (boolean)``
> + If enabled then, AUD (Access Unit Delimiter) NALUs will be generated.
> + That could be useful to find the start of a frame without having to
> + fully parse each NALU. Applicable to the H264 and HEVC encoders.
> +
> ``V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE (boolean)``
> Enable writing sample aspect ratio in the Video Usability
> Information. Applicable to the H264 encoder.
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ac44848d2d6e..ea2de8b51e29 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -875,6 +875,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_HEADER_MODE: return "Sequence Header Mode";
> case V4L2_CID_MPEG_VIDEO_MAX_REF_PIC: return "Max Number of Reference Pics";
> case V4L2_CID_MPEG_VIDEO_FRAME_SKIP_MODE: return "Frame Skip Mode";
> + case V4L2_CID_MPEG_VIDEO_AU_DELIMITER: return "Generate Access Unit Delimiters";
> case V4L2_CID_MPEG_VIDEO_H263_I_FRAME_QP: return "H263 I-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_H263_P_FRAME_QP: return "H263 P-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_H263_B_FRAME_QP: return "H263 B-Frame QP Value";
> @@ -1232,6 +1233,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_ENABLE:
> case V4L2_CID_MPEG_VIDEO_MPEG4_QPEL:
> case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:
> + case V4L2_CID_MPEG_VIDEO_AU_DELIMITER:
> case V4L2_CID_WIDE_DYNAMIC_RANGE:
> case V4L2_CID_IMAGE_STABILIZATION:
> case V4L2_CID_RDS_RECEPTION:
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 54b9072ac49d..1fb1a3ba3985 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -423,6 +423,7 @@ enum v4l2_mpeg_video_multi_slice_mode {
> #define V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE (V4L2_CID_CODEC_BASE+228)
> #define V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME (V4L2_CID_CODEC_BASE+229)
> #define V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD (V4L2_CID_CODEC_BASE+230)
> +#define V4L2_CID_MPEG_VIDEO_AU_DELIMITER (V4L2_CID_CODEC_BASE+231)
>
> /* CIDs for the MPEG-2 Part 2 (H.262) codec */
> #define V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL (V4L2_CID_CODEC_BASE+270)
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/4] media: v4l2-ctrls: Add intra-refresh period control
2021-01-12 10:05 ` Hans Verkuil
@ 2021-01-18 9:20 ` Stanimir Varbanov
0 siblings, 0 replies; 8+ messages in thread
From: Stanimir Varbanov @ 2021-01-18 9:20 UTC (permalink / raw)
To: Hans Verkuil, Stanimir Varbanov, linux-media, linux-kernel,
linux-arm-msm
Cc: Ezequiel Garcia, Nicolas Dufresne, Maheshwar Ajja
On 1/12/21 12:05 PM, Hans Verkuil wrote:
> On 06/12/2020 11:27, Stanimir Varbanov wrote:
>> Add a control to set intra-refresh period.
>>
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 11 +++++++++++
>> drivers/media/v4l2-core/v4l2-ctrls.c | 2 ++
>> include/uapi/linux/v4l2-controls.h | 1 +
>> 3 files changed, 14 insertions(+)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> index 454ecd9a0f83..d65d7c1381b7 100644
>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> @@ -1104,6 +1104,17 @@ enum v4l2_mpeg_video_h264_entropy_mode -
>> macroblocks is refreshed until the cycle completes and starts from
>> the top of the frame. Applicable to H264, H263 and MPEG4 encoder.
>>
>> +``V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD (integer)``
>> + Intra macroblock refresh period. This sets the period to refresh
>> + the whole frame. With other words, this defines the number of frames
>
> With -> In
>
>> + for which the whole frame will be intra-refreshed. An example:
>> + setting period to 1 means that the whole frame will be refreshed,
>> + setting period to 2 means that the half of macroblocks will be
>> + intra-refreshed on frameX and the other half of macroblocks
>> + will be refreshed in frameX + 1 and so on. Setting period to zero
>> + means no period is specified.
>> + Applicable to H264 and HEVC encoders.
>
> I'm confused. Isn't this the same as V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB?
> Except that here you don't give the number of macroblocks but instead the number
> of frames it will take to fully refresh a frame and leave it to the driver to
> calculate the number of macroblocks?
Yes, correct. The periodic control looks more generic because it doesn't
limit the type of intra-refresh (cyclic vs random vs adaptive) but
instead set the number of frames to fully refresh the whole frame and is
also taken from Android mediacodec [1], where the user doesn't care too
much what intra-refresh type will be used but instead how much frames
are needed to fully refresh.
>
> If I am right, then you need to clearly document the relationship between the
> two controls, and what happens if you set them both.
Yep, I can add such description.
One note here; the CYCLIC_INTRA_REFRESH_MB and INTRA_REFRESH_PERIOD are
not interchangeable because intra-refresh period could use different
type of inter-refresh, eg. random vs cyclic.
>
> It seems the venus driver already supports V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB,
> so why add this control as well?
The cyclic intra-refresh control in Venus driver is just enumerable but
ignored when set, in other words it does nothing.
>
> Regards,
>
> Hans
>
[1]
https://developer.android.com/reference/android/media/MediaFormat#KEY_INTRA_REFRESH_PERIOD
--
regards,
Stan
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-01-18 20:08 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-06 10:27 [PATCH v2 0/4] Add random IR and AUD encoder controls Stanimir Varbanov
2020-12-06 10:27 ` [PATCH v2 1/4] media: v4l2-ctrls: Add intra-refresh period control Stanimir Varbanov
2021-01-12 10:05 ` Hans Verkuil
2021-01-18 9:20 ` Stanimir Varbanov
2020-12-06 10:27 ` [PATCH v2 2/4] venus: venc: Add support for intra-refresh period Stanimir Varbanov
2020-12-06 10:27 ` [PATCH v2 3/4] media: v4l2-ctrls: Add control for AUD generation Stanimir Varbanov
2021-01-12 10:06 ` Hans Verkuil
2020-12-06 10:27 ` [PATCH v2 4/4] venus: venc: Add support for AUD NALU control Stanimir Varbanov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).