* [PATCH 0/9] media: cedrus: Add H264 decoding support
@ 2018-06-13 14:07 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Maxime Ripard
Hi,
Here is a preliminary version of the H264 decoding support in the
cedrus driver.
As you might already know, the cedrus driver relies on the Request
API, and is a reverse engineered driver for the video decoding engine
found on the Allwinner SoCs.
This work has been possible thanks to the work done by the people
behind libvdpau-sunxi found here:
https://github.com/linux-sunxi/libvdpau-sunxi/
This driver is based on the last version of the cedrus driver sent by
Paul, based on Request API v13 sent by Hans:
https://lkml.org/lkml/2018/5/7/316
This driver has been tested only with baseline profile videos, and is
missing a few key features to decode videos with higher profiles.
This has been tested using our cedrus-frame-test tool, which should be
a quite generic v4l2-to-drm decoder using the request API to
demonstrate the video decoding:
https://github.com/free-electrons/cedrus-frame-test/, branch h264
However, sending this preliminary version, I'd really like to start a
discussion and get some feedback on the user-space API for the H264
controls exposed through the request API.
I've been using the controls currently integrated into ChromeOS that
have a working version of this particular setup. However, these
controls have a number of shortcomings and inconsistencies with other
decoding API. I've worked with libva so far, but I've noticed already
that:
- The kernel UAPI expects to have the nal_ref_idc variable, while
libva only exposes whether that frame is a reference frame or
not. I've looked at the rockchip driver in the ChromeOS tree, and
our own driver, and they both need only the information about
whether the frame is a reference one or not, so maybe we should
change this?
- The H264 bitstream exposes the picture default reference list (for
both list 0 and list 1), the slice reference list and an override
flag. The libva will only pass the reference list to be used (so
either the picture default's or the slice's) depending on the
override flag. The kernel UAPI wants the picture default reference
list and the slice reference list, but doesn't expose the override
flag, which prevents us from configuring properly the
hardware. Our video decoding engine needs the three information,
but we can easily adapt to having only one. However, having two
doesn't really work for us.
It's pretty much the only one I've noticed so far, but we should
probably fix them already. And there's probably other, feel free to
step in.
Maxime Ripard (8):
media: cedrus: Add wrappers around container_of for our buffers
media: cedrus: Add a macro to check for the validity of a control
media: cedrus: make engine type more generic
media: cedrus: Remove MPEG1 support
media: cedrus: Add ops structure
media: cedrus: Move IRQ maintainance to cedrus_dec_ops
media: cedrus: Add start and stop decoder operations
media: cedrus: Add H264 decoding support
Pawel Osciak (1):
CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
drivers/media/platform/sunxi/cedrus/Makefile | 2 +-
.../platform/sunxi/cedrus/sunxi_cedrus.c | 23 +
.../sunxi/cedrus/sunxi_cedrus_common.h | 81 +++-
.../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 38 +-
.../platform/sunxi/cedrus/sunxi_cedrus_h264.c | 443 ++++++++++++++++++
.../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 35 +-
.../platform/sunxi/cedrus/sunxi_cedrus_hw.h | 6 +-
.../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 52 +-
.../sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 --
.../platform/sunxi/cedrus/sunxi_cedrus_regs.h | 22 +-
.../sunxi/cedrus/sunxi_cedrus_video.c | 37 +-
drivers/media/v4l2-core/v4l2-ctrls.c | 42 ++
include/media/v4l2-ctrls.h | 10 +
include/uapi/linux/v4l2-controls.h | 164 +++++++
include/uapi/linux/videodev2.h | 11 +
15 files changed, 912 insertions(+), 87 deletions(-)
create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_h264.c
delete mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
--
2.17.0
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 0/9] media: cedrus: Add H264 decoding support
@ 2018-06-13 14:07 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
Here is a preliminary version of the H264 decoding support in the
cedrus driver.
As you might already know, the cedrus driver relies on the Request
API, and is a reverse engineered driver for the video decoding engine
found on the Allwinner SoCs.
This work has been possible thanks to the work done by the people
behind libvdpau-sunxi found here:
https://github.com/linux-sunxi/libvdpau-sunxi/
This driver is based on the last version of the cedrus driver sent by
Paul, based on Request API v13 sent by Hans:
https://lkml.org/lkml/2018/5/7/316
This driver has been tested only with baseline profile videos, and is
missing a few key features to decode videos with higher profiles.
This has been tested using our cedrus-frame-test tool, which should be
a quite generic v4l2-to-drm decoder using the request API to
demonstrate the video decoding:
https://github.com/free-electrons/cedrus-frame-test/, branch h264
However, sending this preliminary version, I'd really like to start a
discussion and get some feedback on the user-space API for the H264
controls exposed through the request API.
I've been using the controls currently integrated into ChromeOS that
have a working version of this particular setup. However, these
controls have a number of shortcomings and inconsistencies with other
decoding API. I've worked with libva so far, but I've noticed already
that:
- The kernel UAPI expects to have the nal_ref_idc variable, while
libva only exposes whether that frame is a reference frame or
not. I've looked at the rockchip driver in the ChromeOS tree, and
our own driver, and they both need only the information about
whether the frame is a reference one or not, so maybe we should
change this?
- The H264 bitstream exposes the picture default reference list (for
both list 0 and list 1), the slice reference list and an override
flag. The libva will only pass the reference list to be used (so
either the picture default's or the slice's) depending on the
override flag. The kernel UAPI wants the picture default reference
list and the slice reference list, but doesn't expose the override
flag, which prevents us from configuring properly the
hardware. Our video decoding engine needs the three information,
but we can easily adapt to having only one. However, having two
doesn't really work for us.
It's pretty much the only one I've noticed so far, but we should
probably fix them already. And there's probably other, feel free to
step in.
Maxime Ripard (8):
media: cedrus: Add wrappers around container_of for our buffers
media: cedrus: Add a macro to check for the validity of a control
media: cedrus: make engine type more generic
media: cedrus: Remove MPEG1 support
media: cedrus: Add ops structure
media: cedrus: Move IRQ maintainance to cedrus_dec_ops
media: cedrus: Add start and stop decoder operations
media: cedrus: Add H264 decoding support
Pawel Osciak (1):
CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
drivers/media/platform/sunxi/cedrus/Makefile | 2 +-
.../platform/sunxi/cedrus/sunxi_cedrus.c | 23 +
.../sunxi/cedrus/sunxi_cedrus_common.h | 81 +++-
.../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 38 +-
.../platform/sunxi/cedrus/sunxi_cedrus_h264.c | 443 ++++++++++++++++++
.../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 35 +-
.../platform/sunxi/cedrus/sunxi_cedrus_hw.h | 6 +-
.../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 52 +-
.../sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 --
.../platform/sunxi/cedrus/sunxi_cedrus_regs.h | 22 +-
.../sunxi/cedrus/sunxi_cedrus_video.c | 37 +-
drivers/media/v4l2-core/v4l2-ctrls.c | 42 ++
include/media/v4l2-ctrls.h | 10 +
include/uapi/linux/v4l2-controls.h | 164 +++++++
include/uapi/linux/videodev2.h | 11 +
15 files changed, 912 insertions(+), 87 deletions(-)
create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_h264.c
delete mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
--
2.17.0
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-13 14:07 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Guenter Roeck, Maxime Ripard
From: Pawel Osciak <posciak@chromium.org>
Signed-off-by: Pawel Osciak <posciak@chromium.org>
Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
Tested-by: Tomasz Figa <tfiga@chromium.org>
[rebase44(groeck): include linux/types.h in v4l2-controls.h]
Signed-off-by: Guenter Roeck <groeck@chromium.org>
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
drivers/media/v4l2-core/v4l2-ctrls.c | 42 +++++++
include/media/v4l2-ctrls.h | 10 ++
include/uapi/linux/v4l2-controls.h | 164 +++++++++++++++++++++++++++
include/uapi/linux/videodev2.h | 11 ++
4 files changed, 227 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index cdf860c8e3d8..1f63c725bad1 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -807,6 +807,11 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER:return "H264 Number of HC Layers";
case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP:
return "H264 Set QP Value for HC Layers";
+ case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 SPS";
+ case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 PPS";
+ case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix";
+ case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM: return "H264 Slice Parameters";
+ case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM: return "H264 Decode Parameters";
case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value";
case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value";
case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value";
@@ -1272,6 +1277,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
case V4L2_CID_RDS_TX_ALT_FREQS:
*type = V4L2_CTRL_TYPE_U32;
break;
+ case V4L2_CID_MPEG_VIDEO_H264_SPS:
+ *type = V4L2_CTRL_TYPE_H264_SPS;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_PPS:
+ *type = V4L2_CTRL_TYPE_H264_PPS;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX:
+ *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM:
+ *type = V4L2_CTRL_TYPE_H264_SLICE_PARAM;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM:
+ *type = V4L2_CTRL_TYPE_H264_DECODE_PARAM;
+ break;
case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
break;
@@ -1598,6 +1618,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
return 0;
+ case V4L2_CTRL_TYPE_H264_SPS:
+ case V4L2_CTRL_TYPE_H264_PPS:
+ case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
+ case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
+ case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
+ return 0;
+
default:
return -EINVAL;
}
@@ -2172,6 +2199,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
case V4L2_CTRL_TYPE_U32:
elem_size = sizeof(u32);
break;
+ case V4L2_CTRL_TYPE_H264_SPS:
+ elem_size = sizeof(struct v4l2_ctrl_h264_sps);
+ break;
+ case V4L2_CTRL_TYPE_H264_PPS:
+ elem_size = sizeof(struct v4l2_ctrl_h264_pps);
+ break;
+ case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
+ elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix);
+ break;
+ case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
+ elem_size = sizeof(struct v4l2_ctrl_h264_slice_param);
+ break;
+ case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
+ elem_size = sizeof(struct v4l2_ctrl_h264_decode_param);
+ break;
case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
break;
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index 963c37b02363..9c223793a16a 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -41,6 +41,11 @@ struct poll_table_struct;
* @p_u16: Pointer to a 16-bit unsigned value.
* @p_u32: Pointer to a 32-bit unsigned value.
* @p_char: Pointer to a string.
+ * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps.
+ * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps.
+ * @p_h264_scal_mtrx: Pointer to a struct v4l2_ctrl_h264_scaling_matrix.
+ * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_param.
+ * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_param.
* @p: Pointer to a compound value.
*/
union v4l2_ctrl_ptr {
@@ -50,6 +55,11 @@ union v4l2_ctrl_ptr {
u16 *p_u16;
u32 *p_u32;
char *p_char;
+ struct v4l2_ctrl_h264_sps *p_h264_sps;
+ struct v4l2_ctrl_h264_pps *p_h264_pps;
+ struct v4l2_ctrl_h264_scaling_matrix *p_h264_scal_mtrx;
+ struct v4l2_ctrl_h264_slice_param *p_h264_slice_param;
+ struct v4l2_ctrl_h264_decode_param *p_h264_decode_param;
void *p;
};
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 23da8bfa7e6f..ac307c59683c 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -50,6 +50,8 @@
#ifndef __LINUX_V4L2_CONTROLS_H
#define __LINUX_V4L2_CONTROLS_H
+#include <linux/types.h>
+
/* Control classes */
#define V4L2_CTRL_CLASS_USER 0x00980000 /* Old-style 'user' controls */
#define V4L2_CTRL_CLASS_MPEG 0x00990000 /* MPEG-compression controls */
@@ -531,6 +533,12 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type {
};
#define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER (V4L2_CID_MPEG_BASE+381)
#define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP (V4L2_CID_MPEG_BASE+382)
+#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+383)
+#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+384)
+#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+385)
+#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM (V4L2_CID_MPEG_BASE+386)
+#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM (V4L2_CID_MPEG_BASE+387)
+
#define V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP (V4L2_CID_MPEG_BASE+400)
#define V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP (V4L2_CID_MPEG_BASE+401)
#define V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP (V4L2_CID_MPEG_BASE+402)
@@ -1078,6 +1086,162 @@ enum v4l2_detect_md_mode {
#define V4L2_CID_DETECT_MD_THRESHOLD_GRID (V4L2_CID_DETECT_CLASS_BASE + 3)
#define V4L2_CID_DETECT_MD_REGION_GRID (V4L2_CID_DETECT_CLASS_BASE + 4)
+/* Complex controls */
+
+#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01
+#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02
+#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04
+#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08
+#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10
+#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20
+
+#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01
+#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02
+#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04
+#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08
+#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10
+#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20
+#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40
+struct v4l2_ctrl_h264_sps {
+ __u8 profile_idc;
+ __u8 constraint_set_flags;
+ __u8 level_idc;
+ __u8 seq_parameter_set_id;
+ __u8 chroma_format_idc;
+ __u8 bit_depth_luma_minus8;
+ __u8 bit_depth_chroma_minus8;
+ __u8 log2_max_frame_num_minus4;
+ __u8 pic_order_cnt_type;
+ __u8 log2_max_pic_order_cnt_lsb_minus4;
+ __s32 offset_for_non_ref_pic;
+ __s32 offset_for_top_to_bottom_field;
+ __u8 num_ref_frames_in_pic_order_cnt_cycle;
+ __s32 offset_for_ref_frame[255];
+ __u8 max_num_ref_frames;
+ __u16 pic_width_in_mbs_minus1;
+ __u16 pic_height_in_map_units_minus1;
+ __u8 flags;
+};
+
+#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001
+#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002
+#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004
+#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008
+#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010
+#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020
+#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040
+#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080
+struct v4l2_ctrl_h264_pps {
+ __u8 pic_parameter_set_id;
+ __u8 seq_parameter_set_id;
+ __u8 num_slice_groups_minus1;
+ __u8 num_ref_idx_l0_default_active_minus1;
+ __u8 num_ref_idx_l1_default_active_minus1;
+ __u8 weighted_bipred_idc;
+ __s8 pic_init_qp_minus26;
+ __s8 pic_init_qs_minus26;
+ __s8 chroma_qp_index_offset;
+ __s8 second_chroma_qp_index_offset;
+ __u8 flags;
+};
+
+struct v4l2_ctrl_h264_scaling_matrix {
+ __u8 scaling_list_4x4[6][16];
+ __u8 scaling_list_8x8[6][64];
+};
+
+struct v4l2_h264_weight_factors {
+ __s8 luma_weight[32];
+ __s8 luma_offset[32];
+ __s8 chroma_weight[32][2];
+ __s8 chroma_offset[32][2];
+};
+
+struct v4l2_h264_pred_weight_table {
+ __u8 luma_log2_weight_denom;
+ __u8 chroma_log2_weight_denom;
+ struct v4l2_h264_weight_factors weight_factors[2];
+};
+
+enum v4l2_h264_slice_type {
+ V4L2_H264_SLICE_TYPE_P = 0,
+ V4L2_H264_SLICE_TYPE_B = 1,
+ V4L2_H264_SLICE_TYPE_I = 2,
+ V4L2_H264_SLICE_TYPE_SP = 3,
+ V4L2_H264_SLICE_TYPE_SI = 4,
+};
+
+#define V4L2_SLICE_FLAG_FIELD_PIC 0x01
+#define V4L2_SLICE_FLAG_BOTTOM_FIELD 0x02
+#define V4L2_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04
+#define V4L2_SLICE_FLAG_SP_FOR_SWITCH 0x08
+struct v4l2_ctrl_h264_slice_param {
+ /* Size in bytes, including header */
+ __u32 size;
+ /* Offset in bits to slice_data() from the beginning of this slice. */
+ __u32 header_bit_size;
+
+ __u16 first_mb_in_slice;
+ enum v4l2_h264_slice_type slice_type;
+ __u8 pic_parameter_set_id;
+ __u8 colour_plane_id;
+ __u16 frame_num;
+ __u16 idr_pic_id;
+ __u16 pic_order_cnt_lsb;
+ __s32 delta_pic_order_cnt_bottom;
+ __s32 delta_pic_order_cnt0;
+ __s32 delta_pic_order_cnt1;
+ __u8 redundant_pic_cnt;
+
+ struct v4l2_h264_pred_weight_table pred_weight_table;
+ /* Size in bits of dec_ref_pic_marking() syntax element. */
+ __u32 dec_ref_pic_marking_bit_size;
+ /* Size in bits of pic order count syntax. */
+ __u32 pic_order_cnt_bit_size;
+
+ __u8 cabac_init_idc;
+ __s8 slice_qp_delta;
+ __s8 slice_qs_delta;
+ __u8 disable_deblocking_filter_idc;
+ __s8 slice_alpha_c0_offset_div2;
+ __s8 slice_beta_offset_div2;
+ __u32 slice_group_change_cycle;
+
+ __u8 num_ref_idx_l0_active_minus1;
+ __u8 num_ref_idx_l1_active_minus1;
+ /* Entries on each list are indices
+ * into v4l2_ctrl_h264_decode_param.dpb[]. */
+ __u8 ref_pic_list0[32];
+ __u8 ref_pic_list1[32];
+
+ __u8 flags;
+};
+
+/* If not set, this entry is unused for reference. */
+#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x01
+#define V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM 0x02
+struct v4l2_h264_dpb_entry {
+ __u32 buf_index; /* v4l2_buffer index */
+ __u16 frame_num;
+ __u16 pic_num;
+ /* Note that field is indicated by v4l2_buffer.field */
+ __s32 top_field_order_cnt;
+ __s32 bottom_field_order_cnt;
+ __u8 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
+};
+
+struct v4l2_ctrl_h264_decode_param {
+ __u32 num_slices;
+ __u8 idr_pic_flag;
+ __u8 nal_ref_idc;
+ __s32 top_field_order_cnt;
+ __s32 bottom_field_order_cnt;
+ __u8 ref_pic_list_p0[32];
+ __u8 ref_pic_list_b0[32];
+ __u8 ref_pic_list_b1[32];
+ struct v4l2_h264_dpb_entry dpb[16];
+};
+
struct v4l2_ctrl_mpeg2_frame_hdr {
__u32 slice_len;
__u32 slice_pos;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 242a6bfa1440..4b4a1b25a0db 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -626,6 +626,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
#define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
#define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
+#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
#define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */
#define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */
#define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */
@@ -1589,6 +1590,11 @@ struct v4l2_ext_control {
__u8 __user *p_u8;
__u16 __user *p_u16;
__u32 __user *p_u32;
+ struct v4l2_ctrl_h264_sps __user *p_h264_sps;
+ struct v4l2_ctrl_h264_pps __user *p_h264_pps;
+ struct v4l2_ctrl_h264_scaling_matrix __user *p_h264_scal_mtrx;
+ struct v4l2_ctrl_h264_slice_param __user *p_h264_slice_param;
+ struct v4l2_ctrl_h264_decode_param __user *p_h264_decode_param;
struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
void __user *ptr;
};
@@ -1635,6 +1641,11 @@ enum v4l2_ctrl_type {
V4L2_CTRL_TYPE_U8 = 0x0100,
V4L2_CTRL_TYPE_U16 = 0x0101,
V4L2_CTRL_TYPE_U32 = 0x0102,
+ V4L2_CTRL_TYPE_H264_SPS = 0x0103,
+ V4L2_CTRL_TYPE_H264_PPS = 0x0104,
+ V4L2_CTRL_TYPE_H264_SCALING_MATRIX = 0x0105,
+ V4L2_CTRL_TYPE_H264_SLICE_PARAM = 0x0106,
+ V4L2_CTRL_TYPE_H264_DECODE_PARAM = 0x0107,
V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
};
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-06-13 14:07 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: linux-arm-kernel
From: Pawel Osciak <posciak@chromium.org>
Signed-off-by: Pawel Osciak <posciak@chromium.org>
Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
Tested-by: Tomasz Figa <tfiga@chromium.org>
[rebase44(groeck): include linux/types.h in v4l2-controls.h]
Signed-off-by: Guenter Roeck <groeck@chromium.org>
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
drivers/media/v4l2-core/v4l2-ctrls.c | 42 +++++++
include/media/v4l2-ctrls.h | 10 ++
include/uapi/linux/v4l2-controls.h | 164 +++++++++++++++++++++++++++
include/uapi/linux/videodev2.h | 11 ++
4 files changed, 227 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index cdf860c8e3d8..1f63c725bad1 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -807,6 +807,11 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER:return "H264 Number of HC Layers";
case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP:
return "H264 Set QP Value for HC Layers";
+ case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 SPS";
+ case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 PPS";
+ case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix";
+ case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM: return "H264 Slice Parameters";
+ case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM: return "H264 Decode Parameters";
case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value";
case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value";
case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value";
@@ -1272,6 +1277,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
case V4L2_CID_RDS_TX_ALT_FREQS:
*type = V4L2_CTRL_TYPE_U32;
break;
+ case V4L2_CID_MPEG_VIDEO_H264_SPS:
+ *type = V4L2_CTRL_TYPE_H264_SPS;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_PPS:
+ *type = V4L2_CTRL_TYPE_H264_PPS;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX:
+ *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM:
+ *type = V4L2_CTRL_TYPE_H264_SLICE_PARAM;
+ break;
+ case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM:
+ *type = V4L2_CTRL_TYPE_H264_DECODE_PARAM;
+ break;
case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
break;
@@ -1598,6 +1618,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
return 0;
+ case V4L2_CTRL_TYPE_H264_SPS:
+ case V4L2_CTRL_TYPE_H264_PPS:
+ case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
+ case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
+ case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
+ return 0;
+
default:
return -EINVAL;
}
@@ -2172,6 +2199,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
case V4L2_CTRL_TYPE_U32:
elem_size = sizeof(u32);
break;
+ case V4L2_CTRL_TYPE_H264_SPS:
+ elem_size = sizeof(struct v4l2_ctrl_h264_sps);
+ break;
+ case V4L2_CTRL_TYPE_H264_PPS:
+ elem_size = sizeof(struct v4l2_ctrl_h264_pps);
+ break;
+ case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
+ elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix);
+ break;
+ case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
+ elem_size = sizeof(struct v4l2_ctrl_h264_slice_param);
+ break;
+ case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
+ elem_size = sizeof(struct v4l2_ctrl_h264_decode_param);
+ break;
case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
break;
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index 963c37b02363..9c223793a16a 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -41,6 +41,11 @@ struct poll_table_struct;
* @p_u16: Pointer to a 16-bit unsigned value.
* @p_u32: Pointer to a 32-bit unsigned value.
* @p_char: Pointer to a string.
+ * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps.
+ * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps.
+ * @p_h264_scal_mtrx: Pointer to a struct v4l2_ctrl_h264_scaling_matrix.
+ * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_param.
+ * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_param.
* @p: Pointer to a compound value.
*/
union v4l2_ctrl_ptr {
@@ -50,6 +55,11 @@ union v4l2_ctrl_ptr {
u16 *p_u16;
u32 *p_u32;
char *p_char;
+ struct v4l2_ctrl_h264_sps *p_h264_sps;
+ struct v4l2_ctrl_h264_pps *p_h264_pps;
+ struct v4l2_ctrl_h264_scaling_matrix *p_h264_scal_mtrx;
+ struct v4l2_ctrl_h264_slice_param *p_h264_slice_param;
+ struct v4l2_ctrl_h264_decode_param *p_h264_decode_param;
void *p;
};
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 23da8bfa7e6f..ac307c59683c 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -50,6 +50,8 @@
#ifndef __LINUX_V4L2_CONTROLS_H
#define __LINUX_V4L2_CONTROLS_H
+#include <linux/types.h>
+
/* Control classes */
#define V4L2_CTRL_CLASS_USER 0x00980000 /* Old-style 'user' controls */
#define V4L2_CTRL_CLASS_MPEG 0x00990000 /* MPEG-compression controls */
@@ -531,6 +533,12 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type {
};
#define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER (V4L2_CID_MPEG_BASE+381)
#define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP (V4L2_CID_MPEG_BASE+382)
+#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+383)
+#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+384)
+#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+385)
+#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM (V4L2_CID_MPEG_BASE+386)
+#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM (V4L2_CID_MPEG_BASE+387)
+
#define V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP (V4L2_CID_MPEG_BASE+400)
#define V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP (V4L2_CID_MPEG_BASE+401)
#define V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP (V4L2_CID_MPEG_BASE+402)
@@ -1078,6 +1086,162 @@ enum v4l2_detect_md_mode {
#define V4L2_CID_DETECT_MD_THRESHOLD_GRID (V4L2_CID_DETECT_CLASS_BASE + 3)
#define V4L2_CID_DETECT_MD_REGION_GRID (V4L2_CID_DETECT_CLASS_BASE + 4)
+/* Complex controls */
+
+#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01
+#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02
+#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04
+#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08
+#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10
+#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20
+
+#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01
+#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02
+#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04
+#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08
+#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10
+#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20
+#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40
+struct v4l2_ctrl_h264_sps {
+ __u8 profile_idc;
+ __u8 constraint_set_flags;
+ __u8 level_idc;
+ __u8 seq_parameter_set_id;
+ __u8 chroma_format_idc;
+ __u8 bit_depth_luma_minus8;
+ __u8 bit_depth_chroma_minus8;
+ __u8 log2_max_frame_num_minus4;
+ __u8 pic_order_cnt_type;
+ __u8 log2_max_pic_order_cnt_lsb_minus4;
+ __s32 offset_for_non_ref_pic;
+ __s32 offset_for_top_to_bottom_field;
+ __u8 num_ref_frames_in_pic_order_cnt_cycle;
+ __s32 offset_for_ref_frame[255];
+ __u8 max_num_ref_frames;
+ __u16 pic_width_in_mbs_minus1;
+ __u16 pic_height_in_map_units_minus1;
+ __u8 flags;
+};
+
+#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001
+#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002
+#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004
+#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008
+#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010
+#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020
+#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040
+#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080
+struct v4l2_ctrl_h264_pps {
+ __u8 pic_parameter_set_id;
+ __u8 seq_parameter_set_id;
+ __u8 num_slice_groups_minus1;
+ __u8 num_ref_idx_l0_default_active_minus1;
+ __u8 num_ref_idx_l1_default_active_minus1;
+ __u8 weighted_bipred_idc;
+ __s8 pic_init_qp_minus26;
+ __s8 pic_init_qs_minus26;
+ __s8 chroma_qp_index_offset;
+ __s8 second_chroma_qp_index_offset;
+ __u8 flags;
+};
+
+struct v4l2_ctrl_h264_scaling_matrix {
+ __u8 scaling_list_4x4[6][16];
+ __u8 scaling_list_8x8[6][64];
+};
+
+struct v4l2_h264_weight_factors {
+ __s8 luma_weight[32];
+ __s8 luma_offset[32];
+ __s8 chroma_weight[32][2];
+ __s8 chroma_offset[32][2];
+};
+
+struct v4l2_h264_pred_weight_table {
+ __u8 luma_log2_weight_denom;
+ __u8 chroma_log2_weight_denom;
+ struct v4l2_h264_weight_factors weight_factors[2];
+};
+
+enum v4l2_h264_slice_type {
+ V4L2_H264_SLICE_TYPE_P = 0,
+ V4L2_H264_SLICE_TYPE_B = 1,
+ V4L2_H264_SLICE_TYPE_I = 2,
+ V4L2_H264_SLICE_TYPE_SP = 3,
+ V4L2_H264_SLICE_TYPE_SI = 4,
+};
+
+#define V4L2_SLICE_FLAG_FIELD_PIC 0x01
+#define V4L2_SLICE_FLAG_BOTTOM_FIELD 0x02
+#define V4L2_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04
+#define V4L2_SLICE_FLAG_SP_FOR_SWITCH 0x08
+struct v4l2_ctrl_h264_slice_param {
+ /* Size in bytes, including header */
+ __u32 size;
+ /* Offset in bits to slice_data() from the beginning of this slice. */
+ __u32 header_bit_size;
+
+ __u16 first_mb_in_slice;
+ enum v4l2_h264_slice_type slice_type;
+ __u8 pic_parameter_set_id;
+ __u8 colour_plane_id;
+ __u16 frame_num;
+ __u16 idr_pic_id;
+ __u16 pic_order_cnt_lsb;
+ __s32 delta_pic_order_cnt_bottom;
+ __s32 delta_pic_order_cnt0;
+ __s32 delta_pic_order_cnt1;
+ __u8 redundant_pic_cnt;
+
+ struct v4l2_h264_pred_weight_table pred_weight_table;
+ /* Size in bits of dec_ref_pic_marking() syntax element. */
+ __u32 dec_ref_pic_marking_bit_size;
+ /* Size in bits of pic order count syntax. */
+ __u32 pic_order_cnt_bit_size;
+
+ __u8 cabac_init_idc;
+ __s8 slice_qp_delta;
+ __s8 slice_qs_delta;
+ __u8 disable_deblocking_filter_idc;
+ __s8 slice_alpha_c0_offset_div2;
+ __s8 slice_beta_offset_div2;
+ __u32 slice_group_change_cycle;
+
+ __u8 num_ref_idx_l0_active_minus1;
+ __u8 num_ref_idx_l1_active_minus1;
+ /* Entries on each list are indices
+ * into v4l2_ctrl_h264_decode_param.dpb[]. */
+ __u8 ref_pic_list0[32];
+ __u8 ref_pic_list1[32];
+
+ __u8 flags;
+};
+
+/* If not set, this entry is unused for reference. */
+#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x01
+#define V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM 0x02
+struct v4l2_h264_dpb_entry {
+ __u32 buf_index; /* v4l2_buffer index */
+ __u16 frame_num;
+ __u16 pic_num;
+ /* Note that field is indicated by v4l2_buffer.field */
+ __s32 top_field_order_cnt;
+ __s32 bottom_field_order_cnt;
+ __u8 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
+};
+
+struct v4l2_ctrl_h264_decode_param {
+ __u32 num_slices;
+ __u8 idr_pic_flag;
+ __u8 nal_ref_idc;
+ __s32 top_field_order_cnt;
+ __s32 bottom_field_order_cnt;
+ __u8 ref_pic_list_p0[32];
+ __u8 ref_pic_list_b0[32];
+ __u8 ref_pic_list_b1[32];
+ struct v4l2_h264_dpb_entry dpb[16];
+};
+
struct v4l2_ctrl_mpeg2_frame_hdr {
__u32 slice_len;
__u32 slice_pos;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 242a6bfa1440..4b4a1b25a0db 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -626,6 +626,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
#define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
#define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
+#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
#define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */
#define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */
#define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */
@@ -1589,6 +1590,11 @@ struct v4l2_ext_control {
__u8 __user *p_u8;
__u16 __user *p_u16;
__u32 __user *p_u32;
+ struct v4l2_ctrl_h264_sps __user *p_h264_sps;
+ struct v4l2_ctrl_h264_pps __user *p_h264_pps;
+ struct v4l2_ctrl_h264_scaling_matrix __user *p_h264_scal_mtrx;
+ struct v4l2_ctrl_h264_slice_param __user *p_h264_slice_param;
+ struct v4l2_ctrl_h264_decode_param __user *p_h264_decode_param;
struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
void __user *ptr;
};
@@ -1635,6 +1641,11 @@ enum v4l2_ctrl_type {
V4L2_CTRL_TYPE_U8 = 0x0100,
V4L2_CTRL_TYPE_U16 = 0x0101,
V4L2_CTRL_TYPE_U32 = 0x0102,
+ V4L2_CTRL_TYPE_H264_SPS = 0x0103,
+ V4L2_CTRL_TYPE_H264_PPS = 0x0104,
+ V4L2_CTRL_TYPE_H264_SCALING_MATRIX = 0x0105,
+ V4L2_CTRL_TYPE_H264_SLICE_PARAM = 0x0106,
+ V4L2_CTRL_TYPE_H264_DECODE_PARAM = 0x0107,
V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
};
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 2/9] media: cedrus: Add wrappers around container_of for our buffers
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-13 14:07 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Maxime Ripard
The cedrus driver sub-classes the vb2_v4l2_buffer structure, but doesn't
provide any function to wrap around the proper container_of call that needs
to be duplicated in every calling site.
Add wrappers to make sure its opaque to the users and they don't have to
care anymore.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
.../platform/sunxi/cedrus/sunxi_cedrus_common.h | 12 ++++++++++++
.../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 4 ++--
2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index ee6883ef9cb7..b1ed1c8cb130 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -89,6 +89,18 @@ struct sunxi_cedrus_buffer {
struct list_head list;
};
+static inline
+struct sunxi_cedrus_buffer *vb2_v4l2_to_cedrus_buffer(const struct vb2_v4l2_buffer *p)
+{
+ return container_of(p, struct sunxi_cedrus_buffer, vb);
+}
+
+static inline
+struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
+{
+ return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
+}
+
struct sunxi_cedrus_dev {
struct v4l2_device v4l2_dev;
struct video_device vfd;
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
index 5783bd985855..fc688a5c1ea3 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
@@ -108,8 +108,8 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
return IRQ_HANDLED;
}
- src_buffer = container_of(src_vb, struct sunxi_cedrus_buffer, vb);
- dst_buffer = container_of(dst_vb, struct sunxi_cedrus_buffer, vb);
+ src_buffer = vb2_v4l2_to_cedrus_buffer(src_vb);
+ dst_buffer = vb2_v4l2_to_cedrus_buffer(dst_vb);
/* First bit of MPEG_STATUS indicates success. */
if (ctx->job_abort || !(status & 0x01))
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 2/9] media: cedrus: Add wrappers around container_of for our buffers
@ 2018-06-13 14:07 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: linux-arm-kernel
The cedrus driver sub-classes the vb2_v4l2_buffer structure, but doesn't
provide any function to wrap around the proper container_of call that needs
to be duplicated in every calling site.
Add wrappers to make sure its opaque to the users and they don't have to
care anymore.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
.../platform/sunxi/cedrus/sunxi_cedrus_common.h | 12 ++++++++++++
.../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 4 ++--
2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index ee6883ef9cb7..b1ed1c8cb130 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -89,6 +89,18 @@ struct sunxi_cedrus_buffer {
struct list_head list;
};
+static inline
+struct sunxi_cedrus_buffer *vb2_v4l2_to_cedrus_buffer(const struct vb2_v4l2_buffer *p)
+{
+ return container_of(p, struct sunxi_cedrus_buffer, vb);
+}
+
+static inline
+struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
+{
+ return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
+}
+
struct sunxi_cedrus_dev {
struct v4l2_device v4l2_dev;
struct video_device vfd;
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
index 5783bd985855..fc688a5c1ea3 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
@@ -108,8 +108,8 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
return IRQ_HANDLED;
}
- src_buffer = container_of(src_vb, struct sunxi_cedrus_buffer, vb);
- dst_buffer = container_of(dst_vb, struct sunxi_cedrus_buffer, vb);
+ src_buffer = vb2_v4l2_to_cedrus_buffer(src_vb);
+ dst_buffer = vb2_v4l2_to_cedrus_buffer(dst_vb);
/* First bit of MPEG_STATUS indicates success. */
if (ctx->job_abort || !(status & 0x01))
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 3/9] media: cedrus: Add a macro to check for the validity of a control
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-13 14:07 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Maxime Ripard
During our frame decoding setup, we need to check a number of controls to
make sure that they are properly filled before trying to access them.
It's not too bad with MPEG2 since there's just a single one, but with the
upcoming increase of codecs, and the integration of more complex codecs,
this logic will be duplicated a significant number of times. H264 for
example uses 4 different controls.
Add a macro that expands to the proper check in order to reduce the
duplication.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
.../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 16 +++++++++-------
1 file changed, 9 insertions(+), 7 deletions(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
index 8c92af34ebeb..c19acf9626c4 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
@@ -110,15 +110,16 @@ void sunxi_cedrus_device_run(void *priv)
spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+#define CHECK_CONTROL(ctx, ctrl) \
+ if (!ctx->ctrls[(ctrl)]) { \
+ v4l2_err(&(ctx)->dev->v4l2_dev, "Invalid " #ctrl " control\n"); \
+ (ctx)->job_abort = 1; \
+ goto unlock_complete; \
+ }
+
switch (ctx->vpu_src_fmt->fourcc) {
case V4L2_PIX_FMT_MPEG2_FRAME:
- if (!ctx->ctrls[SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR]) {
- v4l2_err(&ctx->dev->v4l2_dev,
- "Invalid MPEG2 frame header control\n");
- ctx->job_abort = 1;
- goto unlock_complete;
- }
-
+ CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
sunxi_cedrus_mpeg2_setup(ctx, &run);
@@ -128,6 +129,7 @@ void sunxi_cedrus_device_run(void *priv)
default:
ctx->job_abort = 1;
}
+#undef CHECK_CONTROL
unlock_complete:
spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 3/9] media: cedrus: Add a macro to check for the validity of a control
@ 2018-06-13 14:07 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: linux-arm-kernel
During our frame decoding setup, we need to check a number of controls to
make sure that they are properly filled before trying to access them.
It's not too bad with MPEG2 since there's just a single one, but with the
upcoming increase of codecs, and the integration of more complex codecs,
this logic will be duplicated a significant number of times. H264 for
example uses 4 different controls.
Add a macro that expands to the proper check in order to reduce the
duplication.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
.../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 16 +++++++++-------
1 file changed, 9 insertions(+), 7 deletions(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
index 8c92af34ebeb..c19acf9626c4 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
@@ -110,15 +110,16 @@ void sunxi_cedrus_device_run(void *priv)
spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+#define CHECK_CONTROL(ctx, ctrl) \
+ if (!ctx->ctrls[(ctrl)]) { \
+ v4l2_err(&(ctx)->dev->v4l2_dev, "Invalid " #ctrl " control\n"); \
+ (ctx)->job_abort = 1; \
+ goto unlock_complete; \
+ }
+
switch (ctx->vpu_src_fmt->fourcc) {
case V4L2_PIX_FMT_MPEG2_FRAME:
- if (!ctx->ctrls[SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR]) {
- v4l2_err(&ctx->dev->v4l2_dev,
- "Invalid MPEG2 frame header control\n");
- ctx->job_abort = 1;
- goto unlock_complete;
- }
-
+ CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
sunxi_cedrus_mpeg2_setup(ctx, &run);
@@ -128,6 +129,7 @@ void sunxi_cedrus_device_run(void *priv)
default:
ctx->job_abort = 1;
}
+#undef CHECK_CONTROL
unlock_complete:
spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 4/9] media: cedrus: make engine type more generic
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-13 14:07 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Maxime Ripard
The sunxi_cedrus_engine enum actually enumerates pretty much the codecs to
use (or we can easily infer the codec engine from the codec).
Since we will need the codec type as well in some later refactoring, make
that structure more useful by just enumerating the codec, and converting
the existing users.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h | 6 ++++++
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 6 +++---
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h | 6 +-----
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c | 2 +-
4 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index b1ed1c8cb130..a5f83c452006 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -61,6 +61,12 @@ struct sunxi_cedrus_run {
};
};
+enum sunxi_cedrus_codec {
+ SUNXI_CEDRUS_CODEC_MPEG2,
+
+ SUNXI_CEDRUS_CODEC_LAST,
+};
+
struct sunxi_cedrus_ctx {
struct v4l2_fh fh;
struct sunxi_cedrus_dev *dev;
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
index fc688a5c1ea3..bb46a01214e0 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
@@ -41,7 +41,7 @@
#define SYSCON_SRAM_C1_MAP_VE 0x7fffffff
int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
- enum sunxi_cedrus_engine engine)
+ enum sunxi_cedrus_codec codec)
{
u32 reg = 0;
@@ -53,8 +53,8 @@ int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
reg |= VE_CTRL_CACHE_BUS_BW_128;
- switch (engine) {
- case SUNXI_CEDRUS_ENGINE_MPEG:
+ switch (codec) {
+ case SUNXI_CEDRUS_CODEC_MPEG2:
reg |= VE_CTRL_DEC_MODE_MPEG;
break;
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
index 34f3fae462a8..3236c80bfcf4 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
@@ -23,12 +23,8 @@
#ifndef _SUNXI_CEDRUS_HW_H_
#define _SUNXI_CEDRUS_HW_H_
-enum sunxi_cedrus_engine {
- SUNXI_CEDRUS_ENGINE_MPEG,
-};
-
int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
- enum sunxi_cedrus_engine engine);
+ enum sunxi_cedrus_codec codec);
void sunxi_cedrus_engine_disable(struct sunxi_cedrus_dev *dev);
int sunxi_cedrus_hw_probe(struct sunxi_cedrus_dev *dev);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
index 5be3e3b9ceef..85e6fc2fbdb2 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
@@ -83,7 +83,7 @@ void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
}
/* Activate MPEG engine. */
- sunxi_cedrus_engine_enable(dev, SUNXI_CEDRUS_ENGINE_MPEG);
+ sunxi_cedrus_engine_enable(dev, SUNXI_CEDRUS_CODEC_MPEG2);
/* Set quantization matrices. */
for (i = 0; i < 64; i++) {
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 4/9] media: cedrus: make engine type more generic
@ 2018-06-13 14:07 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: linux-arm-kernel
The sunxi_cedrus_engine enum actually enumerates pretty much the codecs to
use (or we can easily infer the codec engine from the codec).
Since we will need the codec type as well in some later refactoring, make
that structure more useful by just enumerating the codec, and converting
the existing users.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h | 6 ++++++
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 6 +++---
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h | 6 +-----
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c | 2 +-
4 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index b1ed1c8cb130..a5f83c452006 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -61,6 +61,12 @@ struct sunxi_cedrus_run {
};
};
+enum sunxi_cedrus_codec {
+ SUNXI_CEDRUS_CODEC_MPEG2,
+
+ SUNXI_CEDRUS_CODEC_LAST,
+};
+
struct sunxi_cedrus_ctx {
struct v4l2_fh fh;
struct sunxi_cedrus_dev *dev;
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
index fc688a5c1ea3..bb46a01214e0 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
@@ -41,7 +41,7 @@
#define SYSCON_SRAM_C1_MAP_VE 0x7fffffff
int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
- enum sunxi_cedrus_engine engine)
+ enum sunxi_cedrus_codec codec)
{
u32 reg = 0;
@@ -53,8 +53,8 @@ int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
reg |= VE_CTRL_CACHE_BUS_BW_128;
- switch (engine) {
- case SUNXI_CEDRUS_ENGINE_MPEG:
+ switch (codec) {
+ case SUNXI_CEDRUS_CODEC_MPEG2:
reg |= VE_CTRL_DEC_MODE_MPEG;
break;
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
index 34f3fae462a8..3236c80bfcf4 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
@@ -23,12 +23,8 @@
#ifndef _SUNXI_CEDRUS_HW_H_
#define _SUNXI_CEDRUS_HW_H_
-enum sunxi_cedrus_engine {
- SUNXI_CEDRUS_ENGINE_MPEG,
-};
-
int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
- enum sunxi_cedrus_engine engine);
+ enum sunxi_cedrus_codec codec);
void sunxi_cedrus_engine_disable(struct sunxi_cedrus_dev *dev);
int sunxi_cedrus_hw_probe(struct sunxi_cedrus_dev *dev);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
index 5be3e3b9ceef..85e6fc2fbdb2 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
@@ -83,7 +83,7 @@ void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
}
/* Activate MPEG engine. */
- sunxi_cedrus_engine_enable(dev, SUNXI_CEDRUS_ENGINE_MPEG);
+ sunxi_cedrus_engine_enable(dev, SUNXI_CEDRUS_CODEC_MPEG2);
/* Set quantization matrices. */
for (i = 0; i < 64; i++) {
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 5/9] media: cedrus: Remove MPEG1 support
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-13 14:07 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Maxime Ripard
The MPEG1 codec is obsolete, and that code hasn't been tested either. Since
it stands in the way of further refactoring, remove it entirely.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 4 +---
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c | 8 ++------
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h | 2 +-
3 files changed, 4 insertions(+), 10 deletions(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
index c19acf9626c4..f274408ab5a7 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
@@ -80,7 +80,6 @@ void sunxi_cedrus_device_run(void *priv)
struct sunxi_cedrus_run run = { 0 };
struct media_request *src_req, *dst_req;
unsigned long flags;
- bool mpeg1 = false;
run.src = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
if (!run.src) {
@@ -123,7 +122,6 @@ void sunxi_cedrus_device_run(void *priv)
run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
sunxi_cedrus_mpeg2_setup(ctx, &run);
- mpeg1 = run.mpeg2.hdr->type == MPEG1;
break;
default:
@@ -146,7 +144,7 @@ void sunxi_cedrus_device_run(void *priv)
if (!ctx->job_abort) {
if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME)
- sunxi_cedrus_mpeg2_trigger(ctx, mpeg1);
+ sunxi_cedrus_mpeg2_trigger(ctx);
} else {
v4l2_m2m_buf_done(run.src, VB2_BUF_STATE_ERROR);
v4l2_m2m_buf_done(run.dst, VB2_BUF_STATE_ERROR);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
index 85e6fc2fbdb2..d1d7a3cfce0d 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
@@ -148,13 +148,9 @@ void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_MPEG_VLD_END);
}
-void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx, bool mpeg1)
+void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
{
struct sunxi_cedrus_dev *dev = ctx->dev;
- /* Trigger MPEG engine. */
- if (mpeg1)
- sunxi_cedrus_write(dev, VE_TRIG_MPEG1, VE_MPEG_TRIGGER);
- else
- sunxi_cedrus_write(dev, VE_TRIG_MPEG2, VE_MPEG_TRIGGER);
+ sunxi_cedrus_write(dev, VE_TRIG_MPEG2, VE_MPEG_TRIGGER);
}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
index b572001d47f2..4c380becfa1a 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
@@ -28,6 +28,6 @@ struct sunxi_cedrus_run;
void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
struct sunxi_cedrus_run *run);
-void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx, bool mpeg1);
+void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx);
#endif
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 5/9] media: cedrus: Remove MPEG1 support
@ 2018-06-13 14:07 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: linux-arm-kernel
The MPEG1 codec is obsolete, and that code hasn't been tested either. Since
it stands in the way of further refactoring, remove it entirely.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 4 +---
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c | 8 ++------
drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h | 2 +-
3 files changed, 4 insertions(+), 10 deletions(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
index c19acf9626c4..f274408ab5a7 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
@@ -80,7 +80,6 @@ void sunxi_cedrus_device_run(void *priv)
struct sunxi_cedrus_run run = { 0 };
struct media_request *src_req, *dst_req;
unsigned long flags;
- bool mpeg1 = false;
run.src = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
if (!run.src) {
@@ -123,7 +122,6 @@ void sunxi_cedrus_device_run(void *priv)
run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
sunxi_cedrus_mpeg2_setup(ctx, &run);
- mpeg1 = run.mpeg2.hdr->type == MPEG1;
break;
default:
@@ -146,7 +144,7 @@ void sunxi_cedrus_device_run(void *priv)
if (!ctx->job_abort) {
if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME)
- sunxi_cedrus_mpeg2_trigger(ctx, mpeg1);
+ sunxi_cedrus_mpeg2_trigger(ctx);
} else {
v4l2_m2m_buf_done(run.src, VB2_BUF_STATE_ERROR);
v4l2_m2m_buf_done(run.dst, VB2_BUF_STATE_ERROR);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
index 85e6fc2fbdb2..d1d7a3cfce0d 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
@@ -148,13 +148,9 @@ void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_MPEG_VLD_END);
}
-void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx, bool mpeg1)
+void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
{
struct sunxi_cedrus_dev *dev = ctx->dev;
- /* Trigger MPEG engine. */
- if (mpeg1)
- sunxi_cedrus_write(dev, VE_TRIG_MPEG1, VE_MPEG_TRIGGER);
- else
- sunxi_cedrus_write(dev, VE_TRIG_MPEG2, VE_MPEG_TRIGGER);
+ sunxi_cedrus_write(dev, VE_TRIG_MPEG2, VE_MPEG_TRIGGER);
}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
index b572001d47f2..4c380becfa1a 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
@@ -28,6 +28,6 @@ struct sunxi_cedrus_run;
void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
struct sunxi_cedrus_run *run);
-void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx, bool mpeg1);
+void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx);
#endif
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 6/9] media: cedrus: Add ops structure
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-13 14:07 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Maxime Ripard
In order to increase the number of codecs supported, we need to decouple
the MPEG2 only code that was there up until now and turn it into something
a bit more generic.
Do that by introducing an intermediate ops structure that would need to be
filled by each supported codec. Start by implementing in that structure the
setup and trigger hooks that are currently the only functions being
implemented by codecs support.
To do so, we need to store the current codec in use, which we do at
start_streaming time.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
.../platform/sunxi/cedrus/sunxi_cedrus.c | 2 ++
.../sunxi/cedrus/sunxi_cedrus_common.h | 11 +++++++
.../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 10 +++---
.../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 11 +++++--
.../sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 -------------------
.../sunxi/cedrus/sunxi_cedrus_video.c | 17 +++++++++-
6 files changed, 42 insertions(+), 42 deletions(-)
delete mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
index ccd41d9a3e41..bc80480f5dfd 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
@@ -244,6 +244,8 @@ static int sunxi_cedrus_probe(struct platform_device *pdev)
if (ret)
return ret;
+ dev->dec_ops[SUNXI_CEDRUS_CODEC_MPEG2] = &sunxi_cedrus_dec_ops_mpeg2;
+
ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
if (ret)
goto unreg_media;
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index a5f83c452006..c2e2c92d103b 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -75,6 +75,7 @@ struct sunxi_cedrus_ctx {
struct v4l2_pix_format_mplane src_fmt;
struct sunxi_cedrus_fmt *vpu_dst_fmt;
struct v4l2_pix_format_mplane dst_fmt;
+ enum sunxi_cedrus_codec current_codec;
struct v4l2_ctrl_handler hdl;
struct v4l2_ctrl *ctrls[SUNXI_CEDRUS_CTRL_MAX];
@@ -107,6 +108,14 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
}
+struct sunxi_cedrus_dec_ops {
+ void (*setup)(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run);
+ void (*trigger)(struct sunxi_cedrus_ctx *ctx);
+};
+
+extern struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2;
+
struct sunxi_cedrus_dev {
struct v4l2_device v4l2_dev;
struct video_device vfd;
@@ -130,6 +139,8 @@ struct sunxi_cedrus_dev {
struct reset_control *rstc;
struct regmap *syscon;
+
+ struct sunxi_cedrus_dec_ops *dec_ops[SUNXI_CEDRUS_CODEC_LAST];
};
static inline void sunxi_cedrus_write(struct sunxi_cedrus_dev *dev,
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
index f274408ab5a7..5e552fa05274 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
@@ -28,7 +28,6 @@
#include <media/v4l2-mem2mem.h>
#include "sunxi_cedrus_common.h"
-#include "sunxi_cedrus_mpeg2.h"
#include "sunxi_cedrus_dec.h"
#include "sunxi_cedrus_hw.h"
@@ -77,6 +76,7 @@ void sunxi_cedrus_device_work(struct work_struct *work)
void sunxi_cedrus_device_run(void *priv)
{
struct sunxi_cedrus_ctx *ctx = priv;
+ struct sunxi_cedrus_dev *dev = ctx->dev;
struct sunxi_cedrus_run run = { 0 };
struct media_request *src_req, *dst_req;
unsigned long flags;
@@ -120,8 +120,6 @@ void sunxi_cedrus_device_run(void *priv)
case V4L2_PIX_FMT_MPEG2_FRAME:
CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
- sunxi_cedrus_mpeg2_setup(ctx, &run);
-
break;
default:
@@ -129,6 +127,9 @@ void sunxi_cedrus_device_run(void *priv)
}
#undef CHECK_CONTROL
+ if (!ctx->job_abort)
+ dev->dec_ops[ctx->current_codec]->setup(ctx, &run);
+
unlock_complete:
spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
@@ -143,8 +144,7 @@ void sunxi_cedrus_device_run(void *priv)
spin_lock_irqsave(&ctx->dev->irq_lock, flags);
if (!ctx->job_abort) {
- if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME)
- sunxi_cedrus_mpeg2_trigger(ctx);
+ dev->dec_ops[ctx->current_codec]->trigger(ctx);
} else {
v4l2_m2m_buf_done(run.src, VB2_BUF_STATE_ERROR);
v4l2_m2m_buf_done(run.dst, VB2_BUF_STATE_ERROR);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
index d1d7a3cfce0d..e25075bb5779 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
@@ -52,8 +52,8 @@ static const u8 mpeg_default_non_intra_quant[64] = {
#define m_niq(i) ((i << 8) | mpeg_default_non_intra_quant[i])
-void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
- struct sunxi_cedrus_run *run)
+static void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
{
struct sunxi_cedrus_dev *dev = ctx->dev;
const struct v4l2_ctrl_mpeg2_frame_hdr *frame_hdr = run->mpeg2.hdr;
@@ -148,9 +148,14 @@ void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_MPEG_VLD_END);
}
-void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
+static void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
{
struct sunxi_cedrus_dev *dev = ctx->dev;
sunxi_cedrus_write(dev, VE_TRIG_MPEG2, VE_MPEG_TRIGGER);
}
+
+struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2 = {
+ .setup = sunxi_cedrus_mpeg2_setup,
+ .trigger = sunxi_cedrus_mpeg2_trigger,
+};
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
deleted file mode 100644
index 4c380becfa1a..000000000000
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
+++ /dev/null
@@ -1,33 +0,0 @@
-/*
- * Sunxi-Cedrus VPU driver
- *
- * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
- * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
- *
- * Based on the vim2m driver, that is:
- *
- * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
- * Pawel Osciak, <pawel@osciak.com>
- * Marek Szyprowski, <m.szyprowski@samsung.com>
- *
- * This software is licensed under the terms of the GNU General Public
- * License version 2, as published by the Free Software Foundation, and
- * may be copied, distributed, and modified under those terms.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- */
-
-#ifndef _SUNXI_CEDRUS_MPEG2_H_
-#define _SUNXI_CEDRUS_MPEG2_H_
-
-struct sunxi_cedrus_ctx;
-struct sunxi_cedrus_run;
-
-void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
- struct sunxi_cedrus_run *run);
-void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx);
-
-#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
index 089abfe6bfeb..fb7b081a5bb7 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
@@ -28,7 +28,6 @@
#include <media/v4l2-mem2mem.h>
#include "sunxi_cedrus_common.h"
-#include "sunxi_cedrus_mpeg2.h"
#include "sunxi_cedrus_dec.h"
#include "sunxi_cedrus_hw.h"
@@ -414,6 +413,21 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
return 0;
}
+static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
+
+ switch (ctx->vpu_src_fmt->fourcc) {
+ case V4L2_PIX_FMT_MPEG2_FRAME:
+ ctx->current_codec = SUNXI_CEDRUS_CODEC_MPEG2;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static void sunxi_cedrus_stop_streaming(struct vb2_queue *q)
{
struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
@@ -462,6 +476,7 @@ static struct vb2_ops sunxi_cedrus_qops = {
.buf_cleanup = sunxi_cedrus_buf_cleanup,
.buf_queue = sunxi_cedrus_buf_queue,
.buf_request_complete = sunxi_cedrus_buf_request_complete,
+ .start_streaming = sunxi_cedrus_start_streaming,
.stop_streaming = sunxi_cedrus_stop_streaming,
.wait_prepare = vb2_ops_wait_prepare,
.wait_finish = vb2_ops_wait_finish,
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 6/9] media: cedrus: Add ops structure
@ 2018-06-13 14:07 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: linux-arm-kernel
In order to increase the number of codecs supported, we need to decouple
the MPEG2 only code that was there up until now and turn it into something
a bit more generic.
Do that by introducing an intermediate ops structure that would need to be
filled by each supported codec. Start by implementing in that structure the
setup and trigger hooks that are currently the only functions being
implemented by codecs support.
To do so, we need to store the current codec in use, which we do at
start_streaming time.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
.../platform/sunxi/cedrus/sunxi_cedrus.c | 2 ++
.../sunxi/cedrus/sunxi_cedrus_common.h | 11 +++++++
.../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 10 +++---
.../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 11 +++++--
.../sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 -------------------
.../sunxi/cedrus/sunxi_cedrus_video.c | 17 +++++++++-
6 files changed, 42 insertions(+), 42 deletions(-)
delete mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
index ccd41d9a3e41..bc80480f5dfd 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
@@ -244,6 +244,8 @@ static int sunxi_cedrus_probe(struct platform_device *pdev)
if (ret)
return ret;
+ dev->dec_ops[SUNXI_CEDRUS_CODEC_MPEG2] = &sunxi_cedrus_dec_ops_mpeg2;
+
ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
if (ret)
goto unreg_media;
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index a5f83c452006..c2e2c92d103b 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -75,6 +75,7 @@ struct sunxi_cedrus_ctx {
struct v4l2_pix_format_mplane src_fmt;
struct sunxi_cedrus_fmt *vpu_dst_fmt;
struct v4l2_pix_format_mplane dst_fmt;
+ enum sunxi_cedrus_codec current_codec;
struct v4l2_ctrl_handler hdl;
struct v4l2_ctrl *ctrls[SUNXI_CEDRUS_CTRL_MAX];
@@ -107,6 +108,14 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
}
+struct sunxi_cedrus_dec_ops {
+ void (*setup)(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run);
+ void (*trigger)(struct sunxi_cedrus_ctx *ctx);
+};
+
+extern struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2;
+
struct sunxi_cedrus_dev {
struct v4l2_device v4l2_dev;
struct video_device vfd;
@@ -130,6 +139,8 @@ struct sunxi_cedrus_dev {
struct reset_control *rstc;
struct regmap *syscon;
+
+ struct sunxi_cedrus_dec_ops *dec_ops[SUNXI_CEDRUS_CODEC_LAST];
};
static inline void sunxi_cedrus_write(struct sunxi_cedrus_dev *dev,
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
index f274408ab5a7..5e552fa05274 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
@@ -28,7 +28,6 @@
#include <media/v4l2-mem2mem.h>
#include "sunxi_cedrus_common.h"
-#include "sunxi_cedrus_mpeg2.h"
#include "sunxi_cedrus_dec.h"
#include "sunxi_cedrus_hw.h"
@@ -77,6 +76,7 @@ void sunxi_cedrus_device_work(struct work_struct *work)
void sunxi_cedrus_device_run(void *priv)
{
struct sunxi_cedrus_ctx *ctx = priv;
+ struct sunxi_cedrus_dev *dev = ctx->dev;
struct sunxi_cedrus_run run = { 0 };
struct media_request *src_req, *dst_req;
unsigned long flags;
@@ -120,8 +120,6 @@ void sunxi_cedrus_device_run(void *priv)
case V4L2_PIX_FMT_MPEG2_FRAME:
CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
- sunxi_cedrus_mpeg2_setup(ctx, &run);
-
break;
default:
@@ -129,6 +127,9 @@ void sunxi_cedrus_device_run(void *priv)
}
#undef CHECK_CONTROL
+ if (!ctx->job_abort)
+ dev->dec_ops[ctx->current_codec]->setup(ctx, &run);
+
unlock_complete:
spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
@@ -143,8 +144,7 @@ void sunxi_cedrus_device_run(void *priv)
spin_lock_irqsave(&ctx->dev->irq_lock, flags);
if (!ctx->job_abort) {
- if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME)
- sunxi_cedrus_mpeg2_trigger(ctx);
+ dev->dec_ops[ctx->current_codec]->trigger(ctx);
} else {
v4l2_m2m_buf_done(run.src, VB2_BUF_STATE_ERROR);
v4l2_m2m_buf_done(run.dst, VB2_BUF_STATE_ERROR);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
index d1d7a3cfce0d..e25075bb5779 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
@@ -52,8 +52,8 @@ static const u8 mpeg_default_non_intra_quant[64] = {
#define m_niq(i) ((i << 8) | mpeg_default_non_intra_quant[i])
-void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
- struct sunxi_cedrus_run *run)
+static void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
{
struct sunxi_cedrus_dev *dev = ctx->dev;
const struct v4l2_ctrl_mpeg2_frame_hdr *frame_hdr = run->mpeg2.hdr;
@@ -148,9 +148,14 @@ void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_MPEG_VLD_END);
}
-void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
+static void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
{
struct sunxi_cedrus_dev *dev = ctx->dev;
sunxi_cedrus_write(dev, VE_TRIG_MPEG2, VE_MPEG_TRIGGER);
}
+
+struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2 = {
+ .setup = sunxi_cedrus_mpeg2_setup,
+ .trigger = sunxi_cedrus_mpeg2_trigger,
+};
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
deleted file mode 100644
index 4c380becfa1a..000000000000
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
+++ /dev/null
@@ -1,33 +0,0 @@
-/*
- * Sunxi-Cedrus VPU driver
- *
- * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
- * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
- *
- * Based on the vim2m driver, that is:
- *
- * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
- * Pawel Osciak, <pawel@osciak.com>
- * Marek Szyprowski, <m.szyprowski@samsung.com>
- *
- * This software is licensed under the terms of the GNU General Public
- * License version 2, as published by the Free Software Foundation, and
- * may be copied, distributed, and modified under those terms.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- */
-
-#ifndef _SUNXI_CEDRUS_MPEG2_H_
-#define _SUNXI_CEDRUS_MPEG2_H_
-
-struct sunxi_cedrus_ctx;
-struct sunxi_cedrus_run;
-
-void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
- struct sunxi_cedrus_run *run);
-void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx);
-
-#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
index 089abfe6bfeb..fb7b081a5bb7 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
@@ -28,7 +28,6 @@
#include <media/v4l2-mem2mem.h>
#include "sunxi_cedrus_common.h"
-#include "sunxi_cedrus_mpeg2.h"
#include "sunxi_cedrus_dec.h"
#include "sunxi_cedrus_hw.h"
@@ -414,6 +413,21 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
return 0;
}
+static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
+
+ switch (ctx->vpu_src_fmt->fourcc) {
+ case V4L2_PIX_FMT_MPEG2_FRAME:
+ ctx->current_codec = SUNXI_CEDRUS_CODEC_MPEG2;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static void sunxi_cedrus_stop_streaming(struct vb2_queue *q)
{
struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
@@ -462,6 +476,7 @@ static struct vb2_ops sunxi_cedrus_qops = {
.buf_cleanup = sunxi_cedrus_buf_cleanup,
.buf_queue = sunxi_cedrus_buf_queue,
.buf_request_complete = sunxi_cedrus_buf_request_complete,
+ .start_streaming = sunxi_cedrus_start_streaming,
.stop_streaming = sunxi_cedrus_stop_streaming,
.wait_prepare = vb2_ops_wait_prepare,
.wait_finish = vb2_ops_wait_finish,
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-13 14:07 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Maxime Ripard
The IRQ handler up until now was hardcoding the use of the MPEG engine to
read the interrupt status, clear it and disable the interrupts.
Obviously, that won't work really well with the introduction of new codecs
that use a separate engine with a separate register set.
In order to make this more future proof, introduce new decodec operations
to deal with the interrupt management. The only one missing is the one to
enable the interrupts in the first place, but that's taken care of by the
trigger hook for now.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
.../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
.../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
.../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
3 files changed, 53 insertions(+), 10 deletions(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index c2e2c92d103b..a2a507eb9fc9 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
}
+enum sunxi_cedrus_irq_status {
+ SUNXI_CEDRUS_IRQ_NONE,
+ SUNXI_CEDRUS_IRQ_ERROR,
+ SUNXI_CEDRUS_IRQ_OK,
+};
+
struct sunxi_cedrus_dec_ops {
+ void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
+ void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
+ enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
void (*setup)(struct sunxi_cedrus_ctx *ctx,
struct sunxi_cedrus_run *run);
void (*trigger)(struct sunxi_cedrus_ctx *ctx);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
index bb46a01214e0..6b97cbd2834e 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
@@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
struct sunxi_cedrus_ctx *ctx;
struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
struct vb2_v4l2_buffer *src_vb, *dst_vb;
+ enum sunxi_cedrus_irq_status status;
unsigned long flags;
- unsigned int value, status;
spin_lock_irqsave(&dev->irq_lock, flags);
- /* Disable MPEG interrupts and stop the MPEG engine */
- value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
- sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
-
- status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
- sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
- sunxi_cedrus_engine_disable(dev);
-
ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
if (!ctx) {
pr_err("Instance released before the end of transaction\n");
spin_unlock_irqrestore(&dev->irq_lock, flags);
- return IRQ_HANDLED;
+ return IRQ_NONE;
}
+ status = dev->dec_ops[ctx->current_codec]->irq_status(ctx);
+ if (status == SUNXI_CEDRUS_IRQ_NONE) {
+ spin_unlock_irqrestore(&dev->irq_lock, flags);
+ return IRQ_NONE;
+ }
+
+ dev->dec_ops[ctx->current_codec]->irq_disable(ctx);
+ dev->dec_ops[ctx->current_codec]->irq_clear(ctx);
+
src_vb = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
index e25075bb5779..51fa0c0f9bf2 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
@@ -52,6 +52,36 @@ static const u8 mpeg_default_non_intra_quant[64] = {
#define m_niq(i) ((i << 8) | mpeg_default_non_intra_quant[i])
+static enum sunxi_cedrus_irq_status
+sunxi_cedrus_mpeg2_irq_status(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ u32 reg = sunxi_cedrus_read(dev, VE_MPEG_STATUS) & 0x7;
+
+ if (!reg)
+ return SUNXI_CEDRUS_IRQ_NONE;
+
+ if (reg & (BIT(1) | BIT(2)))
+ return SUNXI_CEDRUS_IRQ_ERROR;
+
+ return SUNXI_CEDRUS_IRQ_OK;
+}
+
+static void sunxi_cedrus_mpeg2_irq_clear(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+
+ sunxi_cedrus_write(dev, GENMASK(2, 0), VE_MPEG_STATUS);
+}
+
+static void sunxi_cedrus_mpeg2_irq_disable(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ u32 reg = sunxi_cedrus_read(dev, VE_MPEG_CTRL) & ~BIT(3);
+
+ sunxi_cedrus_write(dev, reg, VE_MPEG_CTRL);
+}
+
static void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
struct sunxi_cedrus_run *run)
{
@@ -156,6 +186,9 @@ static void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
}
struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2 = {
+ .irq_clear = sunxi_cedrus_mpeg2_irq_clear,
+ .irq_disable = sunxi_cedrus_mpeg2_irq_disable,
+ .irq_status = sunxi_cedrus_mpeg2_irq_status,
.setup = sunxi_cedrus_mpeg2_setup,
.trigger = sunxi_cedrus_mpeg2_trigger,
};
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
@ 2018-06-13 14:07 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: linux-arm-kernel
The IRQ handler up until now was hardcoding the use of the MPEG engine to
read the interrupt status, clear it and disable the interrupts.
Obviously, that won't work really well with the introduction of new codecs
that use a separate engine with a separate register set.
In order to make this more future proof, introduce new decodec operations
to deal with the interrupt management. The only one missing is the one to
enable the interrupts in the first place, but that's taken care of by the
trigger hook for now.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
.../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
.../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
.../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
3 files changed, 53 insertions(+), 10 deletions(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index c2e2c92d103b..a2a507eb9fc9 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
}
+enum sunxi_cedrus_irq_status {
+ SUNXI_CEDRUS_IRQ_NONE,
+ SUNXI_CEDRUS_IRQ_ERROR,
+ SUNXI_CEDRUS_IRQ_OK,
+};
+
struct sunxi_cedrus_dec_ops {
+ void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
+ void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
+ enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
void (*setup)(struct sunxi_cedrus_ctx *ctx,
struct sunxi_cedrus_run *run);
void (*trigger)(struct sunxi_cedrus_ctx *ctx);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
index bb46a01214e0..6b97cbd2834e 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
@@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
struct sunxi_cedrus_ctx *ctx;
struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
struct vb2_v4l2_buffer *src_vb, *dst_vb;
+ enum sunxi_cedrus_irq_status status;
unsigned long flags;
- unsigned int value, status;
spin_lock_irqsave(&dev->irq_lock, flags);
- /* Disable MPEG interrupts and stop the MPEG engine */
- value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
- sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
-
- status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
- sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
- sunxi_cedrus_engine_disable(dev);
-
ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
if (!ctx) {
pr_err("Instance released before the end of transaction\n");
spin_unlock_irqrestore(&dev->irq_lock, flags);
- return IRQ_HANDLED;
+ return IRQ_NONE;
}
+ status = dev->dec_ops[ctx->current_codec]->irq_status(ctx);
+ if (status == SUNXI_CEDRUS_IRQ_NONE) {
+ spin_unlock_irqrestore(&dev->irq_lock, flags);
+ return IRQ_NONE;
+ }
+
+ dev->dec_ops[ctx->current_codec]->irq_disable(ctx);
+ dev->dec_ops[ctx->current_codec]->irq_clear(ctx);
+
src_vb = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
index e25075bb5779..51fa0c0f9bf2 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
@@ -52,6 +52,36 @@ static const u8 mpeg_default_non_intra_quant[64] = {
#define m_niq(i) ((i << 8) | mpeg_default_non_intra_quant[i])
+static enum sunxi_cedrus_irq_status
+sunxi_cedrus_mpeg2_irq_status(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ u32 reg = sunxi_cedrus_read(dev, VE_MPEG_STATUS) & 0x7;
+
+ if (!reg)
+ return SUNXI_CEDRUS_IRQ_NONE;
+
+ if (reg & (BIT(1) | BIT(2)))
+ return SUNXI_CEDRUS_IRQ_ERROR;
+
+ return SUNXI_CEDRUS_IRQ_OK;
+}
+
+static void sunxi_cedrus_mpeg2_irq_clear(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+
+ sunxi_cedrus_write(dev, GENMASK(2, 0), VE_MPEG_STATUS);
+}
+
+static void sunxi_cedrus_mpeg2_irq_disable(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ u32 reg = sunxi_cedrus_read(dev, VE_MPEG_CTRL) & ~BIT(3);
+
+ sunxi_cedrus_write(dev, reg, VE_MPEG_CTRL);
+}
+
static void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
struct sunxi_cedrus_run *run)
{
@@ -156,6 +186,9 @@ static void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
}
struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2 = {
+ .irq_clear = sunxi_cedrus_mpeg2_irq_clear,
+ .irq_disable = sunxi_cedrus_mpeg2_irq_disable,
+ .irq_status = sunxi_cedrus_mpeg2_irq_status,
.setup = sunxi_cedrus_mpeg2_setup,
.trigger = sunxi_cedrus_mpeg2_trigger,
};
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 8/9] media: cedrus: Add start and stop decoder operations
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-13 14:07 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Maxime Ripard
Some codec needs to perform some additional task when a decoding is started
and stopped, and not only at every frame.
For example, the H264 decoding support needs to allocate buffers that will
be used in the decoding process, but do not need to change over time, or at
each frame.
In order to allow that for codecs, introduce a start and stop hook that
will be called if present at start_streaming and stop_streaming time.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
.../platform/sunxi/cedrus/sunxi_cedrus_common.h | 2 ++
.../platform/sunxi/cedrus/sunxi_cedrus_video.c | 14 +++++++++++++-
2 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index a2a507eb9fc9..20c78ec1f037 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -120,6 +120,8 @@ struct sunxi_cedrus_dec_ops {
enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
void (*setup)(struct sunxi_cedrus_ctx *ctx,
struct sunxi_cedrus_run *run);
+ int (*start)(struct sunxi_cedrus_ctx *ctx);
+ void (*stop)(struct sunxi_cedrus_ctx *ctx);
void (*trigger)(struct sunxi_cedrus_ctx *ctx);
};
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
index fb7b081a5bb7..d93461178857 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
@@ -416,6 +416,8 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
{
struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ int ret = 0;
switch (ctx->vpu_src_fmt->fourcc) {
case V4L2_PIX_FMT_MPEG2_FRAME:
@@ -425,16 +427,26 @@ static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
return -EINVAL;
}
- return 0;
+ if (V4L2_TYPE_IS_OUTPUT(q->type) &&
+ dev->dec_ops[ctx->current_codec]->start)
+ ret = dev->dec_ops[ctx->current_codec]->start(ctx);
+
+ return ret;
}
static void sunxi_cedrus_stop_streaming(struct vb2_queue *q)
{
struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
+ struct sunxi_cedrus_dev *dev = ctx->dev;
struct vb2_v4l2_buffer *vbuf;
unsigned long flags;
flush_scheduled_work();
+
+ if (V4L2_TYPE_IS_OUTPUT(q->type) &&
+ dev->dec_ops[ctx->current_codec]->stop)
+ dev->dec_ops[ctx->current_codec]->stop(ctx);
+
for (;;) {
spin_lock_irqsave(&ctx->dev->irq_lock, flags);
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 8/9] media: cedrus: Add start and stop decoder operations
@ 2018-06-13 14:07 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: linux-arm-kernel
Some codec needs to perform some additional task when a decoding is started
and stopped, and not only at every frame.
For example, the H264 decoding support needs to allocate buffers that will
be used in the decoding process, but do not need to change over time, or at
each frame.
In order to allow that for codecs, introduce a start and stop hook that
will be called if present at start_streaming and stop_streaming time.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
.../platform/sunxi/cedrus/sunxi_cedrus_common.h | 2 ++
.../platform/sunxi/cedrus/sunxi_cedrus_video.c | 14 +++++++++++++-
2 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index a2a507eb9fc9..20c78ec1f037 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -120,6 +120,8 @@ struct sunxi_cedrus_dec_ops {
enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
void (*setup)(struct sunxi_cedrus_ctx *ctx,
struct sunxi_cedrus_run *run);
+ int (*start)(struct sunxi_cedrus_ctx *ctx);
+ void (*stop)(struct sunxi_cedrus_ctx *ctx);
void (*trigger)(struct sunxi_cedrus_ctx *ctx);
};
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
index fb7b081a5bb7..d93461178857 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
@@ -416,6 +416,8 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
{
struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ int ret = 0;
switch (ctx->vpu_src_fmt->fourcc) {
case V4L2_PIX_FMT_MPEG2_FRAME:
@@ -425,16 +427,26 @@ static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
return -EINVAL;
}
- return 0;
+ if (V4L2_TYPE_IS_OUTPUT(q->type) &&
+ dev->dec_ops[ctx->current_codec]->start)
+ ret = dev->dec_ops[ctx->current_codec]->start(ctx);
+
+ return ret;
}
static void sunxi_cedrus_stop_streaming(struct vb2_queue *q)
{
struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
+ struct sunxi_cedrus_dev *dev = ctx->dev;
struct vb2_v4l2_buffer *vbuf;
unsigned long flags;
flush_scheduled_work();
+
+ if (V4L2_TYPE_IS_OUTPUT(q->type) &&
+ dev->dec_ops[ctx->current_codec]->stop)
+ dev->dec_ops[ctx->current_codec]->stop(ctx);
+
for (;;) {
spin_lock_irqsave(&ctx->dev->irq_lock, flags);
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 9/9] media: cedrus: Add H264 decoding support
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-13 14:07 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Maxime Ripard
Introduce some basic H264 decoding support in cedrus. So far, only the
baseline profile videos have been tested, and some more advanced features
used in higher profiles are not even implemented.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
drivers/media/platform/sunxi/cedrus/Makefile | 2 +-
.../platform/sunxi/cedrus/sunxi_cedrus.c | 21 +
.../sunxi/cedrus/sunxi_cedrus_common.h | 41 +-
.../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 12 +
.../platform/sunxi/cedrus/sunxi_cedrus_h264.c | 443 ++++++++++++++++++
.../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 4 +
.../platform/sunxi/cedrus/sunxi_cedrus_regs.h | 22 +-
.../sunxi/cedrus/sunxi_cedrus_video.c | 8 +
8 files changed, 542 insertions(+), 11 deletions(-)
create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_h264.c
diff --git a/drivers/media/platform/sunxi/cedrus/Makefile b/drivers/media/platform/sunxi/cedrus/Makefile
index 98f30df626a9..715f4f67b743 100644
--- a/drivers/media/platform/sunxi/cedrus/Makefile
+++ b/drivers/media/platform/sunxi/cedrus/Makefile
@@ -1,4 +1,4 @@
obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o
sunxi-cedrus-y = sunxi_cedrus.o sunxi_cedrus_video.o sunxi_cedrus_hw.o \
- sunxi_cedrus_dec.o sunxi_cedrus_mpeg2.o
+ sunxi_cedrus_dec.o sunxi_cedrus_mpeg2.o sunxi_cedrus_h264.o
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
index bc80480f5dfd..581a99ba00c8 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
@@ -41,6 +41,10 @@ static int sunxi_cedrus_s_ctrl(struct v4l2_ctrl *ctrl)
container_of(ctrl->handler, struct sunxi_cedrus_ctx, hdl);
switch (ctrl->id) {
+ case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM:
+ case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM:
+ case V4L2_CID_MPEG_VIDEO_H264_SPS:
+ case V4L2_CID_MPEG_VIDEO_H264_PPS:
case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
/* This is kept in memory and used directly. */
break;
@@ -57,6 +61,22 @@ static const struct v4l2_ctrl_ops sunxi_cedrus_ctrl_ops = {
};
static const struct sunxi_cedrus_control controls[] = {
+ [SUNXI_CEDRUS_CTRL_DEC_H264_DECODE_PARAM] = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM,
+ .elem_size = sizeof(struct v4l2_ctrl_h264_decode_param),
+ },
+ [SUNXI_CEDRUS_CTRL_DEC_H264_SLICE_PARAM] = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM,
+ .elem_size = sizeof(struct v4l2_ctrl_h264_slice_param),
+ },
+ [SUNXI_CEDRUS_CTRL_DEC_H264_SPS] = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_SPS,
+ .elem_size = sizeof(struct v4l2_ctrl_h264_sps),
+ },
+ [SUNXI_CEDRUS_CTRL_DEC_H264_PPS] = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_PPS,
+ .elem_size = sizeof(struct v4l2_ctrl_h264_pps),
+ },
[SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR] = {
.id = V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR,
.elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr),
@@ -244,6 +264,7 @@ static int sunxi_cedrus_probe(struct platform_device *pdev)
if (ret)
return ret;
+ dev->dec_ops[SUNXI_CEDRUS_CODEC_H264] = &sunxi_cedrus_dec_ops_h264;
dev->dec_ops[SUNXI_CEDRUS_CODEC_MPEG2] = &sunxi_cedrus_dec_ops_mpeg2;
ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index 20c78ec1f037..1ab06ee68ce6 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -32,7 +32,11 @@
#define SUNXI_CEDRUS_NAME "sunxi-cedrus"
enum sunxi_cedrus_control_id {
- SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR = 0,
+ SUNXI_CEDRUS_CTRL_DEC_H264_DECODE_PARAM = 0,
+ SUNXI_CEDRUS_CTRL_DEC_H264_PPS,
+ SUNXI_CEDRUS_CTRL_DEC_H264_SLICE_PARAM,
+ SUNXI_CEDRUS_CTRL_DEC_H264_SPS,
+ SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR,
SUNXI_CEDRUS_CTRL_MAX,
};
@@ -48,6 +52,13 @@ struct sunxi_cedrus_fmt {
unsigned int num_planes;
};
+struct sunxi_cedrus_h264_run {
+ const struct v4l2_ctrl_h264_decode_param *decode_param;
+ const struct v4l2_ctrl_h264_pps *pps;
+ const struct v4l2_ctrl_h264_slice_param *slice_param;
+ const struct v4l2_ctrl_h264_sps *sps;
+};
+
struct sunxi_cedrus_mpeg2_run {
const struct v4l2_ctrl_mpeg2_frame_hdr *hdr;
};
@@ -57,12 +68,14 @@ struct sunxi_cedrus_run {
struct vb2_v4l2_buffer *dst;
union {
+ struct sunxi_cedrus_h264_run h264;
struct sunxi_cedrus_mpeg2_run mpeg2;
};
};
enum sunxi_cedrus_codec {
SUNXI_CEDRUS_CODEC_MPEG2,
+ SUNXI_CEDRUS_CODEC_H264,
SUNXI_CEDRUS_CODEC_LAST,
};
@@ -88,12 +101,37 @@ struct sunxi_cedrus_ctx {
struct work_struct run_work;
struct list_head src_list;
struct list_head dst_list;
+
+ union {
+ struct {
+ void *mv_col_buf;
+ dma_addr_t mv_col_buf_dma;
+ ssize_t mv_col_buf_size;
+ void *neighbor_info_buf;
+ dma_addr_t neighbor_info_buf_dma;
+ void *pic_info_buf;
+ dma_addr_t pic_info_buf_dma;
+ } h264;
+ } codec;
+};
+
+enum sunxi_cedrus_h264_pic_type {
+ SUNXI_CEDRUS_H264_PIC_TYPE_FRAME = 0,
+ SUNXI_CEDRUS_H264_PIC_TYPE_FIELD,
+ SUNXI_CEDRUS_H264_PIC_TYPE_MBAFF,
};
struct sunxi_cedrus_buffer {
struct vb2_v4l2_buffer vb;
enum vb2_buffer_state state;
struct list_head list;
+
+ union {
+ struct {
+ unsigned int position;
+ enum sunxi_cedrus_h264_pic_type pic_type;
+ } h264;
+ } codec;
};
static inline
@@ -125,6 +163,7 @@ struct sunxi_cedrus_dec_ops {
void (*trigger)(struct sunxi_cedrus_ctx *ctx);
};
+extern struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_h264;
extern struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2;
struct sunxi_cedrus_dev {
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
index 5e552fa05274..c29a2582ed68 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
@@ -122,6 +122,18 @@ void sunxi_cedrus_device_run(void *priv)
run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
break;
+ case V4L2_PIX_FMT_H264_SLICE:
+ CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_DECODE_PARAM);
+ CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_SLICE_PARAM);
+ CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_SPS);
+ CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_PPS);
+
+ run.h264.decode_param = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_DECODE_PARAM);
+ run.h264.pps = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_PPS);
+ run.h264.slice_param = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_SLICE_PARAM);
+ run.h264.sps = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_SPS);
+ break;
+
default:
ctx->job_abort = 1;
}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_h264.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_h264.c
new file mode 100644
index 000000000000..0c86f66eb614
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_h264.c
@@ -0,0 +1,443 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2013 Jens Kuske <jenskuske@gmail.com>
+ * Copyright (c) 2018 Bootlin
+ */
+
+#include <linux/types.h>
+
+#include <media/videobuf2-dma-contig.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_hw.h"
+#include "sunxi_cedrus_regs.h"
+
+enum sunxi_cedrus_h264_sram_off {
+ SUNXI_CEDRUS_SRAM_H264_PRED_WEIGHT_TABLE = 0x000,
+ SUNXI_CEDRUS_SRAM_H264_FRAMEBUFFER_LIST = 0x100,
+ SUNXI_CEDRUS_SRAM_H264_REF_LIST_0 = 0x190,
+ SUNXI_CEDRUS_SRAM_H264_REF_LIST_1 = 0x199,
+ SUNXI_CEDRUS_SRAM_H264_SCALING_LIST_8x8 = 0x200,
+ SUNXI_CEDRUS_SRAM_H264_SCALING_LIST_4x4 = 0x218,
+};
+
+struct sunxi_cedrus_h264_sram_ref_pic {
+ __le32 top_field_order_cnt;
+ __le32 bottom_field_order_cnt;
+ __le32 frame_info;
+ __le32 luma_ptr;
+ __le32 chroma_ptr;
+ __le32 extra_data_ptr;
+ __le32 extra_data_end;
+ __le32 reserved;
+} __packed;
+
+/* One for the output, 16 for the reference images */
+#define SUNXI_CEDRUS_H264_FRAME_NUM 17
+
+#define SUNXI_CEDRUS_PIC_INFO_BUF_SIZE (128 * SZ_1K)
+#define SUNXI_CEDRUS_NEIGHBOR_INFO_BUF_SIZE (16 * SZ_1K)
+
+static void sunxi_cedrus_h264_write_sram(struct sunxi_cedrus_dev *dev,
+ enum sunxi_cedrus_h264_sram_off off,
+ const void *data, size_t len)
+{
+ const u32 *buffer = data;
+ size_t count = DIV_ROUND_UP(len, 4);
+
+ sunxi_cedrus_write(dev, off << 2,
+ VE_AVC_SRAM_PORT_OFFSET);
+
+ do {
+ sunxi_cedrus_write(dev, *buffer++, VE_AVC_SRAM_PORT_DATA);
+ } while (--count);
+}
+
+static void sunxi_cedrus_fill_ref_pic(struct sunxi_cedrus_h264_sram_ref_pic *pic,
+ struct vb2_buffer *buf,
+ dma_addr_t extra_buf,
+ size_t extra_buf_len,
+ unsigned int top_field_order_cnt,
+ unsigned int bottom_field_order_cnt,
+ enum sunxi_cedrus_h264_pic_type pic_type)
+{
+ pic->top_field_order_cnt = top_field_order_cnt;
+ pic->bottom_field_order_cnt = bottom_field_order_cnt;
+ pic->frame_info = pic_type << 8;
+ pic->luma_ptr = vb2_dma_contig_plane_dma_addr(buf, 0) - PHYS_OFFSET;
+ pic->chroma_ptr = vb2_dma_contig_plane_dma_addr(buf, 1) - PHYS_OFFSET;
+ pic->extra_data_ptr = extra_buf - PHYS_OFFSET;
+ pic->extra_data_end = (extra_buf - PHYS_OFFSET) + extra_buf_len;
+}
+
+static void sunxi_cedrus_write_frame_list(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
+{
+ struct sunxi_cedrus_h264_sram_ref_pic pic_list[SUNXI_CEDRUS_H264_FRAME_NUM];
+ const struct v4l2_ctrl_h264_decode_param *dec_param = run->h264.decode_param;
+ const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
+ const struct v4l2_ctrl_h264_sps *sps = run->h264.sps;
+ struct sunxi_cedrus_buffer *output_buf;
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ unsigned long used_dpbs = 0;
+ unsigned int position;
+ unsigned int output = 0;
+ unsigned int i;
+
+ memset(pic_list, 0, sizeof(pic_list));
+
+ for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) {
+ const struct v4l2_h264_dpb_entry *dpb = &dec_param->dpb[i];
+ const struct sunxi_cedrus_buffer *cedrus_buf;
+ struct vb2_buffer *ref_buf;
+
+ if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
+ continue;
+
+ ref_buf = ctx->dst_bufs[dpb->buf_index];
+ cedrus_buf = vb2_to_cedrus_buffer(ref_buf);
+ position = cedrus_buf->codec.h264.position;
+ used_dpbs |= BIT(position);
+
+ sunxi_cedrus_fill_ref_pic(&pic_list[position], ref_buf,
+ ctx->codec.h264.mv_col_buf_dma,
+ ctx->codec.h264.mv_col_buf_size,
+ dpb->top_field_order_cnt,
+ dpb->bottom_field_order_cnt,
+ cedrus_buf->codec.h264.pic_type);
+
+ output = max(position, output);
+ }
+
+ position = find_next_zero_bit(&used_dpbs, 17, output);
+ if (position >= 17)
+ position = find_first_zero_bit(&used_dpbs, 17);
+
+ output_buf = vb2_to_cedrus_buffer(&run->dst->vb2_buf);
+ output_buf->codec.h264.position = position;
+
+ if (slice->flags & V4L2_SLICE_FLAG_FIELD_PIC)
+ output_buf->codec.h264.pic_type = SUNXI_CEDRUS_H264_PIC_TYPE_FIELD;
+ else if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)
+ output_buf->codec.h264.pic_type = SUNXI_CEDRUS_H264_PIC_TYPE_MBAFF;
+ else
+ output_buf->codec.h264.pic_type = SUNXI_CEDRUS_H264_PIC_TYPE_FRAME;
+
+ sunxi_cedrus_fill_ref_pic(&pic_list[position], &run->dst->vb2_buf,
+ ctx->codec.h264.mv_col_buf_dma,
+ ctx->codec.h264.mv_col_buf_size,
+ dec_param->top_field_order_cnt,
+ dec_param->bottom_field_order_cnt,
+ output_buf->codec.h264.pic_type);
+
+ sunxi_cedrus_h264_write_sram(dev, SUNXI_CEDRUS_SRAM_H264_FRAMEBUFFER_LIST,
+ pic_list, sizeof(pic_list));
+
+ sunxi_cedrus_write(dev, position, VE_H264_OUTPUT_FRAME_IDX);
+}
+
+#define SUNXI_CEDRUS_MAX_REF_IDX 32
+
+static void _sunxi_cedrus_write_ref_list(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run,
+ const u8 *ref_list, u8 num_ref,
+ enum sunxi_cedrus_h264_sram_off sram)
+{
+ const struct v4l2_ctrl_h264_decode_param *decode = run->h264.decode_param;
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ u32 sram_array[SUNXI_CEDRUS_MAX_REF_IDX / sizeof(u32)];
+ unsigned int size, i;
+
+ memset(sram_array, 0, sizeof(sram_array));
+
+ for (i = 0; i < num_ref; i += 4) {
+ unsigned int j;
+
+ for (j = 0; j < 4; j++) {
+ const struct v4l2_h264_dpb_entry *dpb;
+ const struct sunxi_cedrus_buffer *cedrus_buf;
+ const struct vb2_v4l2_buffer *ref_buf;
+ unsigned int position;
+ u8 ref_idx = i + j;
+ u8 dpb_idx;
+
+ if (ref_idx >= num_ref)
+ break;
+
+ dpb_idx = ref_list[ref_idx];
+ dpb = &decode->dpb[dpb_idx];
+
+ if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
+ continue;
+
+ ref_buf = to_vb2_v4l2_buffer(ctx->dst_bufs[dpb->buf_index]);
+ cedrus_buf = vb2_v4l2_to_cedrus_buffer(ref_buf);
+ position = cedrus_buf->codec.h264.position;
+
+ sram_array[i] |= position << (j * 8 + 1);
+ if (ref_buf->field == V4L2_FIELD_BOTTOM)
+ sram_array[i] |= BIT(j * 8);
+ }
+ }
+
+ size = min((unsigned int)ALIGN(num_ref, 4), sizeof(sram_array));
+ sunxi_cedrus_h264_write_sram(dev, sram, &sram_array, size);
+}
+
+static void sunxi_cedrus_write_ref_list0(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
+{
+ const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
+
+ _sunxi_cedrus_write_ref_list(ctx, run,
+ slice->ref_pic_list0,
+ slice->num_ref_idx_l0_active_minus1 + 1,
+ SUNXI_CEDRUS_SRAM_H264_REF_LIST_0);
+}
+
+static void sunxi_cedrus_write_ref_list1(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
+{
+ const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
+
+ _sunxi_cedrus_write_ref_list(ctx, run,
+ slice->ref_pic_list1,
+ slice->num_ref_idx_l1_active_minus1 + 1,
+ SUNXI_CEDRUS_SRAM_H264_REF_LIST_1);
+}
+
+static void sunxi_cedrus_set_params(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
+{
+ const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
+ const struct v4l2_ctrl_h264_pps *pps = run->h264.pps;
+ const struct v4l2_ctrl_h264_sps *sps = run->h264.sps;
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ dma_addr_t src_buf_addr;
+ u32 offset = slice->header_bit_size;
+ u32 len = (slice->size * 8) - offset;
+ u32 reg;
+
+ sunxi_cedrus_write(dev, ctx->codec.h264.pic_info_buf_dma - PHYS_OFFSET, 0x250);
+ sunxi_cedrus_write(dev, (ctx->codec.h264.pic_info_buf_dma - PHYS_OFFSET) + 0x48000, 0x254);
+
+ sunxi_cedrus_write(dev, len, VE_H264_VLD_LEN);
+ sunxi_cedrus_write(dev, offset, VE_H264_VLD_OFFSET);
+
+ src_buf_addr = vb2_dma_contig_plane_dma_addr(&run->src->vb2_buf, 0);
+ src_buf_addr -= PHYS_OFFSET;
+ sunxi_cedrus_write(dev, VE_H264_VLD_ADDR_VAL(src_buf_addr) |
+ VE_H264_VLD_ADDR_FIRST | VE_H264_VLD_ADDR_VALID | VE_H264_VLD_ADDR_LAST,
+ VE_H264_VLD_ADDR);
+ sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_H264_VLD_END);
+
+ sunxi_cedrus_write(dev, VE_H264_TRIGGER_TYPE_INIT_SWDEC,
+ VE_H264_TRIGGER_TYPE);
+
+ if ((slice->slice_type == V4L2_H264_SLICE_TYPE_P) ||
+ (slice->slice_type == V4L2_H264_SLICE_TYPE_SP) ||
+ (slice->slice_type == V4L2_H264_SLICE_TYPE_B))
+ sunxi_cedrus_write_ref_list0(ctx, run);
+
+ if (slice->slice_type == V4L2_H264_SLICE_TYPE_B)
+ sunxi_cedrus_write_ref_list1(ctx, run);
+
+ // picture parameters
+ reg = 0;
+ /*
+ * FIXME: the kernel headers are allowing the default value to
+ * be passed, but the libva doesn't give us that.
+ */
+ reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 10;
+ reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 5;
+ reg |= (pps->weighted_bipred_idc & 0x3) << 2;
+ if (pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE)
+ reg |= BIT(15);
+ if (pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED)
+ reg |= BIT(4);
+ if (pps->flags & V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED)
+ reg |= BIT(1);
+ if (pps->flags & V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE)
+ reg |= BIT(0);
+ sunxi_cedrus_write(dev, reg, VE_H264_PIC_HDR);
+
+ // sequence parameters
+ reg = BIT(19);
+ reg |= (sps->pic_width_in_mbs_minus1 & 0xff) << 8;
+ reg |= sps->pic_height_in_map_units_minus1 & 0xff;
+ if (sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)
+ reg |= BIT(18);
+ if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)
+ reg |= BIT(17);
+ if (sps->flags & V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE)
+ reg |= BIT(16);
+ sunxi_cedrus_write(dev, reg, VE_H264_FRAME_SIZE);
+
+ // slice parameters
+ reg = 0;
+ /*
+ * FIXME: This bit marks all the frames as references. This
+ * should probably be set based on nal_ref_idc, but the libva
+ * doesn't pass that information along, so this is not always
+ * available. We should find something else, maybe change the
+ * kernel UAPI somehow?
+ */
+ reg |= BIT(12);
+ reg |= (slice->slice_type & 0xf) << 8;
+ reg |= slice->cabac_init_idc & 0x3;
+ reg |= BIT(5);
+ if (slice->flags & V4L2_SLICE_FLAG_FIELD_PIC)
+ reg |= BIT(4);
+ if (slice->flags & V4L2_SLICE_FLAG_BOTTOM_FIELD)
+ reg |= BIT(3);
+ if (slice->flags & V4L2_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED)
+ reg |= BIT(2);
+ sunxi_cedrus_write(dev, reg, VE_H264_SLICE_HDR);
+
+ reg = 0;
+ reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 24;
+ reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 16;
+ reg |= (slice->disable_deblocking_filter_idc & 0x3) << 8;
+ reg |= (slice->slice_alpha_c0_offset_div2 & 0xf) << 4;
+ reg |= slice->slice_beta_offset_div2 & 0xf;
+ sunxi_cedrus_write(dev, reg, VE_H264_SLICE_HDR2);
+
+ reg = 0;
+ /*
+ * FIXME: This bit tells the video engine to use the default
+ * quantization matrices. This will obviously need to be
+ * changed to support the profiles supporting custom
+ * quantization matrices.
+ */
+ reg |= BIT(24);
+ reg |= (pps->second_chroma_qp_index_offset & 0x3f) << 16;
+ reg |= (pps->chroma_qp_index_offset & 0x3f) << 8;
+ reg |= (pps->pic_init_qp_minus26 + 26 + slice->slice_qp_delta) & 0x3f;
+ sunxi_cedrus_write(dev, reg, VE_H264_QP_PARAM);
+
+ // clear status flags
+ sunxi_cedrus_write(dev, sunxi_cedrus_read(dev, VE_H264_STATUS), VE_H264_STATUS);
+
+ // enable int
+ reg = sunxi_cedrus_read(dev, VE_H264_CTRL) | 0x7;
+ sunxi_cedrus_write(dev, reg, VE_H264_CTRL);
+}
+
+static enum sunxi_cedrus_irq_status
+sunxi_cedrus_h264_irq_status(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ u32 reg = sunxi_cedrus_read(dev, VE_H264_STATUS) & 0x7;
+
+ if (!reg)
+ return SUNXI_CEDRUS_IRQ_NONE;
+
+ if (reg & (BIT(1) | BIT(2)))
+ return SUNXI_CEDRUS_IRQ_ERROR;
+
+ return SUNXI_CEDRUS_IRQ_OK;
+}
+
+static void sunxi_cedrus_h264_irq_clear(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+
+ sunxi_cedrus_write(dev, GENMASK(2, 0), VE_H264_STATUS);
+}
+
+static void sunxi_cedrus_h264_irq_disable(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ u32 reg = sunxi_cedrus_read(dev, VE_H264_CTRL) & ~GENMASK(2, 0);
+
+ sunxi_cedrus_write(dev, reg, VE_H264_CTRL);
+}
+
+static void sunxi_cedrus_h264_setup(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+
+ sunxi_cedrus_engine_enable(dev, SUNXI_CEDRUS_CODEC_H264);
+
+ sunxi_cedrus_write_frame_list(ctx, run);
+ sunxi_cedrus_set_params(ctx, run);
+}
+
+static int sunxi_cedrus_h264_start(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ int ret;
+
+ ctx->codec.h264.pic_info_buf =
+ dma_alloc_coherent(dev->dev, SUNXI_CEDRUS_PIC_INFO_BUF_SIZE,
+ &ctx->codec.h264.pic_info_buf_dma,
+ GFP_KERNEL);
+ if (!ctx->codec.h264.pic_info_buf)
+ return -ENOMEM;
+
+ ctx->codec.h264.neighbor_info_buf =
+ dma_alloc_coherent(dev->dev, SUNXI_CEDRUS_NEIGHBOR_INFO_BUF_SIZE,
+ &ctx->codec.h264.neighbor_info_buf_dma,
+ GFP_KERNEL);
+ if (!ctx->codec.h264.neighbor_info_buf) {
+ ret = -ENOMEM;
+ goto err_pic_buf;
+ }
+
+ ctx->codec.h264.mv_col_buf_size = DIV_ROUND_UP(ctx->src_fmt.width, 16) *
+ DIV_ROUND_UP(ctx->src_fmt.height, 16) * 32;
+ ctx->codec.h264.mv_col_buf = dma_alloc_coherent(dev->dev,
+ ctx->codec.h264.mv_col_buf_size,
+ &ctx->codec.h264.mv_col_buf_dma,
+ GFP_KERNEL);
+ if (!ctx->codec.h264.mv_col_buf) {
+ ret = -ENOMEM;
+ goto err_neighbor_buf;
+ }
+
+ return 0;
+
+err_neighbor_buf:
+ dma_free_coherent(dev->dev, SUNXI_CEDRUS_NEIGHBOR_INFO_BUF_SIZE,
+ ctx->codec.h264.neighbor_info_buf,
+ ctx->codec.h264.neighbor_info_buf_dma);
+err_pic_buf:
+ dma_free_coherent(dev->dev, SUNXI_CEDRUS_PIC_INFO_BUF_SIZE,
+ ctx->codec.h264.pic_info_buf,
+ ctx->codec.h264.pic_info_buf_dma);
+ return ret;
+}
+
+static void sunxi_cedrus_h264_stop(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+
+ dma_free_coherent(dev->dev, ctx->codec.h264.mv_col_buf_size,
+ ctx->codec.h264.mv_col_buf,
+ ctx->codec.h264.mv_col_buf_dma);
+ dma_free_coherent(dev->dev, SUNXI_CEDRUS_NEIGHBOR_INFO_BUF_SIZE,
+ ctx->codec.h264.neighbor_info_buf,
+ ctx->codec.h264.neighbor_info_buf_dma);
+ dma_free_coherent(dev->dev, SUNXI_CEDRUS_PIC_INFO_BUF_SIZE,
+ ctx->codec.h264.pic_info_buf,
+ ctx->codec.h264.pic_info_buf_dma);
+}
+
+static void sunxi_cedrus_h264_trigger(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+
+ sunxi_cedrus_write(dev, VE_H264_TRIGGER_TYPE_AVC_SLICE_DECODE,
+ VE_H264_TRIGGER_TYPE);
+}
+
+struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_h264 = {
+ .irq_clear = sunxi_cedrus_h264_irq_clear,
+ .irq_disable = sunxi_cedrus_h264_irq_disable,
+ .irq_status = sunxi_cedrus_h264_irq_status,
+ .setup = sunxi_cedrus_h264_setup,
+ .start = sunxi_cedrus_h264_start,
+ .stop = sunxi_cedrus_h264_stop,
+ .trigger = sunxi_cedrus_h264_trigger,
+};
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
index 6b97cbd2834e..e304457fce34 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
@@ -58,6 +58,10 @@ int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
reg |= VE_CTRL_DEC_MODE_MPEG;
break;
+ case SUNXI_CEDRUS_CODEC_H264:
+ reg |= VE_CTRL_DEC_MODE_H264;
+ break;
+
default:
return -EINVAL;
}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
index 6705d41dad07..a79de1b154b0 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
@@ -124,10 +124,20 @@
#define VE_H264_PRED_WEIGHT 0x210
#define VE_H264_QP_PARAM 0x21c
#define VE_H264_CTRL 0x220
-#define VE_H264_TRIGGER 0x224
+
+#define VE_H264_TRIGGER_TYPE 0x224
+#define VE_H264_TRIGGER_TYPE_AVC_SLICE_DECODE (8 << 0)
+#define VE_H264_TRIGGER_TYPE_INIT_SWDEC (7 << 0)
+
#define VE_H264_STATUS 0x228
#define VE_H264_CUR_MB_NUM 0x22c
+
#define VE_H264_VLD_ADDR 0x230
+#define VE_H264_VLD_ADDR_FIRST BIT(30)
+#define VE_H264_VLD_ADDR_LAST BIT(29)
+#define VE_H264_VLD_ADDR_VALID BIT(28)
+#define VE_H264_VLD_ADDR_VAL(x) (((x) & 0x0ffffff0) | ((x) >> 28))
+
#define VE_H264_VLD_OFFSET 0x234
#define VE_H264_VLD_LEN 0x238
#define VE_H264_VLD_END 0x23c
@@ -136,14 +146,8 @@
#define VE_H264_EXTRA_BUFFER1 0x250
#define VE_H264_EXTRA_BUFFER2 0x254
#define VE_H264_BASIC_BITS 0x2dc
-#define VE_H264_RAM_WRITE_PTR 0x2e0
-#define VE_H264_RAM_WRITE_DATA 0x2e4
-
-#define VE_SRAM_H264_PRED_WEIGHT_TABLE 0x000
-#define VE_SRAM_H264_FRAMEBUFFER_LIST 0x400
-#define VE_SRAM_H264_REF_LIST0 0x640
-#define VE_SRAM_H264_REF_LIST1 0x664
-#define VE_SRAM_H264_SCALING_LISTS 0x800
+#define VE_AVC_SRAM_PORT_OFFSET 0x2e0
+#define VE_AVC_SRAM_PORT_DATA 0x2e4
#define VE_ISP_INPUT_SIZE 0xa00
#define VE_ISP_INPUT_STRIDE 0xa04
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
index d93461178857..83e55b7825aa 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
@@ -52,6 +52,11 @@ static struct sunxi_cedrus_fmt formats[] = {
.types = SUNXI_CEDRUS_OUTPUT,
.num_planes = 1,
},
+ {
+ .fourcc = V4L2_PIX_FMT_H264_SLICE,
+ .types = SUNXI_CEDRUS_OUTPUT,
+ .num_planes = 1,
+ },
};
#define NUM_FORMATS ARRAY_SIZE(formats)
@@ -420,6 +425,9 @@ static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
int ret = 0;
switch (ctx->vpu_src_fmt->fourcc) {
+ case V4L2_PIX_FMT_H264_SLICE:
+ ctx->current_codec = SUNXI_CEDRUS_CODEC_H264;
+ break;
case V4L2_PIX_FMT_MPEG2_FRAME:
ctx->current_codec = SUNXI_CEDRUS_CODEC_MPEG2;
break;
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* [PATCH 9/9] media: cedrus: Add H264 decoding support
@ 2018-06-13 14:07 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-13 14:07 UTC (permalink / raw)
To: linux-arm-kernel
Introduce some basic H264 decoding support in cedrus. So far, only the
baseline profile videos have been tested, and some more advanced features
used in higher profiles are not even implemented.
Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
---
drivers/media/platform/sunxi/cedrus/Makefile | 2 +-
.../platform/sunxi/cedrus/sunxi_cedrus.c | 21 +
.../sunxi/cedrus/sunxi_cedrus_common.h | 41 +-
.../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 12 +
.../platform/sunxi/cedrus/sunxi_cedrus_h264.c | 443 ++++++++++++++++++
.../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 4 +
.../platform/sunxi/cedrus/sunxi_cedrus_regs.h | 22 +-
.../sunxi/cedrus/sunxi_cedrus_video.c | 8 +
8 files changed, 542 insertions(+), 11 deletions(-)
create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_h264.c
diff --git a/drivers/media/platform/sunxi/cedrus/Makefile b/drivers/media/platform/sunxi/cedrus/Makefile
index 98f30df626a9..715f4f67b743 100644
--- a/drivers/media/platform/sunxi/cedrus/Makefile
+++ b/drivers/media/platform/sunxi/cedrus/Makefile
@@ -1,4 +1,4 @@
obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o
sunxi-cedrus-y = sunxi_cedrus.o sunxi_cedrus_video.o sunxi_cedrus_hw.o \
- sunxi_cedrus_dec.o sunxi_cedrus_mpeg2.o
+ sunxi_cedrus_dec.o sunxi_cedrus_mpeg2.o sunxi_cedrus_h264.o
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
index bc80480f5dfd..581a99ba00c8 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
@@ -41,6 +41,10 @@ static int sunxi_cedrus_s_ctrl(struct v4l2_ctrl *ctrl)
container_of(ctrl->handler, struct sunxi_cedrus_ctx, hdl);
switch (ctrl->id) {
+ case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM:
+ case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM:
+ case V4L2_CID_MPEG_VIDEO_H264_SPS:
+ case V4L2_CID_MPEG_VIDEO_H264_PPS:
case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
/* This is kept in memory and used directly. */
break;
@@ -57,6 +61,22 @@ static const struct v4l2_ctrl_ops sunxi_cedrus_ctrl_ops = {
};
static const struct sunxi_cedrus_control controls[] = {
+ [SUNXI_CEDRUS_CTRL_DEC_H264_DECODE_PARAM] = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM,
+ .elem_size = sizeof(struct v4l2_ctrl_h264_decode_param),
+ },
+ [SUNXI_CEDRUS_CTRL_DEC_H264_SLICE_PARAM] = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM,
+ .elem_size = sizeof(struct v4l2_ctrl_h264_slice_param),
+ },
+ [SUNXI_CEDRUS_CTRL_DEC_H264_SPS] = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_SPS,
+ .elem_size = sizeof(struct v4l2_ctrl_h264_sps),
+ },
+ [SUNXI_CEDRUS_CTRL_DEC_H264_PPS] = {
+ .id = V4L2_CID_MPEG_VIDEO_H264_PPS,
+ .elem_size = sizeof(struct v4l2_ctrl_h264_pps),
+ },
[SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR] = {
.id = V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR,
.elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr),
@@ -244,6 +264,7 @@ static int sunxi_cedrus_probe(struct platform_device *pdev)
if (ret)
return ret;
+ dev->dec_ops[SUNXI_CEDRUS_CODEC_H264] = &sunxi_cedrus_dec_ops_h264;
dev->dec_ops[SUNXI_CEDRUS_CODEC_MPEG2] = &sunxi_cedrus_dec_ops_mpeg2;
ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
index 20c78ec1f037..1ab06ee68ce6 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -32,7 +32,11 @@
#define SUNXI_CEDRUS_NAME "sunxi-cedrus"
enum sunxi_cedrus_control_id {
- SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR = 0,
+ SUNXI_CEDRUS_CTRL_DEC_H264_DECODE_PARAM = 0,
+ SUNXI_CEDRUS_CTRL_DEC_H264_PPS,
+ SUNXI_CEDRUS_CTRL_DEC_H264_SLICE_PARAM,
+ SUNXI_CEDRUS_CTRL_DEC_H264_SPS,
+ SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR,
SUNXI_CEDRUS_CTRL_MAX,
};
@@ -48,6 +52,13 @@ struct sunxi_cedrus_fmt {
unsigned int num_planes;
};
+struct sunxi_cedrus_h264_run {
+ const struct v4l2_ctrl_h264_decode_param *decode_param;
+ const struct v4l2_ctrl_h264_pps *pps;
+ const struct v4l2_ctrl_h264_slice_param *slice_param;
+ const struct v4l2_ctrl_h264_sps *sps;
+};
+
struct sunxi_cedrus_mpeg2_run {
const struct v4l2_ctrl_mpeg2_frame_hdr *hdr;
};
@@ -57,12 +68,14 @@ struct sunxi_cedrus_run {
struct vb2_v4l2_buffer *dst;
union {
+ struct sunxi_cedrus_h264_run h264;
struct sunxi_cedrus_mpeg2_run mpeg2;
};
};
enum sunxi_cedrus_codec {
SUNXI_CEDRUS_CODEC_MPEG2,
+ SUNXI_CEDRUS_CODEC_H264,
SUNXI_CEDRUS_CODEC_LAST,
};
@@ -88,12 +101,37 @@ struct sunxi_cedrus_ctx {
struct work_struct run_work;
struct list_head src_list;
struct list_head dst_list;
+
+ union {
+ struct {
+ void *mv_col_buf;
+ dma_addr_t mv_col_buf_dma;
+ ssize_t mv_col_buf_size;
+ void *neighbor_info_buf;
+ dma_addr_t neighbor_info_buf_dma;
+ void *pic_info_buf;
+ dma_addr_t pic_info_buf_dma;
+ } h264;
+ } codec;
+};
+
+enum sunxi_cedrus_h264_pic_type {
+ SUNXI_CEDRUS_H264_PIC_TYPE_FRAME = 0,
+ SUNXI_CEDRUS_H264_PIC_TYPE_FIELD,
+ SUNXI_CEDRUS_H264_PIC_TYPE_MBAFF,
};
struct sunxi_cedrus_buffer {
struct vb2_v4l2_buffer vb;
enum vb2_buffer_state state;
struct list_head list;
+
+ union {
+ struct {
+ unsigned int position;
+ enum sunxi_cedrus_h264_pic_type pic_type;
+ } h264;
+ } codec;
};
static inline
@@ -125,6 +163,7 @@ struct sunxi_cedrus_dec_ops {
void (*trigger)(struct sunxi_cedrus_ctx *ctx);
};
+extern struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_h264;
extern struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2;
struct sunxi_cedrus_dev {
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
index 5e552fa05274..c29a2582ed68 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
@@ -122,6 +122,18 @@ void sunxi_cedrus_device_run(void *priv)
run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
break;
+ case V4L2_PIX_FMT_H264_SLICE:
+ CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_DECODE_PARAM);
+ CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_SLICE_PARAM);
+ CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_SPS);
+ CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_PPS);
+
+ run.h264.decode_param = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_DECODE_PARAM);
+ run.h264.pps = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_PPS);
+ run.h264.slice_param = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_SLICE_PARAM);
+ run.h264.sps = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_H264_SPS);
+ break;
+
default:
ctx->job_abort = 1;
}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_h264.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_h264.c
new file mode 100644
index 000000000000..0c86f66eb614
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_h264.c
@@ -0,0 +1,443 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2013 Jens Kuske <jenskuske@gmail.com>
+ * Copyright (c) 2018 Bootlin
+ */
+
+#include <linux/types.h>
+
+#include <media/videobuf2-dma-contig.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_hw.h"
+#include "sunxi_cedrus_regs.h"
+
+enum sunxi_cedrus_h264_sram_off {
+ SUNXI_CEDRUS_SRAM_H264_PRED_WEIGHT_TABLE = 0x000,
+ SUNXI_CEDRUS_SRAM_H264_FRAMEBUFFER_LIST = 0x100,
+ SUNXI_CEDRUS_SRAM_H264_REF_LIST_0 = 0x190,
+ SUNXI_CEDRUS_SRAM_H264_REF_LIST_1 = 0x199,
+ SUNXI_CEDRUS_SRAM_H264_SCALING_LIST_8x8 = 0x200,
+ SUNXI_CEDRUS_SRAM_H264_SCALING_LIST_4x4 = 0x218,
+};
+
+struct sunxi_cedrus_h264_sram_ref_pic {
+ __le32 top_field_order_cnt;
+ __le32 bottom_field_order_cnt;
+ __le32 frame_info;
+ __le32 luma_ptr;
+ __le32 chroma_ptr;
+ __le32 extra_data_ptr;
+ __le32 extra_data_end;
+ __le32 reserved;
+} __packed;
+
+/* One for the output, 16 for the reference images */
+#define SUNXI_CEDRUS_H264_FRAME_NUM 17
+
+#define SUNXI_CEDRUS_PIC_INFO_BUF_SIZE (128 * SZ_1K)
+#define SUNXI_CEDRUS_NEIGHBOR_INFO_BUF_SIZE (16 * SZ_1K)
+
+static void sunxi_cedrus_h264_write_sram(struct sunxi_cedrus_dev *dev,
+ enum sunxi_cedrus_h264_sram_off off,
+ const void *data, size_t len)
+{
+ const u32 *buffer = data;
+ size_t count = DIV_ROUND_UP(len, 4);
+
+ sunxi_cedrus_write(dev, off << 2,
+ VE_AVC_SRAM_PORT_OFFSET);
+
+ do {
+ sunxi_cedrus_write(dev, *buffer++, VE_AVC_SRAM_PORT_DATA);
+ } while (--count);
+}
+
+static void sunxi_cedrus_fill_ref_pic(struct sunxi_cedrus_h264_sram_ref_pic *pic,
+ struct vb2_buffer *buf,
+ dma_addr_t extra_buf,
+ size_t extra_buf_len,
+ unsigned int top_field_order_cnt,
+ unsigned int bottom_field_order_cnt,
+ enum sunxi_cedrus_h264_pic_type pic_type)
+{
+ pic->top_field_order_cnt = top_field_order_cnt;
+ pic->bottom_field_order_cnt = bottom_field_order_cnt;
+ pic->frame_info = pic_type << 8;
+ pic->luma_ptr = vb2_dma_contig_plane_dma_addr(buf, 0) - PHYS_OFFSET;
+ pic->chroma_ptr = vb2_dma_contig_plane_dma_addr(buf, 1) - PHYS_OFFSET;
+ pic->extra_data_ptr = extra_buf - PHYS_OFFSET;
+ pic->extra_data_end = (extra_buf - PHYS_OFFSET) + extra_buf_len;
+}
+
+static void sunxi_cedrus_write_frame_list(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
+{
+ struct sunxi_cedrus_h264_sram_ref_pic pic_list[SUNXI_CEDRUS_H264_FRAME_NUM];
+ const struct v4l2_ctrl_h264_decode_param *dec_param = run->h264.decode_param;
+ const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
+ const struct v4l2_ctrl_h264_sps *sps = run->h264.sps;
+ struct sunxi_cedrus_buffer *output_buf;
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ unsigned long used_dpbs = 0;
+ unsigned int position;
+ unsigned int output = 0;
+ unsigned int i;
+
+ memset(pic_list, 0, sizeof(pic_list));
+
+ for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) {
+ const struct v4l2_h264_dpb_entry *dpb = &dec_param->dpb[i];
+ const struct sunxi_cedrus_buffer *cedrus_buf;
+ struct vb2_buffer *ref_buf;
+
+ if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
+ continue;
+
+ ref_buf = ctx->dst_bufs[dpb->buf_index];
+ cedrus_buf = vb2_to_cedrus_buffer(ref_buf);
+ position = cedrus_buf->codec.h264.position;
+ used_dpbs |= BIT(position);
+
+ sunxi_cedrus_fill_ref_pic(&pic_list[position], ref_buf,
+ ctx->codec.h264.mv_col_buf_dma,
+ ctx->codec.h264.mv_col_buf_size,
+ dpb->top_field_order_cnt,
+ dpb->bottom_field_order_cnt,
+ cedrus_buf->codec.h264.pic_type);
+
+ output = max(position, output);
+ }
+
+ position = find_next_zero_bit(&used_dpbs, 17, output);
+ if (position >= 17)
+ position = find_first_zero_bit(&used_dpbs, 17);
+
+ output_buf = vb2_to_cedrus_buffer(&run->dst->vb2_buf);
+ output_buf->codec.h264.position = position;
+
+ if (slice->flags & V4L2_SLICE_FLAG_FIELD_PIC)
+ output_buf->codec.h264.pic_type = SUNXI_CEDRUS_H264_PIC_TYPE_FIELD;
+ else if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)
+ output_buf->codec.h264.pic_type = SUNXI_CEDRUS_H264_PIC_TYPE_MBAFF;
+ else
+ output_buf->codec.h264.pic_type = SUNXI_CEDRUS_H264_PIC_TYPE_FRAME;
+
+ sunxi_cedrus_fill_ref_pic(&pic_list[position], &run->dst->vb2_buf,
+ ctx->codec.h264.mv_col_buf_dma,
+ ctx->codec.h264.mv_col_buf_size,
+ dec_param->top_field_order_cnt,
+ dec_param->bottom_field_order_cnt,
+ output_buf->codec.h264.pic_type);
+
+ sunxi_cedrus_h264_write_sram(dev, SUNXI_CEDRUS_SRAM_H264_FRAMEBUFFER_LIST,
+ pic_list, sizeof(pic_list));
+
+ sunxi_cedrus_write(dev, position, VE_H264_OUTPUT_FRAME_IDX);
+}
+
+#define SUNXI_CEDRUS_MAX_REF_IDX 32
+
+static void _sunxi_cedrus_write_ref_list(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run,
+ const u8 *ref_list, u8 num_ref,
+ enum sunxi_cedrus_h264_sram_off sram)
+{
+ const struct v4l2_ctrl_h264_decode_param *decode = run->h264.decode_param;
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ u32 sram_array[SUNXI_CEDRUS_MAX_REF_IDX / sizeof(u32)];
+ unsigned int size, i;
+
+ memset(sram_array, 0, sizeof(sram_array));
+
+ for (i = 0; i < num_ref; i += 4) {
+ unsigned int j;
+
+ for (j = 0; j < 4; j++) {
+ const struct v4l2_h264_dpb_entry *dpb;
+ const struct sunxi_cedrus_buffer *cedrus_buf;
+ const struct vb2_v4l2_buffer *ref_buf;
+ unsigned int position;
+ u8 ref_idx = i + j;
+ u8 dpb_idx;
+
+ if (ref_idx >= num_ref)
+ break;
+
+ dpb_idx = ref_list[ref_idx];
+ dpb = &decode->dpb[dpb_idx];
+
+ if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
+ continue;
+
+ ref_buf = to_vb2_v4l2_buffer(ctx->dst_bufs[dpb->buf_index]);
+ cedrus_buf = vb2_v4l2_to_cedrus_buffer(ref_buf);
+ position = cedrus_buf->codec.h264.position;
+
+ sram_array[i] |= position << (j * 8 + 1);
+ if (ref_buf->field == V4L2_FIELD_BOTTOM)
+ sram_array[i] |= BIT(j * 8);
+ }
+ }
+
+ size = min((unsigned int)ALIGN(num_ref, 4), sizeof(sram_array));
+ sunxi_cedrus_h264_write_sram(dev, sram, &sram_array, size);
+}
+
+static void sunxi_cedrus_write_ref_list0(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
+{
+ const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
+
+ _sunxi_cedrus_write_ref_list(ctx, run,
+ slice->ref_pic_list0,
+ slice->num_ref_idx_l0_active_minus1 + 1,
+ SUNXI_CEDRUS_SRAM_H264_REF_LIST_0);
+}
+
+static void sunxi_cedrus_write_ref_list1(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
+{
+ const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
+
+ _sunxi_cedrus_write_ref_list(ctx, run,
+ slice->ref_pic_list1,
+ slice->num_ref_idx_l1_active_minus1 + 1,
+ SUNXI_CEDRUS_SRAM_H264_REF_LIST_1);
+}
+
+static void sunxi_cedrus_set_params(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
+{
+ const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
+ const struct v4l2_ctrl_h264_pps *pps = run->h264.pps;
+ const struct v4l2_ctrl_h264_sps *sps = run->h264.sps;
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ dma_addr_t src_buf_addr;
+ u32 offset = slice->header_bit_size;
+ u32 len = (slice->size * 8) - offset;
+ u32 reg;
+
+ sunxi_cedrus_write(dev, ctx->codec.h264.pic_info_buf_dma - PHYS_OFFSET, 0x250);
+ sunxi_cedrus_write(dev, (ctx->codec.h264.pic_info_buf_dma - PHYS_OFFSET) + 0x48000, 0x254);
+
+ sunxi_cedrus_write(dev, len, VE_H264_VLD_LEN);
+ sunxi_cedrus_write(dev, offset, VE_H264_VLD_OFFSET);
+
+ src_buf_addr = vb2_dma_contig_plane_dma_addr(&run->src->vb2_buf, 0);
+ src_buf_addr -= PHYS_OFFSET;
+ sunxi_cedrus_write(dev, VE_H264_VLD_ADDR_VAL(src_buf_addr) |
+ VE_H264_VLD_ADDR_FIRST | VE_H264_VLD_ADDR_VALID | VE_H264_VLD_ADDR_LAST,
+ VE_H264_VLD_ADDR);
+ sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_H264_VLD_END);
+
+ sunxi_cedrus_write(dev, VE_H264_TRIGGER_TYPE_INIT_SWDEC,
+ VE_H264_TRIGGER_TYPE);
+
+ if ((slice->slice_type == V4L2_H264_SLICE_TYPE_P) ||
+ (slice->slice_type == V4L2_H264_SLICE_TYPE_SP) ||
+ (slice->slice_type == V4L2_H264_SLICE_TYPE_B))
+ sunxi_cedrus_write_ref_list0(ctx, run);
+
+ if (slice->slice_type == V4L2_H264_SLICE_TYPE_B)
+ sunxi_cedrus_write_ref_list1(ctx, run);
+
+ // picture parameters
+ reg = 0;
+ /*
+ * FIXME: the kernel headers are allowing the default value to
+ * be passed, but the libva doesn't give us that.
+ */
+ reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 10;
+ reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 5;
+ reg |= (pps->weighted_bipred_idc & 0x3) << 2;
+ if (pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE)
+ reg |= BIT(15);
+ if (pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED)
+ reg |= BIT(4);
+ if (pps->flags & V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED)
+ reg |= BIT(1);
+ if (pps->flags & V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE)
+ reg |= BIT(0);
+ sunxi_cedrus_write(dev, reg, VE_H264_PIC_HDR);
+
+ // sequence parameters
+ reg = BIT(19);
+ reg |= (sps->pic_width_in_mbs_minus1 & 0xff) << 8;
+ reg |= sps->pic_height_in_map_units_minus1 & 0xff;
+ if (sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)
+ reg |= BIT(18);
+ if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)
+ reg |= BIT(17);
+ if (sps->flags & V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE)
+ reg |= BIT(16);
+ sunxi_cedrus_write(dev, reg, VE_H264_FRAME_SIZE);
+
+ // slice parameters
+ reg = 0;
+ /*
+ * FIXME: This bit marks all the frames as references. This
+ * should probably be set based on nal_ref_idc, but the libva
+ * doesn't pass that information along, so this is not always
+ * available. We should find something else, maybe change the
+ * kernel UAPI somehow?
+ */
+ reg |= BIT(12);
+ reg |= (slice->slice_type & 0xf) << 8;
+ reg |= slice->cabac_init_idc & 0x3;
+ reg |= BIT(5);
+ if (slice->flags & V4L2_SLICE_FLAG_FIELD_PIC)
+ reg |= BIT(4);
+ if (slice->flags & V4L2_SLICE_FLAG_BOTTOM_FIELD)
+ reg |= BIT(3);
+ if (slice->flags & V4L2_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED)
+ reg |= BIT(2);
+ sunxi_cedrus_write(dev, reg, VE_H264_SLICE_HDR);
+
+ reg = 0;
+ reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 24;
+ reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 16;
+ reg |= (slice->disable_deblocking_filter_idc & 0x3) << 8;
+ reg |= (slice->slice_alpha_c0_offset_div2 & 0xf) << 4;
+ reg |= slice->slice_beta_offset_div2 & 0xf;
+ sunxi_cedrus_write(dev, reg, VE_H264_SLICE_HDR2);
+
+ reg = 0;
+ /*
+ * FIXME: This bit tells the video engine to use the default
+ * quantization matrices. This will obviously need to be
+ * changed to support the profiles supporting custom
+ * quantization matrices.
+ */
+ reg |= BIT(24);
+ reg |= (pps->second_chroma_qp_index_offset & 0x3f) << 16;
+ reg |= (pps->chroma_qp_index_offset & 0x3f) << 8;
+ reg |= (pps->pic_init_qp_minus26 + 26 + slice->slice_qp_delta) & 0x3f;
+ sunxi_cedrus_write(dev, reg, VE_H264_QP_PARAM);
+
+ // clear status flags
+ sunxi_cedrus_write(dev, sunxi_cedrus_read(dev, VE_H264_STATUS), VE_H264_STATUS);
+
+ // enable int
+ reg = sunxi_cedrus_read(dev, VE_H264_CTRL) | 0x7;
+ sunxi_cedrus_write(dev, reg, VE_H264_CTRL);
+}
+
+static enum sunxi_cedrus_irq_status
+sunxi_cedrus_h264_irq_status(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ u32 reg = sunxi_cedrus_read(dev, VE_H264_STATUS) & 0x7;
+
+ if (!reg)
+ return SUNXI_CEDRUS_IRQ_NONE;
+
+ if (reg & (BIT(1) | BIT(2)))
+ return SUNXI_CEDRUS_IRQ_ERROR;
+
+ return SUNXI_CEDRUS_IRQ_OK;
+}
+
+static void sunxi_cedrus_h264_irq_clear(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+
+ sunxi_cedrus_write(dev, GENMASK(2, 0), VE_H264_STATUS);
+}
+
+static void sunxi_cedrus_h264_irq_disable(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ u32 reg = sunxi_cedrus_read(dev, VE_H264_CTRL) & ~GENMASK(2, 0);
+
+ sunxi_cedrus_write(dev, reg, VE_H264_CTRL);
+}
+
+static void sunxi_cedrus_h264_setup(struct sunxi_cedrus_ctx *ctx,
+ struct sunxi_cedrus_run *run)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+
+ sunxi_cedrus_engine_enable(dev, SUNXI_CEDRUS_CODEC_H264);
+
+ sunxi_cedrus_write_frame_list(ctx, run);
+ sunxi_cedrus_set_params(ctx, run);
+}
+
+static int sunxi_cedrus_h264_start(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+ int ret;
+
+ ctx->codec.h264.pic_info_buf =
+ dma_alloc_coherent(dev->dev, SUNXI_CEDRUS_PIC_INFO_BUF_SIZE,
+ &ctx->codec.h264.pic_info_buf_dma,
+ GFP_KERNEL);
+ if (!ctx->codec.h264.pic_info_buf)
+ return -ENOMEM;
+
+ ctx->codec.h264.neighbor_info_buf =
+ dma_alloc_coherent(dev->dev, SUNXI_CEDRUS_NEIGHBOR_INFO_BUF_SIZE,
+ &ctx->codec.h264.neighbor_info_buf_dma,
+ GFP_KERNEL);
+ if (!ctx->codec.h264.neighbor_info_buf) {
+ ret = -ENOMEM;
+ goto err_pic_buf;
+ }
+
+ ctx->codec.h264.mv_col_buf_size = DIV_ROUND_UP(ctx->src_fmt.width, 16) *
+ DIV_ROUND_UP(ctx->src_fmt.height, 16) * 32;
+ ctx->codec.h264.mv_col_buf = dma_alloc_coherent(dev->dev,
+ ctx->codec.h264.mv_col_buf_size,
+ &ctx->codec.h264.mv_col_buf_dma,
+ GFP_KERNEL);
+ if (!ctx->codec.h264.mv_col_buf) {
+ ret = -ENOMEM;
+ goto err_neighbor_buf;
+ }
+
+ return 0;
+
+err_neighbor_buf:
+ dma_free_coherent(dev->dev, SUNXI_CEDRUS_NEIGHBOR_INFO_BUF_SIZE,
+ ctx->codec.h264.neighbor_info_buf,
+ ctx->codec.h264.neighbor_info_buf_dma);
+err_pic_buf:
+ dma_free_coherent(dev->dev, SUNXI_CEDRUS_PIC_INFO_BUF_SIZE,
+ ctx->codec.h264.pic_info_buf,
+ ctx->codec.h264.pic_info_buf_dma);
+ return ret;
+}
+
+static void sunxi_cedrus_h264_stop(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+
+ dma_free_coherent(dev->dev, ctx->codec.h264.mv_col_buf_size,
+ ctx->codec.h264.mv_col_buf,
+ ctx->codec.h264.mv_col_buf_dma);
+ dma_free_coherent(dev->dev, SUNXI_CEDRUS_NEIGHBOR_INFO_BUF_SIZE,
+ ctx->codec.h264.neighbor_info_buf,
+ ctx->codec.h264.neighbor_info_buf_dma);
+ dma_free_coherent(dev->dev, SUNXI_CEDRUS_PIC_INFO_BUF_SIZE,
+ ctx->codec.h264.pic_info_buf,
+ ctx->codec.h264.pic_info_buf_dma);
+}
+
+static void sunxi_cedrus_h264_trigger(struct sunxi_cedrus_ctx *ctx)
+{
+ struct sunxi_cedrus_dev *dev = ctx->dev;
+
+ sunxi_cedrus_write(dev, VE_H264_TRIGGER_TYPE_AVC_SLICE_DECODE,
+ VE_H264_TRIGGER_TYPE);
+}
+
+struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_h264 = {
+ .irq_clear = sunxi_cedrus_h264_irq_clear,
+ .irq_disable = sunxi_cedrus_h264_irq_disable,
+ .irq_status = sunxi_cedrus_h264_irq_status,
+ .setup = sunxi_cedrus_h264_setup,
+ .start = sunxi_cedrus_h264_start,
+ .stop = sunxi_cedrus_h264_stop,
+ .trigger = sunxi_cedrus_h264_trigger,
+};
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
index 6b97cbd2834e..e304457fce34 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
@@ -58,6 +58,10 @@ int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
reg |= VE_CTRL_DEC_MODE_MPEG;
break;
+ case SUNXI_CEDRUS_CODEC_H264:
+ reg |= VE_CTRL_DEC_MODE_H264;
+ break;
+
default:
return -EINVAL;
}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
index 6705d41dad07..a79de1b154b0 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
@@ -124,10 +124,20 @@
#define VE_H264_PRED_WEIGHT 0x210
#define VE_H264_QP_PARAM 0x21c
#define VE_H264_CTRL 0x220
-#define VE_H264_TRIGGER 0x224
+
+#define VE_H264_TRIGGER_TYPE 0x224
+#define VE_H264_TRIGGER_TYPE_AVC_SLICE_DECODE (8 << 0)
+#define VE_H264_TRIGGER_TYPE_INIT_SWDEC (7 << 0)
+
#define VE_H264_STATUS 0x228
#define VE_H264_CUR_MB_NUM 0x22c
+
#define VE_H264_VLD_ADDR 0x230
+#define VE_H264_VLD_ADDR_FIRST BIT(30)
+#define VE_H264_VLD_ADDR_LAST BIT(29)
+#define VE_H264_VLD_ADDR_VALID BIT(28)
+#define VE_H264_VLD_ADDR_VAL(x) (((x) & 0x0ffffff0) | ((x) >> 28))
+
#define VE_H264_VLD_OFFSET 0x234
#define VE_H264_VLD_LEN 0x238
#define VE_H264_VLD_END 0x23c
@@ -136,14 +146,8 @@
#define VE_H264_EXTRA_BUFFER1 0x250
#define VE_H264_EXTRA_BUFFER2 0x254
#define VE_H264_BASIC_BITS 0x2dc
-#define VE_H264_RAM_WRITE_PTR 0x2e0
-#define VE_H264_RAM_WRITE_DATA 0x2e4
-
-#define VE_SRAM_H264_PRED_WEIGHT_TABLE 0x000
-#define VE_SRAM_H264_FRAMEBUFFER_LIST 0x400
-#define VE_SRAM_H264_REF_LIST0 0x640
-#define VE_SRAM_H264_REF_LIST1 0x664
-#define VE_SRAM_H264_SCALING_LISTS 0x800
+#define VE_AVC_SRAM_PORT_OFFSET 0x2e0
+#define VE_AVC_SRAM_PORT_DATA 0x2e4
#define VE_ISP_INPUT_SIZE 0xa00
#define VE_ISP_INPUT_STRIDE 0xa04
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
index d93461178857..83e55b7825aa 100644
--- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
@@ -52,6 +52,11 @@ static struct sunxi_cedrus_fmt formats[] = {
.types = SUNXI_CEDRUS_OUTPUT,
.num_planes = 1,
},
+ {
+ .fourcc = V4L2_PIX_FMT_H264_SLICE,
+ .types = SUNXI_CEDRUS_OUTPUT,
+ .num_planes = 1,
+ },
};
#define NUM_FORMATS ARRAY_SIZE(formats)
@@ -420,6 +425,9 @@ static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
int ret = 0;
switch (ctx->vpu_src_fmt->fourcc) {
+ case V4L2_PIX_FMT_H264_SLICE:
+ ctx->current_codec = SUNXI_CEDRUS_CODEC_H264;
+ break;
case V4L2_PIX_FMT_MPEG2_FRAME:
ctx->current_codec = SUNXI_CEDRUS_CODEC_MPEG2;
break;
--
2.17.0
^ permalink raw reply related [flat|nested] 102+ messages in thread
* Re: [PATCH 0/9] media: cedrus: Add H264 decoding support
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-14 13:00 ` Tomasz Figa
-1 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-06-14 13:00 UTC (permalink / raw)
To: Maxime Ripard
Cc: Hans Verkuil, Alexandre Courbot, Sakari Ailus, Laurent Pinchart,
Pawel Osciak, Paul Kocialkowski, Chen-Yu Tsai,
Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, nicolas.dufresne, jenskuske,
linux-sunxi, thomas.petazzoni
Hi Maxime,
On Wed, Jun 13, 2018 at 11:07 PM Maxime Ripard
<maxime.ripard@bootlin.com> wrote:
>
> Hi,
>
> Here is a preliminary version of the H264 decoding support in the
> cedrus driver.
Thanks for the series! Let me reply inline to some of the points raised here.
>
> As you might already know, the cedrus driver relies on the Request
> API, and is a reverse engineered driver for the video decoding engine
> found on the Allwinner SoCs.
>
> This work has been possible thanks to the work done by the people
> behind libvdpau-sunxi found here:
> https://github.com/linux-sunxi/libvdpau-sunxi/
>
> This driver is based on the last version of the cedrus driver sent by
> Paul, based on Request API v13 sent by Hans:
> https://lkml.org/lkml/2018/5/7/316
Just FYI, there is v15 already. :)
>
> This driver has been tested only with baseline profile videos, and is
> missing a few key features to decode videos with higher profiles.
> This has been tested using our cedrus-frame-test tool, which should be
> a quite generic v4l2-to-drm decoder using the request API to
> demonstrate the video decoding:
> https://github.com/free-electrons/cedrus-frame-test/, branch h264
>
> However, sending this preliminary version, I'd really like to start a
> discussion and get some feedback on the user-space API for the H264
> controls exposed through the request API.
>
> I've been using the controls currently integrated into ChromeOS that
> have a working version of this particular setup. However, these
> controls have a number of shortcomings and inconsistencies with other
> decoding API. I've worked with libva so far, but I've noticed already
> that:
Note that these controls are supposed to be defined exactly like the
bitstream headers deserialized into C structs in memory. I believe
Pawel (on CC) defined them based on the actual H264 specification.
> - The kernel UAPI expects to have the nal_ref_idc variable, while
> libva only exposes whether that frame is a reference frame or
> not. I've looked at the rockchip driver in the ChromeOS tree, and
> our own driver, and they both need only the information about
> whether the frame is a reference one or not, so maybe we should
> change this?
The fact that 2 drivers only need partial information doesn't mean
that we should ignore the data being already in the bitstream. IMHO
this API should to provide all the metadata available in the stream to
the kernel driver, as a replacement for bitstream parsing in firmware
(or in kernel... yuck).
> - The H264 bitstream exposes the picture default reference list (for
> both list 0 and list 1), the slice reference list and an override
> flag. The libva will only pass the reference list to be used (so
> either the picture default's or the slice's) depending on the
> override flag. The kernel UAPI wants the picture default reference
> list and the slice reference list, but doesn't expose the override
> flag, which prevents us from configuring properly the
> hardware. Our video decoding engine needs the three information,
> but we can easily adapt to having only one. However, having two
> doesn't really work for us.
Where does the override flag come from? If it's in the bitstream, then
I guess it was just missed when creating the structures.
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 0/9] media: cedrus: Add H264 decoding support
@ 2018-06-14 13:00 ` Tomasz Figa
0 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-06-14 13:00 UTC (permalink / raw)
To: linux-arm-kernel
Hi Maxime,
On Wed, Jun 13, 2018 at 11:07 PM Maxime Ripard
<maxime.ripard@bootlin.com> wrote:
>
> Hi,
>
> Here is a preliminary version of the H264 decoding support in the
> cedrus driver.
Thanks for the series! Let me reply inline to some of the points raised here.
>
> As you might already know, the cedrus driver relies on the Request
> API, and is a reverse engineered driver for the video decoding engine
> found on the Allwinner SoCs.
>
> This work has been possible thanks to the work done by the people
> behind libvdpau-sunxi found here:
> https://github.com/linux-sunxi/libvdpau-sunxi/
>
> This driver is based on the last version of the cedrus driver sent by
> Paul, based on Request API v13 sent by Hans:
> https://lkml.org/lkml/2018/5/7/316
Just FYI, there is v15 already. :)
>
> This driver has been tested only with baseline profile videos, and is
> missing a few key features to decode videos with higher profiles.
> This has been tested using our cedrus-frame-test tool, which should be
> a quite generic v4l2-to-drm decoder using the request API to
> demonstrate the video decoding:
> https://github.com/free-electrons/cedrus-frame-test/, branch h264
>
> However, sending this preliminary version, I'd really like to start a
> discussion and get some feedback on the user-space API for the H264
> controls exposed through the request API.
>
> I've been using the controls currently integrated into ChromeOS that
> have a working version of this particular setup. However, these
> controls have a number of shortcomings and inconsistencies with other
> decoding API. I've worked with libva so far, but I've noticed already
> that:
Note that these controls are supposed to be defined exactly like the
bitstream headers deserialized into C structs in memory. I believe
Pawel (on CC) defined them based on the actual H264 specification.
> - The kernel UAPI expects to have the nal_ref_idc variable, while
> libva only exposes whether that frame is a reference frame or
> not. I've looked at the rockchip driver in the ChromeOS tree, and
> our own driver, and they both need only the information about
> whether the frame is a reference one or not, so maybe we should
> change this?
The fact that 2 drivers only need partial information doesn't mean
that we should ignore the data being already in the bitstream. IMHO
this API should to provide all the metadata available in the stream to
the kernel driver, as a replacement for bitstream parsing in firmware
(or in kernel... yuck).
> - The H264 bitstream exposes the picture default reference list (for
> both list 0 and list 1), the slice reference list and an override
> flag. The libva will only pass the reference list to be used (so
> either the picture default's or the slice's) depending on the
> override flag. The kernel UAPI wants the picture default reference
> list and the slice reference list, but doesn't expose the override
> flag, which prevents us from configuring properly the
> hardware. Our video decoding engine needs the three information,
> but we can easily adapt to having only one. However, having two
> doesn't really work for us.
Where does the override flag come from? If it's in the bitstream, then
I guess it was just missed when creating the structures.
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 0/9] media: cedrus: Add H264 decoding support
2018-06-14 13:00 ` Tomasz Figa
@ 2018-06-14 16:37 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-14 16:37 UTC (permalink / raw)
To: Tomasz Figa
Cc: Hans Verkuil, Alexandre Courbot, Sakari Ailus, Laurent Pinchart,
Pawel Osciak, Paul Kocialkowski, Chen-Yu Tsai,
Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, nicolas.dufresne, jenskuske,
linux-sunxi, thomas.petazzoni
Hi Tomasz,
On Thu, Jun 14, 2018 at 10:00:43PM +0900, Tomasz Figa wrote:
> Hi Maxime,
>
> On Wed, Jun 13, 2018 at 11:07 PM Maxime Ripard
> <maxime.ripard@bootlin.com> wrote:
> >
> > Hi,
> >
> > Here is a preliminary version of the H264 decoding support in the
> > cedrus driver.
>
> Thanks for the series! Let me reply inline to some of the points raised here.
>
> > As you might already know, the cedrus driver relies on the Request
> > API, and is a reverse engineered driver for the video decoding engine
> > found on the Allwinner SoCs.
> >
> > This work has been possible thanks to the work done by the people
> > behind libvdpau-sunxi found here:
> > https://github.com/linux-sunxi/libvdpau-sunxi/
> >
> > This driver is based on the last version of the cedrus driver sent by
> > Paul, based on Request API v13 sent by Hans:
> > https://lkml.org/lkml/2018/5/7/316
>
> Just FYI, there is v15 already. :)
Yeah, we know, Paul is currently working on rebasing to that version :)
> > This driver has been tested only with baseline profile videos, and is
> > missing a few key features to decode videos with higher profiles.
> > This has been tested using our cedrus-frame-test tool, which should be
> > a quite generic v4l2-to-drm decoder using the request API to
> > demonstrate the video decoding:
> > https://github.com/free-electrons/cedrus-frame-test/, branch h264
> >
> > However, sending this preliminary version, I'd really like to start a
> > discussion and get some feedback on the user-space API for the H264
> > controls exposed through the request API.
> >
> > I've been using the controls currently integrated into ChromeOS that
> > have a working version of this particular setup. However, these
> > controls have a number of shortcomings and inconsistencies with other
> > decoding API. I've worked with libva so far, but I've noticed already
> > that:
>
> Note that these controls are supposed to be defined exactly like the
> bitstream headers deserialized into C structs in memory. I believe
> Pawel (on CC) defined them based on the actual H264 specification.
>
> > - The kernel UAPI expects to have the nal_ref_idc variable, while
> > libva only exposes whether that frame is a reference frame or
> > not. I've looked at the rockchip driver in the ChromeOS tree, and
> > our own driver, and they both need only the information about
> > whether the frame is a reference one or not, so maybe we should
> > change this?
>
> The fact that 2 drivers only need partial information doesn't mean
> that we should ignore the data being already in the bitstream. IMHO
> this API should to provide all the metadata available in the stream to
> the kernel driver, as a replacement for bitstream parsing in firmware
> (or in kernel... yuck).
The point is more that libva will only pass the result of (nal_ref_idc
!= 0). So in the libva plugin, you won't be able to fill the proper
value to the kernel, since you don't have access to it.
> > - The H264 bitstream exposes the picture default reference list (for
> > both list 0 and list 1), the slice reference list and an override
> > flag. The libva will only pass the reference list to be used (so
> > either the picture default's or the slice's) depending on the
> > override flag. The kernel UAPI wants the picture default reference
> > list and the slice reference list, but doesn't expose the override
> > flag, which prevents us from configuring properly the
> > hardware. Our video decoding engine needs the three information,
> > but we can easily adapt to having only one. However, having two
> > doesn't really work for us.
>
> Where does the override flag come from? If it's in the bitstream, then
> I guess it was just missed when creating the structures.
It's in the bitstream yeah. I'll add it then.
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 0/9] media: cedrus: Add H264 decoding support
@ 2018-06-14 16:37 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-14 16:37 UTC (permalink / raw)
To: linux-arm-kernel
Hi Tomasz,
On Thu, Jun 14, 2018 at 10:00:43PM +0900, Tomasz Figa wrote:
> Hi Maxime,
>
> On Wed, Jun 13, 2018 at 11:07 PM Maxime Ripard
> <maxime.ripard@bootlin.com> wrote:
> >
> > Hi,
> >
> > Here is a preliminary version of the H264 decoding support in the
> > cedrus driver.
>
> Thanks for the series! Let me reply inline to some of the points raised here.
>
> > As you might already know, the cedrus driver relies on the Request
> > API, and is a reverse engineered driver for the video decoding engine
> > found on the Allwinner SoCs.
> >
> > This work has been possible thanks to the work done by the people
> > behind libvdpau-sunxi found here:
> > https://github.com/linux-sunxi/libvdpau-sunxi/
> >
> > This driver is based on the last version of the cedrus driver sent by
> > Paul, based on Request API v13 sent by Hans:
> > https://lkml.org/lkml/2018/5/7/316
>
> Just FYI, there is v15 already. :)
Yeah, we know, Paul is currently working on rebasing to that version :)
> > This driver has been tested only with baseline profile videos, and is
> > missing a few key features to decode videos with higher profiles.
> > This has been tested using our cedrus-frame-test tool, which should be
> > a quite generic v4l2-to-drm decoder using the request API to
> > demonstrate the video decoding:
> > https://github.com/free-electrons/cedrus-frame-test/, branch h264
> >
> > However, sending this preliminary version, I'd really like to start a
> > discussion and get some feedback on the user-space API for the H264
> > controls exposed through the request API.
> >
> > I've been using the controls currently integrated into ChromeOS that
> > have a working version of this particular setup. However, these
> > controls have a number of shortcomings and inconsistencies with other
> > decoding API. I've worked with libva so far, but I've noticed already
> > that:
>
> Note that these controls are supposed to be defined exactly like the
> bitstream headers deserialized into C structs in memory. I believe
> Pawel (on CC) defined them based on the actual H264 specification.
>
> > - The kernel UAPI expects to have the nal_ref_idc variable, while
> > libva only exposes whether that frame is a reference frame or
> > not. I've looked at the rockchip driver in the ChromeOS tree, and
> > our own driver, and they both need only the information about
> > whether the frame is a reference one or not, so maybe we should
> > change this?
>
> The fact that 2 drivers only need partial information doesn't mean
> that we should ignore the data being already in the bitstream. IMHO
> this API should to provide all the metadata available in the stream to
> the kernel driver, as a replacement for bitstream parsing in firmware
> (or in kernel... yuck).
The point is more that libva will only pass the result of (nal_ref_idc
!= 0). So in the libva plugin, you won't be able to fill the proper
value to the kernel, since you don't have access to it.
> > - The H264 bitstream exposes the picture default reference list (for
> > both list 0 and list 1), the slice reference list and an override
> > flag. The libva will only pass the reference list to be used (so
> > either the picture default's or the slice's) depending on the
> > override flag. The kernel UAPI wants the picture default reference
> > list and the slice reference list, but doesn't expose the override
> > flag, which prevents us from configuring properly the
> > hardware. Our video decoding engine needs the three information,
> > but we can easily adapt to having only one. However, having two
> > doesn't really work for us.
>
> Where does the override flag come from? If it's in the bitstream, then
> I guess it was just missed when creating the structures.
It's in the bitstream yeah. I'll add it then.
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-15 11:59 ` Hans Verkuil
-1 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2018-06-15 11:59 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Guenter Roeck
On 13/06/18 16:07, Maxime Ripard wrote:
> From: Pawel Osciak <posciak@chromium.org>
Obviously this needs a proper commit message.
>
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> Tested-by: Tomasz Figa <tfiga@chromium.org>
> [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> Signed-off-by: Guenter Roeck <groeck@chromium.org>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> drivers/media/v4l2-core/v4l2-ctrls.c | 42 +++++++
> include/media/v4l2-ctrls.h | 10 ++
> include/uapi/linux/v4l2-controls.h | 164 +++++++++++++++++++++++++++
> include/uapi/linux/videodev2.h | 11 ++
> 4 files changed, 227 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index cdf860c8e3d8..1f63c725bad1 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -807,6 +807,11 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER:return "H264 Number of HC Layers";
> case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP:
> return "H264 Set QP Value for HC Layers";
> + case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 SPS";
> + case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 PPS";
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix";
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM: return "H264 Slice Parameters";
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM: return "H264 Decode Parameters";
> case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value";
> @@ -1272,6 +1277,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> case V4L2_CID_RDS_TX_ALT_FREQS:
> *type = V4L2_CTRL_TYPE_U32;
> break;
> + case V4L2_CID_MPEG_VIDEO_H264_SPS:
> + *type = V4L2_CTRL_TYPE_H264_SPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_PPS:
> + *type = V4L2_CTRL_TYPE_H264_PPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX:
> + *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM:
> + *type = V4L2_CTRL_TYPE_H264_SLICE_PARAM;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM:
> + *type = V4L2_CTRL_TYPE_H264_DECODE_PARAM;
> + break;
> case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> *type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> break;
> @@ -1598,6 +1618,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
> case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> return 0;
>
> + case V4L2_CTRL_TYPE_H264_SPS:
> + case V4L2_CTRL_TYPE_H264_PPS:
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
> + return 0;
> +
> default:
> return -EINVAL;
> }
> @@ -2172,6 +2199,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
> case V4L2_CTRL_TYPE_U32:
> elem_size = sizeof(u32);
> break;
> + case V4L2_CTRL_TYPE_H264_SPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_sps);
> + break;
> + case V4L2_CTRL_TYPE_H264_PPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_pps);
> + break;
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix);
> + break;
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
> + elem_size = sizeof(struct v4l2_ctrl_h264_slice_param);
> + break;
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
> + elem_size = sizeof(struct v4l2_ctrl_h264_decode_param);
> + break;
> case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> break;
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index 963c37b02363..9c223793a16a 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -41,6 +41,11 @@ struct poll_table_struct;
> * @p_u16: Pointer to a 16-bit unsigned value.
> * @p_u32: Pointer to a 32-bit unsigned value.
> * @p_char: Pointer to a string.
> + * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps.
> + * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps.
> + * @p_h264_scal_mtrx: Pointer to a struct v4l2_ctrl_h264_scaling_matrix.
> + * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_param.
> + * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_param.
> * @p: Pointer to a compound value.
> */
> union v4l2_ctrl_ptr {
> @@ -50,6 +55,11 @@ union v4l2_ctrl_ptr {
> u16 *p_u16;
> u32 *p_u32;
> char *p_char;
> + struct v4l2_ctrl_h264_sps *p_h264_sps;
> + struct v4l2_ctrl_h264_pps *p_h264_pps;
> + struct v4l2_ctrl_h264_scaling_matrix *p_h264_scal_mtrx;
> + struct v4l2_ctrl_h264_slice_param *p_h264_slice_param;
> + struct v4l2_ctrl_h264_decode_param *p_h264_decode_param;
> void *p;
> };
>
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 23da8bfa7e6f..ac307c59683c 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -50,6 +50,8 @@
> #ifndef __LINUX_V4L2_CONTROLS_H
> #define __LINUX_V4L2_CONTROLS_H
>
> +#include <linux/types.h>
> +
> /* Control classes */
> #define V4L2_CTRL_CLASS_USER 0x00980000 /* Old-style 'user' controls */
> #define V4L2_CTRL_CLASS_MPEG 0x00990000 /* MPEG-compression controls */
> @@ -531,6 +533,12 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type {
> };
> #define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER (V4L2_CID_MPEG_BASE+381)
> #define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP (V4L2_CID_MPEG_BASE+382)
> +#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+383)
> +#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+384)
> +#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+385)
> +#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM (V4L2_CID_MPEG_BASE+386)
> +#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM (V4L2_CID_MPEG_BASE+387)
> +
> #define V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP (V4L2_CID_MPEG_BASE+400)
> #define V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP (V4L2_CID_MPEG_BASE+401)
> #define V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP (V4L2_CID_MPEG_BASE+402)
> @@ -1078,6 +1086,162 @@ enum v4l2_detect_md_mode {
> #define V4L2_CID_DETECT_MD_THRESHOLD_GRID (V4L2_CID_DETECT_CLASS_BASE + 3)
> #define V4L2_CID_DETECT_MD_REGION_GRID (V4L2_CID_DETECT_CLASS_BASE + 4)
>
> +/* Complex controls */
The right term is 'Compounds controls'.
> +
> +#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01
> +#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02
> +#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04
> +#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08
> +#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10
> +#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20
> +
> +#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01
> +#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02
> +#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04
> +#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08
> +#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10
> +#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20
> +#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40
Add a newline here. Same for the other structs below.
> +struct v4l2_ctrl_h264_sps {
> + __u8 profile_idc;
> + __u8 constraint_set_flags;
> + __u8 level_idc;
> + __u8 seq_parameter_set_id;
> + __u8 chroma_format_idc;
> + __u8 bit_depth_luma_minus8;
> + __u8 bit_depth_chroma_minus8;
> + __u8 log2_max_frame_num_minus4;
> + __u8 pic_order_cnt_type;
> + __u8 log2_max_pic_order_cnt_lsb_minus4;
There is a hole in the struct here. Is that OK? Are there alignment requirements?
> + __s32 offset_for_non_ref_pic;
> + __s32 offset_for_top_to_bottom_field;
> + __u8 num_ref_frames_in_pic_order_cnt_cycle;
> + __s32 offset_for_ref_frame[255];
Perhaps use a define instead of hardcoding 255? Not sure if that makes sense.
Same for other arrays below.
> + __u8 max_num_ref_frames;
> + __u16 pic_width_in_mbs_minus1;
> + __u16 pic_height_in_map_units_minus1;
> + __u8 flags;
> +};
You have to test the struct layout for 32 bit and 64 bit systems (the latter for both
64 bit arm and Intel). The layout should be the same for all of them since the
control framework does not support compat32 conversions for compound controls.
> +
> +#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001
> +#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002
> +#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004
> +#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008
> +#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010
> +#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020
> +#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040
> +#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080
> +struct v4l2_ctrl_h264_pps {
> + __u8 pic_parameter_set_id;
> + __u8 seq_parameter_set_id;
> + __u8 num_slice_groups_minus1;
> + __u8 num_ref_idx_l0_default_active_minus1;
> + __u8 num_ref_idx_l1_default_active_minus1;
> + __u8 weighted_bipred_idc;
> + __s8 pic_init_qp_minus26;
> + __s8 pic_init_qs_minus26;
> + __s8 chroma_qp_index_offset;
> + __s8 second_chroma_qp_index_offset;
> + __u8 flags;
> +};
> +
> +struct v4l2_ctrl_h264_scaling_matrix {
> + __u8 scaling_list_4x4[6][16];
> + __u8 scaling_list_8x8[6][64];
> +};
> +
> +struct v4l2_h264_weight_factors {
> + __s8 luma_weight[32];
> + __s8 luma_offset[32];
> + __s8 chroma_weight[32][2];
> + __s8 chroma_offset[32][2];
> +};
> +
> +struct v4l2_h264_pred_weight_table {
> + __u8 luma_log2_weight_denom;
> + __u8 chroma_log2_weight_denom;
> + struct v4l2_h264_weight_factors weight_factors[2];
> +};
> +
> +enum v4l2_h264_slice_type {
> + V4L2_H264_SLICE_TYPE_P = 0,
> + V4L2_H264_SLICE_TYPE_B = 1,
> + V4L2_H264_SLICE_TYPE_I = 2,
> + V4L2_H264_SLICE_TYPE_SP = 3,
> + V4L2_H264_SLICE_TYPE_SI = 4,
> +};
> +
> +#define V4L2_SLICE_FLAG_FIELD_PIC 0x01
> +#define V4L2_SLICE_FLAG_BOTTOM_FIELD 0x02
> +#define V4L2_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04
> +#define V4L2_SLICE_FLAG_SP_FOR_SWITCH 0x08
> +struct v4l2_ctrl_h264_slice_param {
> + /* Size in bytes, including header */
> + __u32 size;
> + /* Offset in bits to slice_data() from the beginning of this slice. */
> + __u32 header_bit_size;
> +
> + __u16 first_mb_in_slice;
> + enum v4l2_h264_slice_type slice_type;
Avoid enums in a struct.
> + __u8 pic_parameter_set_id;
> + __u8 colour_plane_id;
> + __u16 frame_num;
> + __u16 idr_pic_id;
> + __u16 pic_order_cnt_lsb;
> + __s32 delta_pic_order_cnt_bottom;
> + __s32 delta_pic_order_cnt0;
> + __s32 delta_pic_order_cnt1;
> + __u8 redundant_pic_cnt;
> +
> + struct v4l2_h264_pred_weight_table pred_weight_table;
> + /* Size in bits of dec_ref_pic_marking() syntax element. */
> + __u32 dec_ref_pic_marking_bit_size;
> + /* Size in bits of pic order count syntax. */
> + __u32 pic_order_cnt_bit_size;
> +
> + __u8 cabac_init_idc;
> + __s8 slice_qp_delta;
> + __s8 slice_qs_delta;
> + __u8 disable_deblocking_filter_idc;
> + __s8 slice_alpha_c0_offset_div2;
> + __s8 slice_beta_offset_div2;
> + __u32 slice_group_change_cycle;
> +
> + __u8 num_ref_idx_l0_active_minus1;
> + __u8 num_ref_idx_l1_active_minus1;
> + /* Entries on each list are indices
> + * into v4l2_ctrl_h264_decode_param.dpb[]. */
> + __u8 ref_pic_list0[32];
> + __u8 ref_pic_list1[32];
> +
> + __u8 flags;
> +};
> +
> +/* If not set, this entry is unused for reference. */
> +#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x01
> +#define V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM 0x02
> +struct v4l2_h264_dpb_entry {
> + __u32 buf_index; /* v4l2_buffer index */
> + __u16 frame_num;
> + __u16 pic_num;
> + /* Note that field is indicated by v4l2_buffer.field */
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u8 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
> +};
> +
> +struct v4l2_ctrl_h264_decode_param {
> + __u32 num_slices;
> + __u8 idr_pic_flag;
> + __u8 nal_ref_idc;
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u8 ref_pic_list_p0[32];
> + __u8 ref_pic_list_b0[32];
> + __u8 ref_pic_list_b1[32];
> + struct v4l2_h264_dpb_entry dpb[16];
> +};
> +
> struct v4l2_ctrl_mpeg2_frame_hdr {
> __u32 slice_len;
> __u32 slice_pos;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 242a6bfa1440..4b4a1b25a0db 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> #define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */
> #define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */
> #define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */
> @@ -1589,6 +1590,11 @@ struct v4l2_ext_control {
> __u8 __user *p_u8;
> __u16 __user *p_u16;
> __u32 __user *p_u32;
> + struct v4l2_ctrl_h264_sps __user *p_h264_sps;
> + struct v4l2_ctrl_h264_pps __user *p_h264_pps;
> + struct v4l2_ctrl_h264_scaling_matrix __user *p_h264_scal_mtrx;
> + struct v4l2_ctrl_h264_slice_param __user *p_h264_slice_param;
> + struct v4l2_ctrl_h264_decode_param __user *p_h264_decode_param;
> struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
> void __user *ptr;
> };
> @@ -1635,6 +1641,11 @@ enum v4l2_ctrl_type {
> V4L2_CTRL_TYPE_U8 = 0x0100,
> V4L2_CTRL_TYPE_U16 = 0x0101,
> V4L2_CTRL_TYPE_U32 = 0x0102,
> + V4L2_CTRL_TYPE_H264_SPS = 0x0103,
> + V4L2_CTRL_TYPE_H264_PPS = 0x0104,
> + V4L2_CTRL_TYPE_H264_SCALING_MATRIX = 0x0105,
> + V4L2_CTRL_TYPE_H264_SLICE_PARAM = 0x0106,
> + V4L2_CTRL_TYPE_H264_DECODE_PARAM = 0x0107,
> V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> };
>
>
Documentation is also missing, but I assume that will be added later in a separate
patch.
I *think* that the alignments of the structs are the same for all architectures,
but I would feel much happier if you can confirm that by testing this.
Regards,
Hans
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-06-15 11:59 ` Hans Verkuil
0 siblings, 0 replies; 102+ messages in thread
From: Hans Verkuil @ 2018-06-15 11:59 UTC (permalink / raw)
To: linux-arm-kernel
On 13/06/18 16:07, Maxime Ripard wrote:
> From: Pawel Osciak <posciak@chromium.org>
Obviously this needs a proper commit message.
>
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> Tested-by: Tomasz Figa <tfiga@chromium.org>
> [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> Signed-off-by: Guenter Roeck <groeck@chromium.org>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> drivers/media/v4l2-core/v4l2-ctrls.c | 42 +++++++
> include/media/v4l2-ctrls.h | 10 ++
> include/uapi/linux/v4l2-controls.h | 164 +++++++++++++++++++++++++++
> include/uapi/linux/videodev2.h | 11 ++
> 4 files changed, 227 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index cdf860c8e3d8..1f63c725bad1 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -807,6 +807,11 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER:return "H264 Number of HC Layers";
> case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP:
> return "H264 Set QP Value for HC Layers";
> + case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 SPS";
> + case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 PPS";
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix";
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM: return "H264 Slice Parameters";
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM: return "H264 Decode Parameters";
> case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value";
> @@ -1272,6 +1277,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> case V4L2_CID_RDS_TX_ALT_FREQS:
> *type = V4L2_CTRL_TYPE_U32;
> break;
> + case V4L2_CID_MPEG_VIDEO_H264_SPS:
> + *type = V4L2_CTRL_TYPE_H264_SPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_PPS:
> + *type = V4L2_CTRL_TYPE_H264_PPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX:
> + *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM:
> + *type = V4L2_CTRL_TYPE_H264_SLICE_PARAM;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM:
> + *type = V4L2_CTRL_TYPE_H264_DECODE_PARAM;
> + break;
> case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> *type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> break;
> @@ -1598,6 +1618,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
> case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> return 0;
>
> + case V4L2_CTRL_TYPE_H264_SPS:
> + case V4L2_CTRL_TYPE_H264_PPS:
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
> + return 0;
> +
> default:
> return -EINVAL;
> }
> @@ -2172,6 +2199,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
> case V4L2_CTRL_TYPE_U32:
> elem_size = sizeof(u32);
> break;
> + case V4L2_CTRL_TYPE_H264_SPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_sps);
> + break;
> + case V4L2_CTRL_TYPE_H264_PPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_pps);
> + break;
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix);
> + break;
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
> + elem_size = sizeof(struct v4l2_ctrl_h264_slice_param);
> + break;
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
> + elem_size = sizeof(struct v4l2_ctrl_h264_decode_param);
> + break;
> case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> break;
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index 963c37b02363..9c223793a16a 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -41,6 +41,11 @@ struct poll_table_struct;
> * @p_u16: Pointer to a 16-bit unsigned value.
> * @p_u32: Pointer to a 32-bit unsigned value.
> * @p_char: Pointer to a string.
> + * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps.
> + * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps.
> + * @p_h264_scal_mtrx: Pointer to a struct v4l2_ctrl_h264_scaling_matrix.
> + * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_param.
> + * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_param.
> * @p: Pointer to a compound value.
> */
> union v4l2_ctrl_ptr {
> @@ -50,6 +55,11 @@ union v4l2_ctrl_ptr {
> u16 *p_u16;
> u32 *p_u32;
> char *p_char;
> + struct v4l2_ctrl_h264_sps *p_h264_sps;
> + struct v4l2_ctrl_h264_pps *p_h264_pps;
> + struct v4l2_ctrl_h264_scaling_matrix *p_h264_scal_mtrx;
> + struct v4l2_ctrl_h264_slice_param *p_h264_slice_param;
> + struct v4l2_ctrl_h264_decode_param *p_h264_decode_param;
> void *p;
> };
>
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 23da8bfa7e6f..ac307c59683c 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -50,6 +50,8 @@
> #ifndef __LINUX_V4L2_CONTROLS_H
> #define __LINUX_V4L2_CONTROLS_H
>
> +#include <linux/types.h>
> +
> /* Control classes */
> #define V4L2_CTRL_CLASS_USER 0x00980000 /* Old-style 'user' controls */
> #define V4L2_CTRL_CLASS_MPEG 0x00990000 /* MPEG-compression controls */
> @@ -531,6 +533,12 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type {
> };
> #define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER (V4L2_CID_MPEG_BASE+381)
> #define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP (V4L2_CID_MPEG_BASE+382)
> +#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+383)
> +#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+384)
> +#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+385)
> +#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM (V4L2_CID_MPEG_BASE+386)
> +#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM (V4L2_CID_MPEG_BASE+387)
> +
> #define V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP (V4L2_CID_MPEG_BASE+400)
> #define V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP (V4L2_CID_MPEG_BASE+401)
> #define V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP (V4L2_CID_MPEG_BASE+402)
> @@ -1078,6 +1086,162 @@ enum v4l2_detect_md_mode {
> #define V4L2_CID_DETECT_MD_THRESHOLD_GRID (V4L2_CID_DETECT_CLASS_BASE + 3)
> #define V4L2_CID_DETECT_MD_REGION_GRID (V4L2_CID_DETECT_CLASS_BASE + 4)
>
> +/* Complex controls */
The right term is 'Compounds controls'.
> +
> +#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01
> +#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02
> +#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04
> +#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08
> +#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10
> +#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20
> +
> +#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01
> +#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02
> +#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04
> +#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08
> +#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10
> +#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20
> +#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40
Add a newline here. Same for the other structs below.
> +struct v4l2_ctrl_h264_sps {
> + __u8 profile_idc;
> + __u8 constraint_set_flags;
> + __u8 level_idc;
> + __u8 seq_parameter_set_id;
> + __u8 chroma_format_idc;
> + __u8 bit_depth_luma_minus8;
> + __u8 bit_depth_chroma_minus8;
> + __u8 log2_max_frame_num_minus4;
> + __u8 pic_order_cnt_type;
> + __u8 log2_max_pic_order_cnt_lsb_minus4;
There is a hole in the struct here. Is that OK? Are there alignment requirements?
> + __s32 offset_for_non_ref_pic;
> + __s32 offset_for_top_to_bottom_field;
> + __u8 num_ref_frames_in_pic_order_cnt_cycle;
> + __s32 offset_for_ref_frame[255];
Perhaps use a define instead of hardcoding 255? Not sure if that makes sense.
Same for other arrays below.
> + __u8 max_num_ref_frames;
> + __u16 pic_width_in_mbs_minus1;
> + __u16 pic_height_in_map_units_minus1;
> + __u8 flags;
> +};
You have to test the struct layout for 32 bit and 64 bit systems (the latter for both
64 bit arm and Intel). The layout should be the same for all of them since the
control framework does not support compat32 conversions for compound controls.
> +
> +#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001
> +#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002
> +#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004
> +#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008
> +#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010
> +#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020
> +#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040
> +#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080
> +struct v4l2_ctrl_h264_pps {
> + __u8 pic_parameter_set_id;
> + __u8 seq_parameter_set_id;
> + __u8 num_slice_groups_minus1;
> + __u8 num_ref_idx_l0_default_active_minus1;
> + __u8 num_ref_idx_l1_default_active_minus1;
> + __u8 weighted_bipred_idc;
> + __s8 pic_init_qp_minus26;
> + __s8 pic_init_qs_minus26;
> + __s8 chroma_qp_index_offset;
> + __s8 second_chroma_qp_index_offset;
> + __u8 flags;
> +};
> +
> +struct v4l2_ctrl_h264_scaling_matrix {
> + __u8 scaling_list_4x4[6][16];
> + __u8 scaling_list_8x8[6][64];
> +};
> +
> +struct v4l2_h264_weight_factors {
> + __s8 luma_weight[32];
> + __s8 luma_offset[32];
> + __s8 chroma_weight[32][2];
> + __s8 chroma_offset[32][2];
> +};
> +
> +struct v4l2_h264_pred_weight_table {
> + __u8 luma_log2_weight_denom;
> + __u8 chroma_log2_weight_denom;
> + struct v4l2_h264_weight_factors weight_factors[2];
> +};
> +
> +enum v4l2_h264_slice_type {
> + V4L2_H264_SLICE_TYPE_P = 0,
> + V4L2_H264_SLICE_TYPE_B = 1,
> + V4L2_H264_SLICE_TYPE_I = 2,
> + V4L2_H264_SLICE_TYPE_SP = 3,
> + V4L2_H264_SLICE_TYPE_SI = 4,
> +};
> +
> +#define V4L2_SLICE_FLAG_FIELD_PIC 0x01
> +#define V4L2_SLICE_FLAG_BOTTOM_FIELD 0x02
> +#define V4L2_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04
> +#define V4L2_SLICE_FLAG_SP_FOR_SWITCH 0x08
> +struct v4l2_ctrl_h264_slice_param {
> + /* Size in bytes, including header */
> + __u32 size;
> + /* Offset in bits to slice_data() from the beginning of this slice. */
> + __u32 header_bit_size;
> +
> + __u16 first_mb_in_slice;
> + enum v4l2_h264_slice_type slice_type;
Avoid enums in a struct.
> + __u8 pic_parameter_set_id;
> + __u8 colour_plane_id;
> + __u16 frame_num;
> + __u16 idr_pic_id;
> + __u16 pic_order_cnt_lsb;
> + __s32 delta_pic_order_cnt_bottom;
> + __s32 delta_pic_order_cnt0;
> + __s32 delta_pic_order_cnt1;
> + __u8 redundant_pic_cnt;
> +
> + struct v4l2_h264_pred_weight_table pred_weight_table;
> + /* Size in bits of dec_ref_pic_marking() syntax element. */
> + __u32 dec_ref_pic_marking_bit_size;
> + /* Size in bits of pic order count syntax. */
> + __u32 pic_order_cnt_bit_size;
> +
> + __u8 cabac_init_idc;
> + __s8 slice_qp_delta;
> + __s8 slice_qs_delta;
> + __u8 disable_deblocking_filter_idc;
> + __s8 slice_alpha_c0_offset_div2;
> + __s8 slice_beta_offset_div2;
> + __u32 slice_group_change_cycle;
> +
> + __u8 num_ref_idx_l0_active_minus1;
> + __u8 num_ref_idx_l1_active_minus1;
> + /* Entries on each list are indices
> + * into v4l2_ctrl_h264_decode_param.dpb[]. */
> + __u8 ref_pic_list0[32];
> + __u8 ref_pic_list1[32];
> +
> + __u8 flags;
> +};
> +
> +/* If not set, this entry is unused for reference. */
> +#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x01
> +#define V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM 0x02
> +struct v4l2_h264_dpb_entry {
> + __u32 buf_index; /* v4l2_buffer index */
> + __u16 frame_num;
> + __u16 pic_num;
> + /* Note that field is indicated by v4l2_buffer.field */
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u8 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
> +};
> +
> +struct v4l2_ctrl_h264_decode_param {
> + __u32 num_slices;
> + __u8 idr_pic_flag;
> + __u8 nal_ref_idc;
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u8 ref_pic_list_p0[32];
> + __u8 ref_pic_list_b0[32];
> + __u8 ref_pic_list_b1[32];
> + struct v4l2_h264_dpb_entry dpb[16];
> +};
> +
> struct v4l2_ctrl_mpeg2_frame_hdr {
> __u32 slice_len;
> __u32 slice_pos;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 242a6bfa1440..4b4a1b25a0db 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> #define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */
> #define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */
> #define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */
> @@ -1589,6 +1590,11 @@ struct v4l2_ext_control {
> __u8 __user *p_u8;
> __u16 __user *p_u16;
> __u32 __user *p_u32;
> + struct v4l2_ctrl_h264_sps __user *p_h264_sps;
> + struct v4l2_ctrl_h264_pps __user *p_h264_pps;
> + struct v4l2_ctrl_h264_scaling_matrix __user *p_h264_scal_mtrx;
> + struct v4l2_ctrl_h264_slice_param __user *p_h264_slice_param;
> + struct v4l2_ctrl_h264_decode_param __user *p_h264_decode_param;
> struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
> void __user *ptr;
> };
> @@ -1635,6 +1641,11 @@ enum v4l2_ctrl_type {
> V4L2_CTRL_TYPE_U8 = 0x0100,
> V4L2_CTRL_TYPE_U16 = 0x0101,
> V4L2_CTRL_TYPE_U32 = 0x0102,
> + V4L2_CTRL_TYPE_H264_SPS = 0x0103,
> + V4L2_CTRL_TYPE_H264_PPS = 0x0104,
> + V4L2_CTRL_TYPE_H264_SCALING_MATRIX = 0x0105,
> + V4L2_CTRL_TYPE_H264_SLICE_PARAM = 0x0106,
> + V4L2_CTRL_TYPE_H264_DECODE_PARAM = 0x0107,
> V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> };
>
>
Documentation is also missing, but I assume that will be added later in a separate
patch.
I *think* that the alignments of the structs are the same for all architectures,
but I would feel much happier if you can confirm that by testing this.
Regards,
Hans
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-06-15 11:59 ` Hans Verkuil
@ 2018-06-15 13:01 ` Guenter Roeck
-1 siblings, 0 replies; 102+ messages in thread
From: Guenter Roeck @ 2018-06-15 13:01 UTC (permalink / raw)
To: hverkuil
Cc: maxime.ripard, hans.verkuil, acourbot, sakari.ailus,
laurent.pinchart, Tomasz Figa, posciak, paul.kocialkowski, wens,
linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne,
jenskuske, linux-sunxi, thomas.petazzoni, Guenter Roeck
On Fri, Jun 15, 2018 at 5:00 AM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 13/06/18 16:07, Maxime Ripard wrote:
> > From: Pawel Osciak <posciak@chromium.org>
>
> Obviously this needs a proper commit message.
>
> >
> > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
That internal note should go away.
Guenter
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-06-15 13:01 ` Guenter Roeck
0 siblings, 0 replies; 102+ messages in thread
From: Guenter Roeck @ 2018-06-15 13:01 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Jun 15, 2018 at 5:00 AM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 13/06/18 16:07, Maxime Ripard wrote:
> > From: Pawel Osciak <posciak@chromium.org>
>
> Obviously this needs a proper commit message.
>
> >
> > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
That internal note should go away.
Guenter
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-21 8:58 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 8:58 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni, Guenter Roeck
[-- Attachment #1: Type: text/plain, Size: 14543 bytes --]
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> From: Pawel Osciak <posciak@chromium.org>
>
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> Tested-by: Tomasz Figa <tfiga@chromium.org>
> [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> Signed-off-by: Guenter Roeck <groeck@chromium.org>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> drivers/media/v4l2-core/v4l2-ctrls.c | 42 +++++++
> include/media/v4l2-ctrls.h | 10 ++
> include/uapi/linux/v4l2-controls.h | 164 +++++++++++++++++++++++++++
> include/uapi/linux/videodev2.h | 11 ++
> 4 files changed, 227 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index cdf860c8e3d8..1f63c725bad1 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -807,6 +807,11 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER:return "H264 Number of HC Layers";
> case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP:
> return "H264 Set QP Value for HC Layers";
> + case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 SPS";
> + case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 PPS";
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix";
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM: return "H264 Slice Parameters";
The MPEG2 fashion of this control is called
V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_HEADER and it would be good to have
consistency between both.
I think having a _SLICE_PARAMS suffix for both (instead of _SLICE_HEADER
for MPEG2 and _SLICE_PARAM for H264) would make the most sense.
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM: return "H264 Decode Parameters";
Same here, I think having a _DECODE_PARAMS suffix instead would be best.
Cheers,
Paul
> case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value";
> @@ -1272,6 +1277,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> case V4L2_CID_RDS_TX_ALT_FREQS:
> *type = V4L2_CTRL_TYPE_U32;
> break;
> + case V4L2_CID_MPEG_VIDEO_H264_SPS:
> + *type = V4L2_CTRL_TYPE_H264_SPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_PPS:
> + *type = V4L2_CTRL_TYPE_H264_PPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX:
> + *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM:
> + *type = V4L2_CTRL_TYPE_H264_SLICE_PARAM;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM:
> + *type = V4L2_CTRL_TYPE_H264_DECODE_PARAM;
> + break;
> case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> *type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> break;
> @@ -1598,6 +1618,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
> case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> return 0;
>
> + case V4L2_CTRL_TYPE_H264_SPS:
> + case V4L2_CTRL_TYPE_H264_PPS:
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
> + return 0;
> +
> default:
> return -EINVAL;
> }
> @@ -2172,6 +2199,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
> case V4L2_CTRL_TYPE_U32:
> elem_size = sizeof(u32);
> break;
> + case V4L2_CTRL_TYPE_H264_SPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_sps);
> + break;
> + case V4L2_CTRL_TYPE_H264_PPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_pps);
> + break;
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix);
> + break;
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
> + elem_size = sizeof(struct v4l2_ctrl_h264_slice_param);
> + break;
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
> + elem_size = sizeof(struct v4l2_ctrl_h264_decode_param);
> + break;
> case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> break;
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index 963c37b02363..9c223793a16a 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -41,6 +41,11 @@ struct poll_table_struct;
> * @p_u16: Pointer to a 16-bit unsigned value.
> * @p_u32: Pointer to a 32-bit unsigned value.
> * @p_char: Pointer to a string.
> + * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps.
> + * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps.
> + * @p_h264_scal_mtrx: Pointer to a struct v4l2_ctrl_h264_scaling_matrix.
> + * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_param.
> + * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_param.
> * @p: Pointer to a compound value.
> */
> union v4l2_ctrl_ptr {
> @@ -50,6 +55,11 @@ union v4l2_ctrl_ptr {
> u16 *p_u16;
> u32 *p_u32;
> char *p_char;
> + struct v4l2_ctrl_h264_sps *p_h264_sps;
> + struct v4l2_ctrl_h264_pps *p_h264_pps;
> + struct v4l2_ctrl_h264_scaling_matrix *p_h264_scal_mtrx;
> + struct v4l2_ctrl_h264_slice_param *p_h264_slice_param;
> + struct v4l2_ctrl_h264_decode_param *p_h264_decode_param;
> void *p;
> };
>
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 23da8bfa7e6f..ac307c59683c 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -50,6 +50,8 @@
> #ifndef __LINUX_V4L2_CONTROLS_H
> #define __LINUX_V4L2_CONTROLS_H
>
> +#include <linux/types.h>
> +
> /* Control classes */
> #define V4L2_CTRL_CLASS_USER 0x00980000 /* Old-style 'user' controls */
> #define V4L2_CTRL_CLASS_MPEG 0x00990000 /* MPEG-compression controls */
> @@ -531,6 +533,12 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type {
> };
> #define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER (V4L2_CID_MPEG_BASE+381)
> #define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP (V4L2_CID_MPEG_BASE+382)
> +#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+383)
> +#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+384)
> +#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+385)
> +#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM (V4L2_CID_MPEG_BASE+386)
> +#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM (V4L2_CID_MPEG_BASE+387)
> +
> #define V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP (V4L2_CID_MPEG_BASE+400)
> #define V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP (V4L2_CID_MPEG_BASE+401)
> #define V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP (V4L2_CID_MPEG_BASE+402)
> @@ -1078,6 +1086,162 @@ enum v4l2_detect_md_mode {
> #define V4L2_CID_DETECT_MD_THRESHOLD_GRID (V4L2_CID_DETECT_CLASS_BASE + 3)
> #define V4L2_CID_DETECT_MD_REGION_GRID (V4L2_CID_DETECT_CLASS_BASE + 4)
>
> +/* Complex controls */
> +
> +#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01
> +#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02
> +#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04
> +#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08
> +#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10
> +#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20
> +
> +#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01
> +#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02
> +#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04
> +#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08
> +#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10
> +#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20
> +#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40
> +struct v4l2_ctrl_h264_sps {
> + __u8 profile_idc;
> + __u8 constraint_set_flags;
> + __u8 level_idc;
> + __u8 seq_parameter_set_id;
> + __u8 chroma_format_idc;
> + __u8 bit_depth_luma_minus8;
> + __u8 bit_depth_chroma_minus8;
> + __u8 log2_max_frame_num_minus4;
> + __u8 pic_order_cnt_type;
> + __u8 log2_max_pic_order_cnt_lsb_minus4;
> + __s32 offset_for_non_ref_pic;
> + __s32 offset_for_top_to_bottom_field;
> + __u8 num_ref_frames_in_pic_order_cnt_cycle;
> + __s32 offset_for_ref_frame[255];
> + __u8 max_num_ref_frames;
> + __u16 pic_width_in_mbs_minus1;
> + __u16 pic_height_in_map_units_minus1;
> + __u8 flags;
> +};
> +
> +#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001
> +#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002
> +#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004
> +#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008
> +#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010
> +#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020
> +#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040
> +#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080
> +struct v4l2_ctrl_h264_pps {
> + __u8 pic_parameter_set_id;
> + __u8 seq_parameter_set_id;
> + __u8 num_slice_groups_minus1;
> + __u8 num_ref_idx_l0_default_active_minus1;
> + __u8 num_ref_idx_l1_default_active_minus1;
> + __u8 weighted_bipred_idc;
> + __s8 pic_init_qp_minus26;
> + __s8 pic_init_qs_minus26;
> + __s8 chroma_qp_index_offset;
> + __s8 second_chroma_qp_index_offset;
> + __u8 flags;
> +};
> +
> +struct v4l2_ctrl_h264_scaling_matrix {
> + __u8 scaling_list_4x4[6][16];
> + __u8 scaling_list_8x8[6][64];
> +};
> +
> +struct v4l2_h264_weight_factors {
> + __s8 luma_weight[32];
> + __s8 luma_offset[32];
> + __s8 chroma_weight[32][2];
> + __s8 chroma_offset[32][2];
> +};
> +
> +struct v4l2_h264_pred_weight_table {
> + __u8 luma_log2_weight_denom;
> + __u8 chroma_log2_weight_denom;
> + struct v4l2_h264_weight_factors weight_factors[2];
> +};
> +
> +enum v4l2_h264_slice_type {
> + V4L2_H264_SLICE_TYPE_P = 0,
> + V4L2_H264_SLICE_TYPE_B = 1,
> + V4L2_H264_SLICE_TYPE_I = 2,
> + V4L2_H264_SLICE_TYPE_SP = 3,
> + V4L2_H264_SLICE_TYPE_SI = 4,
> +};
>
> +#define V4L2_SLICE_FLAG_FIELD_PIC 0x01
> +#define V4L2_SLICE_FLAG_BOTTOM_FIELD 0x02
> +#define V4L2_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04
> +#define V4L2_SLICE_FLAG_SP_FOR_SWITCH 0x08
> +struct v4l2_ctrl_h264_slice_param {
> + /* Size in bytes, including header */
> + __u32 size;
> + /* Offset in bits to slice_data() from the beginning of this slice. */
> + __u32 header_bit_size;
> +
> + __u16 first_mb_in_slice;
> + enum v4l2_h264_slice_type slice_type;
> + __u8 pic_parameter_set_id;
> + __u8 colour_plane_id;
> + __u16 frame_num;
> + __u16 idr_pic_id;
> + __u16 pic_order_cnt_lsb;
> + __s32 delta_pic_order_cnt_bottom;
> + __s32 delta_pic_order_cnt0;
> + __s32 delta_pic_order_cnt1;
> + __u8 redundant_pic_cnt;
> +
> + struct v4l2_h264_pred_weight_table pred_weight_table;
> + /* Size in bits of dec_ref_pic_marking() syntax element. */
> + __u32 dec_ref_pic_marking_bit_size;
> + /* Size in bits of pic order count syntax. */
> + __u32 pic_order_cnt_bit_size;
> +
> + __u8 cabac_init_idc;
> + __s8 slice_qp_delta;
> + __s8 slice_qs_delta;
> + __u8 disable_deblocking_filter_idc;
> + __s8 slice_alpha_c0_offset_div2;
> + __s8 slice_beta_offset_div2;
> + __u32 slice_group_change_cycle;
> +
> + __u8 num_ref_idx_l0_active_minus1;
> + __u8 num_ref_idx_l1_active_minus1;
> + /* Entries on each list are indices
> + * into v4l2_ctrl_h264_decode_param.dpb[]. */
> + __u8 ref_pic_list0[32];
> + __u8 ref_pic_list1[32];
> +
> + __u8 flags;
> +};
> +
> +/* If not set, this entry is unused for reference. */
> +#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x01
> +#define V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM 0x02
> +struct v4l2_h264_dpb_entry {
> + __u32 buf_index; /* v4l2_buffer index */
> + __u16 frame_num;
> + __u16 pic_num;
> + /* Note that field is indicated by v4l2_buffer.field */
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u8 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
> +};
> +
> +struct v4l2_ctrl_h264_decode_param {
> + __u32 num_slices;
> + __u8 idr_pic_flag;
> + __u8 nal_ref_idc;
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u8 ref_pic_list_p0[32];
> + __u8 ref_pic_list_b0[32];
> + __u8 ref_pic_list_b1[32];
> + struct v4l2_h264_dpb_entry dpb[16];
> +};
> +
> struct v4l2_ctrl_mpeg2_frame_hdr {
> __u32 slice_len;
> __u32 slice_pos;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 242a6bfa1440..4b4a1b25a0db 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> #define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */
> #define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */
> #define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */
> @@ -1589,6 +1590,11 @@ struct v4l2_ext_control {
> __u8 __user *p_u8;
> __u16 __user *p_u16;
> __u32 __user *p_u32;
> + struct v4l2_ctrl_h264_sps __user *p_h264_sps;
> + struct v4l2_ctrl_h264_pps __user *p_h264_pps;
> + struct v4l2_ctrl_h264_scaling_matrix __user *p_h264_scal_mtrx;
> + struct v4l2_ctrl_h264_slice_param __user *p_h264_slice_param;
> + struct v4l2_ctrl_h264_decode_param __user *p_h264_decode_param;
> struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
> void __user *ptr;
> };
> @@ -1635,6 +1641,11 @@ enum v4l2_ctrl_type {
> V4L2_CTRL_TYPE_U8 = 0x0100,
> V4L2_CTRL_TYPE_U16 = 0x0101,
> V4L2_CTRL_TYPE_U32 = 0x0102,
> + V4L2_CTRL_TYPE_H264_SPS = 0x0103,
> + V4L2_CTRL_TYPE_H264_PPS = 0x0104,
> + V4L2_CTRL_TYPE_H264_SCALING_MATRIX = 0x0105,
> + V4L2_CTRL_TYPE_H264_SLICE_PARAM = 0x0106,
> + V4L2_CTRL_TYPE_H264_DECODE_PARAM = 0x0107,
> V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> };
>
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-06-21 8:58 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 8:58 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> From: Pawel Osciak <posciak@chromium.org>
>
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> Tested-by: Tomasz Figa <tfiga@chromium.org>
> [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> Signed-off-by: Guenter Roeck <groeck@chromium.org>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> drivers/media/v4l2-core/v4l2-ctrls.c | 42 +++++++
> include/media/v4l2-ctrls.h | 10 ++
> include/uapi/linux/v4l2-controls.h | 164 +++++++++++++++++++++++++++
> include/uapi/linux/videodev2.h | 11 ++
> 4 files changed, 227 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index cdf860c8e3d8..1f63c725bad1 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -807,6 +807,11 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER:return "H264 Number of HC Layers";
> case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP:
> return "H264 Set QP Value for HC Layers";
> + case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 SPS";
> + case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 PPS";
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix";
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM: return "H264 Slice Parameters";
The MPEG2 fashion of this control is called
V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_HEADER and it would be good to have
consistency between both.
I think having a _SLICE_PARAMS suffix for both (instead of _SLICE_HEADER
for MPEG2 and _SLICE_PARAM for H264) would make the most sense.
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM: return "H264 Decode Parameters";
Same here, I think having a _DECODE_PARAMS suffix instead would be best.
Cheers,
Paul
> case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value";
> case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value";
> @@ -1272,6 +1277,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> case V4L2_CID_RDS_TX_ALT_FREQS:
> *type = V4L2_CTRL_TYPE_U32;
> break;
> + case V4L2_CID_MPEG_VIDEO_H264_SPS:
> + *type = V4L2_CTRL_TYPE_H264_SPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_PPS:
> + *type = V4L2_CTRL_TYPE_H264_PPS;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX:
> + *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM:
> + *type = V4L2_CTRL_TYPE_H264_SLICE_PARAM;
> + break;
> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM:
> + *type = V4L2_CTRL_TYPE_H264_DECODE_PARAM;
> + break;
> case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> *type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> break;
> @@ -1598,6 +1618,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
> case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> return 0;
>
> + case V4L2_CTRL_TYPE_H264_SPS:
> + case V4L2_CTRL_TYPE_H264_PPS:
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
> + return 0;
> +
> default:
> return -EINVAL;
> }
> @@ -2172,6 +2199,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
> case V4L2_CTRL_TYPE_U32:
> elem_size = sizeof(u32);
> break;
> + case V4L2_CTRL_TYPE_H264_SPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_sps);
> + break;
> + case V4L2_CTRL_TYPE_H264_PPS:
> + elem_size = sizeof(struct v4l2_ctrl_h264_pps);
> + break;
> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX:
> + elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix);
> + break;
> + case V4L2_CTRL_TYPE_H264_SLICE_PARAM:
> + elem_size = sizeof(struct v4l2_ctrl_h264_slice_param);
> + break;
> + case V4L2_CTRL_TYPE_H264_DECODE_PARAM:
> + elem_size = sizeof(struct v4l2_ctrl_h264_decode_param);
> + break;
> case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> break;
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index 963c37b02363..9c223793a16a 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -41,6 +41,11 @@ struct poll_table_struct;
> * @p_u16: Pointer to a 16-bit unsigned value.
> * @p_u32: Pointer to a 32-bit unsigned value.
> * @p_char: Pointer to a string.
> + * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps.
> + * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps.
> + * @p_h264_scal_mtrx: Pointer to a struct v4l2_ctrl_h264_scaling_matrix.
> + * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_param.
> + * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_param.
> * @p: Pointer to a compound value.
> */
> union v4l2_ctrl_ptr {
> @@ -50,6 +55,11 @@ union v4l2_ctrl_ptr {
> u16 *p_u16;
> u32 *p_u32;
> char *p_char;
> + struct v4l2_ctrl_h264_sps *p_h264_sps;
> + struct v4l2_ctrl_h264_pps *p_h264_pps;
> + struct v4l2_ctrl_h264_scaling_matrix *p_h264_scal_mtrx;
> + struct v4l2_ctrl_h264_slice_param *p_h264_slice_param;
> + struct v4l2_ctrl_h264_decode_param *p_h264_decode_param;
> void *p;
> };
>
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 23da8bfa7e6f..ac307c59683c 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -50,6 +50,8 @@
> #ifndef __LINUX_V4L2_CONTROLS_H
> #define __LINUX_V4L2_CONTROLS_H
>
> +#include <linux/types.h>
> +
> /* Control classes */
> #define V4L2_CTRL_CLASS_USER 0x00980000 /* Old-style 'user' controls */
> #define V4L2_CTRL_CLASS_MPEG 0x00990000 /* MPEG-compression controls */
> @@ -531,6 +533,12 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type {
> };
> #define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER (V4L2_CID_MPEG_BASE+381)
> #define V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP (V4L2_CID_MPEG_BASE+382)
> +#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+383)
> +#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+384)
> +#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+385)
> +#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAM (V4L2_CID_MPEG_BASE+386)
> +#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAM (V4L2_CID_MPEG_BASE+387)
> +
> #define V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP (V4L2_CID_MPEG_BASE+400)
> #define V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP (V4L2_CID_MPEG_BASE+401)
> #define V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP (V4L2_CID_MPEG_BASE+402)
> @@ -1078,6 +1086,162 @@ enum v4l2_detect_md_mode {
> #define V4L2_CID_DETECT_MD_THRESHOLD_GRID (V4L2_CID_DETECT_CLASS_BASE + 3)
> #define V4L2_CID_DETECT_MD_REGION_GRID (V4L2_CID_DETECT_CLASS_BASE + 4)
>
> +/* Complex controls */
> +
> +#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01
> +#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02
> +#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04
> +#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08
> +#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10
> +#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20
> +
> +#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01
> +#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02
> +#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04
> +#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08
> +#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10
> +#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20
> +#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40
> +struct v4l2_ctrl_h264_sps {
> + __u8 profile_idc;
> + __u8 constraint_set_flags;
> + __u8 level_idc;
> + __u8 seq_parameter_set_id;
> + __u8 chroma_format_idc;
> + __u8 bit_depth_luma_minus8;
> + __u8 bit_depth_chroma_minus8;
> + __u8 log2_max_frame_num_minus4;
> + __u8 pic_order_cnt_type;
> + __u8 log2_max_pic_order_cnt_lsb_minus4;
> + __s32 offset_for_non_ref_pic;
> + __s32 offset_for_top_to_bottom_field;
> + __u8 num_ref_frames_in_pic_order_cnt_cycle;
> + __s32 offset_for_ref_frame[255];
> + __u8 max_num_ref_frames;
> + __u16 pic_width_in_mbs_minus1;
> + __u16 pic_height_in_map_units_minus1;
> + __u8 flags;
> +};
> +
> +#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001
> +#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002
> +#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004
> +#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008
> +#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010
> +#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020
> +#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040
> +#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080
> +struct v4l2_ctrl_h264_pps {
> + __u8 pic_parameter_set_id;
> + __u8 seq_parameter_set_id;
> + __u8 num_slice_groups_minus1;
> + __u8 num_ref_idx_l0_default_active_minus1;
> + __u8 num_ref_idx_l1_default_active_minus1;
> + __u8 weighted_bipred_idc;
> + __s8 pic_init_qp_minus26;
> + __s8 pic_init_qs_minus26;
> + __s8 chroma_qp_index_offset;
> + __s8 second_chroma_qp_index_offset;
> + __u8 flags;
> +};
> +
> +struct v4l2_ctrl_h264_scaling_matrix {
> + __u8 scaling_list_4x4[6][16];
> + __u8 scaling_list_8x8[6][64];
> +};
> +
> +struct v4l2_h264_weight_factors {
> + __s8 luma_weight[32];
> + __s8 luma_offset[32];
> + __s8 chroma_weight[32][2];
> + __s8 chroma_offset[32][2];
> +};
> +
> +struct v4l2_h264_pred_weight_table {
> + __u8 luma_log2_weight_denom;
> + __u8 chroma_log2_weight_denom;
> + struct v4l2_h264_weight_factors weight_factors[2];
> +};
> +
> +enum v4l2_h264_slice_type {
> + V4L2_H264_SLICE_TYPE_P = 0,
> + V4L2_H264_SLICE_TYPE_B = 1,
> + V4L2_H264_SLICE_TYPE_I = 2,
> + V4L2_H264_SLICE_TYPE_SP = 3,
> + V4L2_H264_SLICE_TYPE_SI = 4,
> +};
>
> +#define V4L2_SLICE_FLAG_FIELD_PIC 0x01
> +#define V4L2_SLICE_FLAG_BOTTOM_FIELD 0x02
> +#define V4L2_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04
> +#define V4L2_SLICE_FLAG_SP_FOR_SWITCH 0x08
> +struct v4l2_ctrl_h264_slice_param {
> + /* Size in bytes, including header */
> + __u32 size;
> + /* Offset in bits to slice_data() from the beginning of this slice. */
> + __u32 header_bit_size;
> +
> + __u16 first_mb_in_slice;
> + enum v4l2_h264_slice_type slice_type;
> + __u8 pic_parameter_set_id;
> + __u8 colour_plane_id;
> + __u16 frame_num;
> + __u16 idr_pic_id;
> + __u16 pic_order_cnt_lsb;
> + __s32 delta_pic_order_cnt_bottom;
> + __s32 delta_pic_order_cnt0;
> + __s32 delta_pic_order_cnt1;
> + __u8 redundant_pic_cnt;
> +
> + struct v4l2_h264_pred_weight_table pred_weight_table;
> + /* Size in bits of dec_ref_pic_marking() syntax element. */
> + __u32 dec_ref_pic_marking_bit_size;
> + /* Size in bits of pic order count syntax. */
> + __u32 pic_order_cnt_bit_size;
> +
> + __u8 cabac_init_idc;
> + __s8 slice_qp_delta;
> + __s8 slice_qs_delta;
> + __u8 disable_deblocking_filter_idc;
> + __s8 slice_alpha_c0_offset_div2;
> + __s8 slice_beta_offset_div2;
> + __u32 slice_group_change_cycle;
> +
> + __u8 num_ref_idx_l0_active_minus1;
> + __u8 num_ref_idx_l1_active_minus1;
> + /* Entries on each list are indices
> + * into v4l2_ctrl_h264_decode_param.dpb[]. */
> + __u8 ref_pic_list0[32];
> + __u8 ref_pic_list1[32];
> +
> + __u8 flags;
> +};
> +
> +/* If not set, this entry is unused for reference. */
> +#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x01
> +#define V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM 0x02
> +struct v4l2_h264_dpb_entry {
> + __u32 buf_index; /* v4l2_buffer index */
> + __u16 frame_num;
> + __u16 pic_num;
> + /* Note that field is indicated by v4l2_buffer.field */
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u8 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
> +};
> +
> +struct v4l2_ctrl_h264_decode_param {
> + __u32 num_slices;
> + __u8 idr_pic_flag;
> + __u8 nal_ref_idc;
> + __s32 top_field_order_cnt;
> + __s32 bottom_field_order_cnt;
> + __u8 ref_pic_list_p0[32];
> + __u8 ref_pic_list_b0[32];
> + __u8 ref_pic_list_b1[32];
> + struct v4l2_h264_dpb_entry dpb[16];
> +};
> +
> struct v4l2_ctrl_mpeg2_frame_hdr {
> __u32 slice_len;
> __u32 slice_pos;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 242a6bfa1440..4b4a1b25a0db 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> #define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */
> #define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */
> #define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */
> @@ -1589,6 +1590,11 @@ struct v4l2_ext_control {
> __u8 __user *p_u8;
> __u16 __user *p_u16;
> __u32 __user *p_u32;
> + struct v4l2_ctrl_h264_sps __user *p_h264_sps;
> + struct v4l2_ctrl_h264_pps __user *p_h264_pps;
> + struct v4l2_ctrl_h264_scaling_matrix __user *p_h264_scal_mtrx;
> + struct v4l2_ctrl_h264_slice_param __user *p_h264_slice_param;
> + struct v4l2_ctrl_h264_decode_param __user *p_h264_decode_param;
> struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
> void __user *ptr;
> };
> @@ -1635,6 +1641,11 @@ enum v4l2_ctrl_type {
> V4L2_CTRL_TYPE_U8 = 0x0100,
> V4L2_CTRL_TYPE_U16 = 0x0101,
> V4L2_CTRL_TYPE_U32 = 0x0102,
> + V4L2_CTRL_TYPE_H264_SPS = 0x0103,
> + V4L2_CTRL_TYPE_H264_PPS = 0x0104,
> + V4L2_CTRL_TYPE_H264_SCALING_MATRIX = 0x0105,
> + V4L2_CTRL_TYPE_H264_SLICE_PARAM = 0x0106,
> + V4L2_CTRL_TYPE_H264_DECODE_PARAM = 0x0107,
> V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> };
>
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180621/ced8e9d4/attachment-0001.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 2/9] media: cedrus: Add wrappers around container_of for our buffers
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-21 9:03 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 9:03 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 2592 bytes --]
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> The cedrus driver sub-classes the vb2_v4l2_buffer structure, but doesn't
> provide any function to wrap around the proper container_of call that needs
> to be duplicated in every calling site.
>
> Add wrappers to make sure its opaque to the users and they don't have to
> care anymore.
>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../platform/sunxi/cedrus/sunxi_cedrus_common.h | 12 ++++++++++++
> .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 4 ++--
> 2 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index ee6883ef9cb7..b1ed1c8cb130 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -89,6 +89,18 @@ struct sunxi_cedrus_buffer {
> struct list_head list;
> };
>
> +static inline
I'm not a big fan of spreading the type qualifiers on two distinct lines
instead of having a longer single line (reminds me of the GNU coding
style a bit), but this is purely cosmetic.
Cheers,
Paul
> +struct sunxi_cedrus_buffer *vb2_v4l2_to_cedrus_buffer(const struct vb2_v4l2_buffer *p)
> +{
> + return container_of(p, struct sunxi_cedrus_buffer, vb);
> +}
> +
> +static inline
> +struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> +{
> + return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> +}
> +
> struct sunxi_cedrus_dev {
> struct v4l2_device v4l2_dev;
> struct video_device vfd;
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> index 5783bd985855..fc688a5c1ea3 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> @@ -108,8 +108,8 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> return IRQ_HANDLED;
> }
>
> - src_buffer = container_of(src_vb, struct sunxi_cedrus_buffer, vb);
> - dst_buffer = container_of(dst_vb, struct sunxi_cedrus_buffer, vb);
> + src_buffer = vb2_v4l2_to_cedrus_buffer(src_vb);
> + dst_buffer = vb2_v4l2_to_cedrus_buffer(dst_vb);
>
> /* First bit of MPEG_STATUS indicates success. */
> if (ctx->job_abort || !(status & 0x01))
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 2/9] media: cedrus: Add wrappers around container_of for our buffers
@ 2018-06-21 9:03 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 9:03 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> The cedrus driver sub-classes the vb2_v4l2_buffer structure, but doesn't
> provide any function to wrap around the proper container_of call that needs
> to be duplicated in every calling site.
>
> Add wrappers to make sure its opaque to the users and they don't have to
> care anymore.
>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../platform/sunxi/cedrus/sunxi_cedrus_common.h | 12 ++++++++++++
> .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 4 ++--
> 2 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index ee6883ef9cb7..b1ed1c8cb130 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -89,6 +89,18 @@ struct sunxi_cedrus_buffer {
> struct list_head list;
> };
>
> +static inline
I'm not a big fan of spreading the type qualifiers on two distinct lines
instead of having a longer single line (reminds me of the GNU coding
style a bit), but this is purely cosmetic.
Cheers,
Paul
> +struct sunxi_cedrus_buffer *vb2_v4l2_to_cedrus_buffer(const struct vb2_v4l2_buffer *p)
> +{
> + return container_of(p, struct sunxi_cedrus_buffer, vb);
> +}
> +
> +static inline
> +struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> +{
> + return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> +}
> +
> struct sunxi_cedrus_dev {
> struct v4l2_device v4l2_dev;
> struct video_device vfd;
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> index 5783bd985855..fc688a5c1ea3 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> @@ -108,8 +108,8 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> return IRQ_HANDLED;
> }
>
> - src_buffer = container_of(src_vb, struct sunxi_cedrus_buffer, vb);
> - dst_buffer = container_of(dst_vb, struct sunxi_cedrus_buffer, vb);
> + src_buffer = vb2_v4l2_to_cedrus_buffer(src_vb);
> + dst_buffer = vb2_v4l2_to_cedrus_buffer(dst_vb);
>
> /* First bit of MPEG_STATUS indicates success. */
> if (ctx->job_abort || !(status & 0x01))
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180621/33933c40/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 3/9] media: cedrus: Add a macro to check for the validity of a control
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-21 9:13 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 9:13 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 2656 bytes --]
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> During our frame decoding setup, we need to check a number of controls to
> make sure that they are properly filled before trying to access them.
>
> It's not too bad with MPEG2 since there's just a single one, but with the
> upcoming increase of codecs, and the integration of more complex codecs,
> this logic will be duplicated a significant number of times. H264 for
> example uses 4 different controls.
>
> Add a macro that expands to the proper check in order to reduce the
> duplication.
>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 16 +++++++++-------
> 1 file changed, 9 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> index 8c92af34ebeb..c19acf9626c4 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> @@ -110,15 +110,16 @@ void sunxi_cedrus_device_run(void *priv)
>
> spin_lock_irqsave(&ctx->dev->irq_lock, flags);
>
> +#define CHECK_CONTROL(ctx, ctrl) \
> + if (!ctx->ctrls[(ctrl)]) { \
Although this was not introduced in this patch, I believe this approach
won't work since ctx->ctrls[i] is a pointer returned from
v4l2_ctrl_new_custom(hdl, &cfg, NULL), which will always be non-null
after calling cedrus_init_ctrls.
Perhaps checking ctx->ctrls[i].p_cur.p would be the right thing to
check, but I'm unsure about that.
> + v4l2_err(&(ctx)->dev->v4l2_dev, "Invalid " #ctrl " control\n"); \
> + (ctx)->job_abort = 1; \
> + goto unlock_complete; \
> + }
> +
> switch (ctx->vpu_src_fmt->fourcc) {
> case V4L2_PIX_FMT_MPEG2_FRAME:
> - if (!ctx->ctrls[SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR]) {
> - v4l2_err(&ctx->dev->v4l2_dev,
> - "Invalid MPEG2 frame header control\n");
> - ctx->job_abort = 1;
> - goto unlock_complete;
> - }
> -
> + CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
> run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
> sunxi_cedrus_mpeg2_setup(ctx, &run);
>
> @@ -128,6 +129,7 @@ void sunxi_cedrus_device_run(void *priv)
> default:
> ctx->job_abort = 1;
> }
Maybe add a newline here?
Cheers,
Paul
> +#undef CHECK_CONTROL
>
> unlock_complete:
> spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 3/9] media: cedrus: Add a macro to check for the validity of a control
@ 2018-06-21 9:13 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 9:13 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> During our frame decoding setup, we need to check a number of controls to
> make sure that they are properly filled before trying to access them.
>
> It's not too bad with MPEG2 since there's just a single one, but with the
> upcoming increase of codecs, and the integration of more complex codecs,
> this logic will be duplicated a significant number of times. H264 for
> example uses 4 different controls.
>
> Add a macro that expands to the proper check in order to reduce the
> duplication.
>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 16 +++++++++-------
> 1 file changed, 9 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> index 8c92af34ebeb..c19acf9626c4 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> @@ -110,15 +110,16 @@ void sunxi_cedrus_device_run(void *priv)
>
> spin_lock_irqsave(&ctx->dev->irq_lock, flags);
>
> +#define CHECK_CONTROL(ctx, ctrl) \
> + if (!ctx->ctrls[(ctrl)]) { \
Although this was not introduced in this patch, I believe this approach
won't work since ctx->ctrls[i] is a pointer returned from
v4l2_ctrl_new_custom(hdl, &cfg, NULL), which will always be non-null
after calling cedrus_init_ctrls.
Perhaps checking ctx->ctrls[i].p_cur.p would be the right thing to
check, but I'm unsure about that.
> + v4l2_err(&(ctx)->dev->v4l2_dev, "Invalid " #ctrl " control\n"); \
> + (ctx)->job_abort = 1; \
> + goto unlock_complete; \
> + }
> +
> switch (ctx->vpu_src_fmt->fourcc) {
> case V4L2_PIX_FMT_MPEG2_FRAME:
> - if (!ctx->ctrls[SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR]) {
> - v4l2_err(&ctx->dev->v4l2_dev,
> - "Invalid MPEG2 frame header control\n");
> - ctx->job_abort = 1;
> - goto unlock_complete;
> - }
> -
> + CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
> run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
> sunxi_cedrus_mpeg2_setup(ctx, &run);
>
> @@ -128,6 +129,7 @@ void sunxi_cedrus_device_run(void *priv)
> default:
> ctx->job_abort = 1;
> }
Maybe add a newline here?
Cheers,
Paul
> +#undef CHECK_CONTROL
>
> unlock_complete:
> spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180621/b3b35974/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 4/9] media: cedrus: make engine type more generic
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-21 9:33 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 9:33 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 3982 bytes --]
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> The sunxi_cedrus_engine enum actually enumerates pretty much the codecs to
> use (or we can easily infer the codec engine from the codec).
>
> Since we will need the codec type as well in some later refactoring, make
> that structure more useful by just enumerating the codec, and converting
> the existing users.
With the comment below taken in account, this is:
Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h | 6 ++++++
> drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 6 +++---
> drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h | 6 +-----
> drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c | 2 +-
> 4 files changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index b1ed1c8cb130..a5f83c452006 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -61,6 +61,12 @@ struct sunxi_cedrus_run {
> };
> };
>
> +enum sunxi_cedrus_codec {
Feel free to rename to cedrus_codec when rebasing on top of the latest
patchset introducing the driver.
Cheers,
Paul
> + SUNXI_CEDRUS_CODEC_MPEG2,
> +
> + SUNXI_CEDRUS_CODEC_LAST,
> +};
> +
> struct sunxi_cedrus_ctx {
> struct v4l2_fh fh;
> struct sunxi_cedrus_dev *dev;
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> index fc688a5c1ea3..bb46a01214e0 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> @@ -41,7 +41,7 @@
> #define SYSCON_SRAM_C1_MAP_VE 0x7fffffff
>
> int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
> - enum sunxi_cedrus_engine engine)
> + enum sunxi_cedrus_codec codec)
> {
> u32 reg = 0;
>
> @@ -53,8 +53,8 @@ int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
>
> reg |= VE_CTRL_CACHE_BUS_BW_128;
>
> - switch (engine) {
> - case SUNXI_CEDRUS_ENGINE_MPEG:
> + switch (codec) {
> + case SUNXI_CEDRUS_CODEC_MPEG2:
> reg |= VE_CTRL_DEC_MODE_MPEG;
> break;
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
> index 34f3fae462a8..3236c80bfcf4 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
> @@ -23,12 +23,8 @@
> #ifndef _SUNXI_CEDRUS_HW_H_
> #define _SUNXI_CEDRUS_HW_H_
>
> -enum sunxi_cedrus_engine {
> - SUNXI_CEDRUS_ENGINE_MPEG,
> -};
> -
> int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
> - enum sunxi_cedrus_engine engine);
> + enum sunxi_cedrus_codec codec);
> void sunxi_cedrus_engine_disable(struct sunxi_cedrus_dev *dev);
>
> int sunxi_cedrus_hw_probe(struct sunxi_cedrus_dev *dev);
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> index 5be3e3b9ceef..85e6fc2fbdb2 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> @@ -83,7 +83,7 @@ void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> }
>
> /* Activate MPEG engine. */
> - sunxi_cedrus_engine_enable(dev, SUNXI_CEDRUS_ENGINE_MPEG);
> + sunxi_cedrus_engine_enable(dev, SUNXI_CEDRUS_CODEC_MPEG2);
>
> /* Set quantization matrices. */
> for (i = 0; i < 64; i++) {
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 4/9] media: cedrus: make engine type more generic
@ 2018-06-21 9:33 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 9:33 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> The sunxi_cedrus_engine enum actually enumerates pretty much the codecs to
> use (or we can easily infer the codec engine from the codec).
>
> Since we will need the codec type as well in some later refactoring, make
> that structure more useful by just enumerating the codec, and converting
> the existing users.
With the comment below taken in account, this is:
Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h | 6 ++++++
> drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c | 6 +++---
> drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h | 6 +-----
> drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c | 2 +-
> 4 files changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index b1ed1c8cb130..a5f83c452006 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -61,6 +61,12 @@ struct sunxi_cedrus_run {
> };
> };
>
> +enum sunxi_cedrus_codec {
Feel free to rename to cedrus_codec when rebasing on top of the latest
patchset introducing the driver.
Cheers,
Paul
> + SUNXI_CEDRUS_CODEC_MPEG2,
> +
> + SUNXI_CEDRUS_CODEC_LAST,
> +};
> +
> struct sunxi_cedrus_ctx {
> struct v4l2_fh fh;
> struct sunxi_cedrus_dev *dev;
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> index fc688a5c1ea3..bb46a01214e0 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> @@ -41,7 +41,7 @@
> #define SYSCON_SRAM_C1_MAP_VE 0x7fffffff
>
> int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
> - enum sunxi_cedrus_engine engine)
> + enum sunxi_cedrus_codec codec)
> {
> u32 reg = 0;
>
> @@ -53,8 +53,8 @@ int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
>
> reg |= VE_CTRL_CACHE_BUS_BW_128;
>
> - switch (engine) {
> - case SUNXI_CEDRUS_ENGINE_MPEG:
> + switch (codec) {
> + case SUNXI_CEDRUS_CODEC_MPEG2:
> reg |= VE_CTRL_DEC_MODE_MPEG;
> break;
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
> index 34f3fae462a8..3236c80bfcf4 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
> @@ -23,12 +23,8 @@
> #ifndef _SUNXI_CEDRUS_HW_H_
> #define _SUNXI_CEDRUS_HW_H_
>
> -enum sunxi_cedrus_engine {
> - SUNXI_CEDRUS_ENGINE_MPEG,
> -};
> -
> int sunxi_cedrus_engine_enable(struct sunxi_cedrus_dev *dev,
> - enum sunxi_cedrus_engine engine);
> + enum sunxi_cedrus_codec codec);
> void sunxi_cedrus_engine_disable(struct sunxi_cedrus_dev *dev);
>
> int sunxi_cedrus_hw_probe(struct sunxi_cedrus_dev *dev);
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> index 5be3e3b9ceef..85e6fc2fbdb2 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> @@ -83,7 +83,7 @@ void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> }
>
> /* Activate MPEG engine. */
> - sunxi_cedrus_engine_enable(dev, SUNXI_CEDRUS_ENGINE_MPEG);
> + sunxi_cedrus_engine_enable(dev, SUNXI_CEDRUS_CODEC_MPEG2);
>
> /* Set quantization matrices. */
> for (i = 0; i < 64; i++) {
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180621/8ee3cf01/attachment-0001.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 6/9] media: cedrus: Add ops structure
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-21 9:49 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 9:49 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 10462 bytes --]
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> In order to increase the number of codecs supported, we need to decouple
> the MPEG2 only code that was there up until now and turn it into something
> a bit more generic.
>
> Do that by introducing an intermediate ops structure that would need to be
> filled by each supported codec. Start by implementing in that structure the
> setup and trigger hooks that are currently the only functions being
> implemented by codecs support.
>
> To do so, we need to store the current codec in use, which we do at
> start_streaming time.
With the comments below taken in account, this is:
Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../platform/sunxi/cedrus/sunxi_cedrus.c | 2 ++
> .../sunxi/cedrus/sunxi_cedrus_common.h | 11 +++++++
> .../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 10 +++---
> .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 11 +++++--
> .../sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 -------------------
> .../sunxi/cedrus/sunxi_cedrus_video.c | 17 +++++++++-
> 6 files changed, 42 insertions(+), 42 deletions(-)
> delete mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> index ccd41d9a3e41..bc80480f5dfd 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> @@ -244,6 +244,8 @@ static int sunxi_cedrus_probe(struct platform_device *pdev)
> if (ret)
> return ret;
>
> + dev->dec_ops[SUNXI_CEDRUS_CODEC_MPEG2] = &sunxi_cedrus_dec_ops_mpeg2;
> +
> ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> if (ret)
> goto unreg_media;
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index a5f83c452006..c2e2c92d103b 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -75,6 +75,7 @@ struct sunxi_cedrus_ctx {
> struct v4l2_pix_format_mplane src_fmt;
> struct sunxi_cedrus_fmt *vpu_dst_fmt;
> struct v4l2_pix_format_mplane dst_fmt;
> + enum sunxi_cedrus_codec current_codec;
Nit: for consistency with the way things are named, "codec_current"
probably makes more sense.
IMO using the natural English order is fine for temporary variables, but
less so for variables used in common parts like structures. This allows
seeing "_" as a logical hierarchical delimiter that automatically makes
us end up with consistent prefixes that can easily be grepped for and
derived.
But that's just my 2 cents, it's really not a big deal, especially in
this case!
> struct v4l2_ctrl_handler hdl;
> struct v4l2_ctrl *ctrls[SUNXI_CEDRUS_CTRL_MAX];
> @@ -107,6 +108,14 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> }
>
> +struct sunxi_cedrus_dec_ops {
> + void (*setup)(struct sunxi_cedrus_ctx *ctx,
> + struct sunxi_cedrus_run *run);
> + void (*trigger)(struct sunxi_cedrus_ctx *ctx);
By the way, are we sure that these functions won't ever fail?
I think this is the case for MPEG2 (there is virtually nothing to check
for errors) but perhaps it's different for H264.
> +};
> +
> +extern struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2;
> +
> struct sunxi_cedrus_dev {
> struct v4l2_device v4l2_dev;
> struct video_device vfd;
> @@ -130,6 +139,8 @@ struct sunxi_cedrus_dev {
> struct reset_control *rstc;
>
> struct regmap *syscon;
> +
> + struct sunxi_cedrus_dec_ops *dec_ops[SUNXI_CEDRUS_CODEC_LAST];
> };
>
> static inline void sunxi_cedrus_write(struct sunxi_cedrus_dev *dev,
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> index f274408ab5a7..5e552fa05274 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> @@ -28,7 +28,6 @@
> #include <media/v4l2-mem2mem.h>
>
> #include "sunxi_cedrus_common.h"
> -#include "sunxi_cedrus_mpeg2.h"
> #include "sunxi_cedrus_dec.h"
> #include "sunxi_cedrus_hw.h"
>
> @@ -77,6 +76,7 @@ void sunxi_cedrus_device_work(struct work_struct *work)
> void sunxi_cedrus_device_run(void *priv)
> {
> struct sunxi_cedrus_ctx *ctx = priv;
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> struct sunxi_cedrus_run run = { 0 };
> struct media_request *src_req, *dst_req;
> unsigned long flags;
> @@ -120,8 +120,6 @@ void sunxi_cedrus_device_run(void *priv)
> case V4L2_PIX_FMT_MPEG2_FRAME:
> CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
> run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
> - sunxi_cedrus_mpeg2_setup(ctx, &run);
> -
> break;
>
> default:
> @@ -129,6 +127,9 @@ void sunxi_cedrus_device_run(void *priv)
> }
> #undef CHECK_CONTROL
>
> + if (!ctx->job_abort)
> + dev->dec_ops[ctx->current_codec]->setup(ctx, &run);
> +
> unlock_complete:
> spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
>
> @@ -143,8 +144,7 @@ void sunxi_cedrus_device_run(void *priv)
> spin_lock_irqsave(&ctx->dev->irq_lock, flags);
>
> if (!ctx->job_abort) {
> - if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME)
> - sunxi_cedrus_mpeg2_trigger(ctx);
> + dev->dec_ops[ctx->current_codec]->trigger(ctx);
> } else {
> v4l2_m2m_buf_done(run.src, VB2_BUF_STATE_ERROR);
> v4l2_m2m_buf_done(run.dst, VB2_BUF_STATE_ERROR);
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> index d1d7a3cfce0d..e25075bb5779 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> @@ -52,8 +52,8 @@ static const u8 mpeg_default_non_intra_quant[64] = {
>
> #define m_niq(i) ((i << 8) | mpeg_default_non_intra_quant[i])
>
> -void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> - struct sunxi_cedrus_run *run)
> +static void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> + struct sunxi_cedrus_run *run)
> {
> struct sunxi_cedrus_dev *dev = ctx->dev;
> const struct v4l2_ctrl_mpeg2_frame_hdr *frame_hdr = run->mpeg2.hdr;
> @@ -148,9 +148,14 @@ void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_MPEG_VLD_END);
> }
>
> -void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
> +static void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
> {
> struct sunxi_cedrus_dev *dev = ctx->dev;
>
> sunxi_cedrus_write(dev, VE_TRIG_MPEG2, VE_MPEG_TRIGGER);
> }
> +
> +struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2 = {
> + .setup = sunxi_cedrus_mpeg2_setup,
> + .trigger = sunxi_cedrus_mpeg2_trigger,
> +};
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
> deleted file mode 100644
> index 4c380becfa1a..000000000000
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
> +++ /dev/null
> @@ -1,33 +0,0 @@
> -/*
> - * Sunxi-Cedrus VPU driver
> - *
> - * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> - * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
> - *
> - * Based on the vim2m driver, that is:
> - *
> - * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
> - * Pawel Osciak, <pawel@osciak.com>
> - * Marek Szyprowski, <m.szyprowski@samsung.com>
> - *
> - * This software is licensed under the terms of the GNU General Public
> - * License version 2, as published by the Free Software Foundation, and
> - * may be copied, distributed, and modified under those terms.
> - *
> - * This program is distributed in the hope that it will be useful,
> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> - * GNU General Public License for more details.
> - */
> -
> -#ifndef _SUNXI_CEDRUS_MPEG2_H_
> -#define _SUNXI_CEDRUS_MPEG2_H_
> -
> -struct sunxi_cedrus_ctx;
> -struct sunxi_cedrus_run;
> -
> -void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> - struct sunxi_cedrus_run *run);
> -void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx);
> -
> -#endif
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> index 089abfe6bfeb..fb7b081a5bb7 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> @@ -28,7 +28,6 @@
> #include <media/v4l2-mem2mem.h>
>
> #include "sunxi_cedrus_common.h"
> -#include "sunxi_cedrus_mpeg2.h"
> #include "sunxi_cedrus_dec.h"
> #include "sunxi_cedrus_hw.h"
>
> @@ -414,6 +413,21 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
> return 0;
> }
>
> +static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> + struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> +
> + switch (ctx->vpu_src_fmt->fourcc) {
> + case V4L2_PIX_FMT_MPEG2_FRAME:
> + ctx->current_codec = SUNXI_CEDRUS_CODEC_MPEG2;
> + break;
> + default:
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> static void sunxi_cedrus_stop_streaming(struct vb2_queue *q)
> {
> struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> @@ -462,6 +476,7 @@ static struct vb2_ops sunxi_cedrus_qops = {
> .buf_cleanup = sunxi_cedrus_buf_cleanup,
> .buf_queue = sunxi_cedrus_buf_queue,
> .buf_request_complete = sunxi_cedrus_buf_request_complete,
> + .start_streaming = sunxi_cedrus_start_streaming,
> .stop_streaming = sunxi_cedrus_stop_streaming,
> .wait_prepare = vb2_ops_wait_prepare,
> .wait_finish = vb2_ops_wait_finish,
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 6/9] media: cedrus: Add ops structure
@ 2018-06-21 9:49 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 9:49 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> In order to increase the number of codecs supported, we need to decouple
> the MPEG2 only code that was there up until now and turn it into something
> a bit more generic.
>
> Do that by introducing an intermediate ops structure that would need to be
> filled by each supported codec. Start by implementing in that structure the
> setup and trigger hooks that are currently the only functions being
> implemented by codecs support.
>
> To do so, we need to store the current codec in use, which we do at
> start_streaming time.
With the comments below taken in account, this is:
Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../platform/sunxi/cedrus/sunxi_cedrus.c | 2 ++
> .../sunxi/cedrus/sunxi_cedrus_common.h | 11 +++++++
> .../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 10 +++---
> .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 11 +++++--
> .../sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 -------------------
> .../sunxi/cedrus/sunxi_cedrus_video.c | 17 +++++++++-
> 6 files changed, 42 insertions(+), 42 deletions(-)
> delete mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> index ccd41d9a3e41..bc80480f5dfd 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> @@ -244,6 +244,8 @@ static int sunxi_cedrus_probe(struct platform_device *pdev)
> if (ret)
> return ret;
>
> + dev->dec_ops[SUNXI_CEDRUS_CODEC_MPEG2] = &sunxi_cedrus_dec_ops_mpeg2;
> +
> ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> if (ret)
> goto unreg_media;
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index a5f83c452006..c2e2c92d103b 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -75,6 +75,7 @@ struct sunxi_cedrus_ctx {
> struct v4l2_pix_format_mplane src_fmt;
> struct sunxi_cedrus_fmt *vpu_dst_fmt;
> struct v4l2_pix_format_mplane dst_fmt;
> + enum sunxi_cedrus_codec current_codec;
Nit: for consistency with the way things are named, "codec_current"
probably makes more sense.
IMO using the natural English order is fine for temporary variables, but
less so for variables used in common parts like structures. This allows
seeing "_" as a logical hierarchical delimiter that automatically makes
us end up with consistent prefixes that can easily be grepped for and
derived.
But that's just my 2 cents, it's really not a big deal, especially in
this case!
> struct v4l2_ctrl_handler hdl;
> struct v4l2_ctrl *ctrls[SUNXI_CEDRUS_CTRL_MAX];
> @@ -107,6 +108,14 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> }
>
> +struct sunxi_cedrus_dec_ops {
> + void (*setup)(struct sunxi_cedrus_ctx *ctx,
> + struct sunxi_cedrus_run *run);
> + void (*trigger)(struct sunxi_cedrus_ctx *ctx);
By the way, are we sure that these functions won't ever fail?
I think this is the case for MPEG2 (there is virtually nothing to check
for errors) but perhaps it's different for H264.
> +};
> +
> +extern struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2;
> +
> struct sunxi_cedrus_dev {
> struct v4l2_device v4l2_dev;
> struct video_device vfd;
> @@ -130,6 +139,8 @@ struct sunxi_cedrus_dev {
> struct reset_control *rstc;
>
> struct regmap *syscon;
> +
> + struct sunxi_cedrus_dec_ops *dec_ops[SUNXI_CEDRUS_CODEC_LAST];
> };
>
> static inline void sunxi_cedrus_write(struct sunxi_cedrus_dev *dev,
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> index f274408ab5a7..5e552fa05274 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
> @@ -28,7 +28,6 @@
> #include <media/v4l2-mem2mem.h>
>
> #include "sunxi_cedrus_common.h"
> -#include "sunxi_cedrus_mpeg2.h"
> #include "sunxi_cedrus_dec.h"
> #include "sunxi_cedrus_hw.h"
>
> @@ -77,6 +76,7 @@ void sunxi_cedrus_device_work(struct work_struct *work)
> void sunxi_cedrus_device_run(void *priv)
> {
> struct sunxi_cedrus_ctx *ctx = priv;
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> struct sunxi_cedrus_run run = { 0 };
> struct media_request *src_req, *dst_req;
> unsigned long flags;
> @@ -120,8 +120,6 @@ void sunxi_cedrus_device_run(void *priv)
> case V4L2_PIX_FMT_MPEG2_FRAME:
> CHECK_CONTROL(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
> run.mpeg2.hdr = get_ctrl_ptr(ctx, SUNXI_CEDRUS_CTRL_DEC_MPEG2_FRAME_HDR);
> - sunxi_cedrus_mpeg2_setup(ctx, &run);
> -
> break;
>
> default:
> @@ -129,6 +127,9 @@ void sunxi_cedrus_device_run(void *priv)
> }
> #undef CHECK_CONTROL
>
> + if (!ctx->job_abort)
> + dev->dec_ops[ctx->current_codec]->setup(ctx, &run);
> +
> unlock_complete:
> spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
>
> @@ -143,8 +144,7 @@ void sunxi_cedrus_device_run(void *priv)
> spin_lock_irqsave(&ctx->dev->irq_lock, flags);
>
> if (!ctx->job_abort) {
> - if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME)
> - sunxi_cedrus_mpeg2_trigger(ctx);
> + dev->dec_ops[ctx->current_codec]->trigger(ctx);
> } else {
> v4l2_m2m_buf_done(run.src, VB2_BUF_STATE_ERROR);
> v4l2_m2m_buf_done(run.dst, VB2_BUF_STATE_ERROR);
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> index d1d7a3cfce0d..e25075bb5779 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> @@ -52,8 +52,8 @@ static const u8 mpeg_default_non_intra_quant[64] = {
>
> #define m_niq(i) ((i << 8) | mpeg_default_non_intra_quant[i])
>
> -void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> - struct sunxi_cedrus_run *run)
> +static void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> + struct sunxi_cedrus_run *run)
> {
> struct sunxi_cedrus_dev *dev = ctx->dev;
> const struct v4l2_ctrl_mpeg2_frame_hdr *frame_hdr = run->mpeg2.hdr;
> @@ -148,9 +148,14 @@ void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_MPEG_VLD_END);
> }
>
> -void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
> +static void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
> {
> struct sunxi_cedrus_dev *dev = ctx->dev;
>
> sunxi_cedrus_write(dev, VE_TRIG_MPEG2, VE_MPEG_TRIGGER);
> }
> +
> +struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2 = {
> + .setup = sunxi_cedrus_mpeg2_setup,
> + .trigger = sunxi_cedrus_mpeg2_trigger,
> +};
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
> deleted file mode 100644
> index 4c380becfa1a..000000000000
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
> +++ /dev/null
> @@ -1,33 +0,0 @@
> -/*
> - * Sunxi-Cedrus VPU driver
> - *
> - * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> - * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
> - *
> - * Based on the vim2m driver, that is:
> - *
> - * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
> - * Pawel Osciak, <pawel@osciak.com>
> - * Marek Szyprowski, <m.szyprowski@samsung.com>
> - *
> - * This software is licensed under the terms of the GNU General Public
> - * License version 2, as published by the Free Software Foundation, and
> - * may be copied, distributed, and modified under those terms.
> - *
> - * This program is distributed in the hope that it will be useful,
> - * but WITHOUT ANY WARRANTY; without even the implied warranty of
> - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> - * GNU General Public License for more details.
> - */
> -
> -#ifndef _SUNXI_CEDRUS_MPEG2_H_
> -#define _SUNXI_CEDRUS_MPEG2_H_
> -
> -struct sunxi_cedrus_ctx;
> -struct sunxi_cedrus_run;
> -
> -void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> - struct sunxi_cedrus_run *run);
> -void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx);
> -
> -#endif
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> index 089abfe6bfeb..fb7b081a5bb7 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> @@ -28,7 +28,6 @@
> #include <media/v4l2-mem2mem.h>
>
> #include "sunxi_cedrus_common.h"
> -#include "sunxi_cedrus_mpeg2.h"
> #include "sunxi_cedrus_dec.h"
> #include "sunxi_cedrus_hw.h"
>
> @@ -414,6 +413,21 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
> return 0;
> }
>
> +static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> + struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> +
> + switch (ctx->vpu_src_fmt->fourcc) {
> + case V4L2_PIX_FMT_MPEG2_FRAME:
> + ctx->current_codec = SUNXI_CEDRUS_CODEC_MPEG2;
> + break;
> + default:
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> static void sunxi_cedrus_stop_streaming(struct vb2_queue *q)
> {
> struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> @@ -462,6 +476,7 @@ static struct vb2_ops sunxi_cedrus_qops = {
> .buf_cleanup = sunxi_cedrus_buf_cleanup,
> .buf_queue = sunxi_cedrus_buf_queue,
> .buf_request_complete = sunxi_cedrus_buf_request_complete,
> + .start_streaming = sunxi_cedrus_start_streaming,
> .stop_streaming = sunxi_cedrus_stop_streaming,
> .wait_prepare = vb2_ops_wait_prepare,
> .wait_finish = vb2_ops_wait_finish,
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180621/d52423c9/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-21 15:35 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 15:35 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 5641 bytes --]
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> The IRQ handler up until now was hardcoding the use of the MPEG engine to
> read the interrupt status, clear it and disable the interrupts.
>
> Obviously, that won't work really well with the introduction of new codecs
> that use a separate engine with a separate register set.
>
> In order to make this more future proof, introduce new decodec operations
> to deal with the interrupt management. The only one missing is the one to
> enable the interrupts in the first place, but that's taken care of by the
> trigger hook for now.
Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> 3 files changed, 53 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index c2e2c92d103b..a2a507eb9fc9 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> }
>
> +enum sunxi_cedrus_irq_status {
> + SUNXI_CEDRUS_IRQ_NONE,
> + SUNXI_CEDRUS_IRQ_ERROR,
> + SUNXI_CEDRUS_IRQ_OK,
> +};
> +
> struct sunxi_cedrus_dec_ops {
> + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> void (*setup)(struct sunxi_cedrus_ctx *ctx,
> struct sunxi_cedrus_run *run);
> void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> index bb46a01214e0..6b97cbd2834e 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> struct sunxi_cedrus_ctx *ctx;
> struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> struct vb2_v4l2_buffer *src_vb, *dst_vb;
> + enum sunxi_cedrus_irq_status status;
> unsigned long flags;
> - unsigned int value, status;
>
> spin_lock_irqsave(&dev->irq_lock, flags);
>
> - /* Disable MPEG interrupts and stop the MPEG engine */
> - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> -
> - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> - sunxi_cedrus_engine_disable(dev);
> -
> ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
> if (!ctx) {
> pr_err("Instance released before the end of transaction\n");
> spin_unlock_irqrestore(&dev->irq_lock, flags);
>
> - return IRQ_HANDLED;
> + return IRQ_NONE;
> }
>
> + status = dev->dec_ops[ctx->current_codec]->irq_status(ctx);
> + if (status == SUNXI_CEDRUS_IRQ_NONE) {
> + spin_unlock_irqrestore(&dev->irq_lock, flags);
> + return IRQ_NONE;
> + }
> +
> + dev->dec_ops[ctx->current_codec]->irq_disable(ctx);
> + dev->dec_ops[ctx->current_codec]->irq_clear(ctx);
> +
> src_vb = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> index e25075bb5779..51fa0c0f9bf2 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> @@ -52,6 +52,36 @@ static const u8 mpeg_default_non_intra_quant[64] = {
>
> #define m_niq(i) ((i << 8) | mpeg_default_non_intra_quant[i])
>
> +static enum sunxi_cedrus_irq_status
> +sunxi_cedrus_mpeg2_irq_status(struct sunxi_cedrus_ctx *ctx)
> +{
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + u32 reg = sunxi_cedrus_read(dev, VE_MPEG_STATUS) & 0x7;
> +
> + if (!reg)
> + return SUNXI_CEDRUS_IRQ_NONE;
> +
> + if (reg & (BIT(1) | BIT(2)))
> + return SUNXI_CEDRUS_IRQ_ERROR;
> +
> + return SUNXI_CEDRUS_IRQ_OK;
> +}
> +
> +static void sunxi_cedrus_mpeg2_irq_clear(struct sunxi_cedrus_ctx *ctx)
> +{
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> +
> + sunxi_cedrus_write(dev, GENMASK(2, 0), VE_MPEG_STATUS);
> +}
> +
> +static void sunxi_cedrus_mpeg2_irq_disable(struct sunxi_cedrus_ctx *ctx)
> +{
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + u32 reg = sunxi_cedrus_read(dev, VE_MPEG_CTRL) & ~BIT(3);
> +
> + sunxi_cedrus_write(dev, reg, VE_MPEG_CTRL);
> +}
> +
> static void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> struct sunxi_cedrus_run *run)
> {
> @@ -156,6 +186,9 @@ static void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
> }
>
> struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2 = {
> + .irq_clear = sunxi_cedrus_mpeg2_irq_clear,
> + .irq_disable = sunxi_cedrus_mpeg2_irq_disable,
> + .irq_status = sunxi_cedrus_mpeg2_irq_status,
> .setup = sunxi_cedrus_mpeg2_setup,
> .trigger = sunxi_cedrus_mpeg2_trigger,
> };
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
@ 2018-06-21 15:35 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 15:35 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> The IRQ handler up until now was hardcoding the use of the MPEG engine to
> read the interrupt status, clear it and disable the interrupts.
>
> Obviously, that won't work really well with the introduction of new codecs
> that use a separate engine with a separate register set.
>
> In order to make this more future proof, introduce new decodec operations
> to deal with the interrupt management. The only one missing is the one to
> enable the interrupts in the first place, but that's taken care of by the
> trigger hook for now.
Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> 3 files changed, 53 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index c2e2c92d103b..a2a507eb9fc9 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> }
>
> +enum sunxi_cedrus_irq_status {
> + SUNXI_CEDRUS_IRQ_NONE,
> + SUNXI_CEDRUS_IRQ_ERROR,
> + SUNXI_CEDRUS_IRQ_OK,
> +};
> +
> struct sunxi_cedrus_dec_ops {
> + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> void (*setup)(struct sunxi_cedrus_ctx *ctx,
> struct sunxi_cedrus_run *run);
> void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> index bb46a01214e0..6b97cbd2834e 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> struct sunxi_cedrus_ctx *ctx;
> struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> struct vb2_v4l2_buffer *src_vb, *dst_vb;
> + enum sunxi_cedrus_irq_status status;
> unsigned long flags;
> - unsigned int value, status;
>
> spin_lock_irqsave(&dev->irq_lock, flags);
>
> - /* Disable MPEG interrupts and stop the MPEG engine */
> - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> -
> - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> - sunxi_cedrus_engine_disable(dev);
> -
> ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
> if (!ctx) {
> pr_err("Instance released before the end of transaction\n");
> spin_unlock_irqrestore(&dev->irq_lock, flags);
>
> - return IRQ_HANDLED;
> + return IRQ_NONE;
> }
>
> + status = dev->dec_ops[ctx->current_codec]->irq_status(ctx);
> + if (status == SUNXI_CEDRUS_IRQ_NONE) {
> + spin_unlock_irqrestore(&dev->irq_lock, flags);
> + return IRQ_NONE;
> + }
> +
> + dev->dec_ops[ctx->current_codec]->irq_disable(ctx);
> + dev->dec_ops[ctx->current_codec]->irq_clear(ctx);
> +
> src_vb = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> index e25075bb5779..51fa0c0f9bf2 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
> @@ -52,6 +52,36 @@ static const u8 mpeg_default_non_intra_quant[64] = {
>
> #define m_niq(i) ((i << 8) | mpeg_default_non_intra_quant[i])
>
> +static enum sunxi_cedrus_irq_status
> +sunxi_cedrus_mpeg2_irq_status(struct sunxi_cedrus_ctx *ctx)
> +{
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + u32 reg = sunxi_cedrus_read(dev, VE_MPEG_STATUS) & 0x7;
> +
> + if (!reg)
> + return SUNXI_CEDRUS_IRQ_NONE;
> +
> + if (reg & (BIT(1) | BIT(2)))
> + return SUNXI_CEDRUS_IRQ_ERROR;
> +
> + return SUNXI_CEDRUS_IRQ_OK;
> +}
> +
> +static void sunxi_cedrus_mpeg2_irq_clear(struct sunxi_cedrus_ctx *ctx)
> +{
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> +
> + sunxi_cedrus_write(dev, GENMASK(2, 0), VE_MPEG_STATUS);
> +}
> +
> +static void sunxi_cedrus_mpeg2_irq_disable(struct sunxi_cedrus_ctx *ctx)
> +{
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + u32 reg = sunxi_cedrus_read(dev, VE_MPEG_CTRL) & ~BIT(3);
> +
> + sunxi_cedrus_write(dev, reg, VE_MPEG_CTRL);
> +}
> +
> static void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
> struct sunxi_cedrus_run *run)
> {
> @@ -156,6 +186,9 @@ static void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx)
> }
>
> struct sunxi_cedrus_dec_ops sunxi_cedrus_dec_ops_mpeg2 = {
> + .irq_clear = sunxi_cedrus_mpeg2_irq_clear,
> + .irq_disable = sunxi_cedrus_mpeg2_irq_disable,
> + .irq_status = sunxi_cedrus_mpeg2_irq_status,
> .setup = sunxi_cedrus_mpeg2_setup,
> .trigger = sunxi_cedrus_mpeg2_trigger,
> };
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180621/6142c67d/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 8/9] media: cedrus: Add start and stop decoder operations
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-21 15:38 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 15:38 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 3575 bytes --]
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> Some codec needs to perform some additional task when a decoding is started
> and stopped, and not only at every frame.
>
> For example, the H264 decoding support needs to allocate buffers that will
> be used in the decoding process, but do not need to change over time, or at
> each frame.
>
> In order to allow that for codecs, introduce a start and stop hook that
> will be called if present at start_streaming and stop_streaming time.
>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../platform/sunxi/cedrus/sunxi_cedrus_common.h | 2 ++
> .../platform/sunxi/cedrus/sunxi_cedrus_video.c | 14 +++++++++++++-
> 2 files changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index a2a507eb9fc9..20c78ec1f037 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -120,6 +120,8 @@ struct sunxi_cedrus_dec_ops {
> enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> void (*setup)(struct sunxi_cedrus_ctx *ctx,
> struct sunxi_cedrus_run *run);
> + int (*start)(struct sunxi_cedrus_ctx *ctx);
> + void (*stop)(struct sunxi_cedrus_ctx *ctx);
> void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> };
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> index fb7b081a5bb7..d93461178857 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> @@ -416,6 +416,8 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
> static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> {
> struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + int ret = 0;
>
> switch (ctx->vpu_src_fmt->fourcc) {
> case V4L2_PIX_FMT_MPEG2_FRAME:
> @@ -425,16 +427,26 @@ static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> return -EINVAL;
> }
>
> - return 0;
> + if (V4L2_TYPE_IS_OUTPUT(q->type) &&
I suppose this check was put in place to ensure that ->start is only
called once, but what if start_streaming is called multiple times on
output? Am I totally unsure about whether the API guarantees that we
only get one start_streaming call per buffer queue, regardless of how
many userspace issues.
If we don't have such a guarantee, we probably need an internal
mechanism to avoid having ->start called more than once.
> + dev->dec_ops[ctx->current_codec]->start)
> + ret = dev->dec_ops[ctx->current_codec]->start(ctx);
> +
> + return ret;
> }
>
> static void sunxi_cedrus_stop_streaming(struct vb2_queue *q)
> {
> struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> struct vb2_v4l2_buffer *vbuf;
> unsigned long flags;
>
> flush_scheduled_work();
> +
> + if (V4L2_TYPE_IS_OUTPUT(q->type) &&
Ditto.
Cheers,
Paul
> + dev->dec_ops[ctx->current_codec]->stop)
> + dev->dec_ops[ctx->current_codec]->stop(ctx);
> +
> for (;;) {
> spin_lock_irqsave(&ctx->dev->irq_lock, flags);
>
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 8/9] media: cedrus: Add start and stop decoder operations
@ 2018-06-21 15:38 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-21 15:38 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> Some codec needs to perform some additional task when a decoding is started
> and stopped, and not only at every frame.
>
> For example, the H264 decoding support needs to allocate buffers that will
> be used in the decoding process, but do not need to change over time, or at
> each frame.
>
> In order to allow that for codecs, introduce a start and stop hook that
> will be called if present at start_streaming and stop_streaming time.
>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../platform/sunxi/cedrus/sunxi_cedrus_common.h | 2 ++
> .../platform/sunxi/cedrus/sunxi_cedrus_video.c | 14 +++++++++++++-
> 2 files changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index a2a507eb9fc9..20c78ec1f037 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -120,6 +120,8 @@ struct sunxi_cedrus_dec_ops {
> enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> void (*setup)(struct sunxi_cedrus_ctx *ctx,
> struct sunxi_cedrus_run *run);
> + int (*start)(struct sunxi_cedrus_ctx *ctx);
> + void (*stop)(struct sunxi_cedrus_ctx *ctx);
> void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> };
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> index fb7b081a5bb7..d93461178857 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> @@ -416,6 +416,8 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
> static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> {
> struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + int ret = 0;
>
> switch (ctx->vpu_src_fmt->fourcc) {
> case V4L2_PIX_FMT_MPEG2_FRAME:
> @@ -425,16 +427,26 @@ static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> return -EINVAL;
> }
>
> - return 0;
> + if (V4L2_TYPE_IS_OUTPUT(q->type) &&
I suppose this check was put in place to ensure that ->start is only
called once, but what if start_streaming is called multiple times on
output? Am I totally unsure about whether the API guarantees that we
only get one start_streaming call per buffer queue, regardless of how
many userspace issues.
If we don't have such a guarantee, we probably need an internal
mechanism to avoid having ->start called more than once.
> + dev->dec_ops[ctx->current_codec]->start)
> + ret = dev->dec_ops[ctx->current_codec]->start(ctx);
> +
> + return ret;
> }
>
> static void sunxi_cedrus_stop_streaming(struct vb2_queue *q)
> {
> struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> struct vb2_v4l2_buffer *vbuf;
> unsigned long flags;
>
> flush_scheduled_work();
> +
> + if (V4L2_TYPE_IS_OUTPUT(q->type) &&
Ditto.
Cheers,
Paul
> + dev->dec_ops[ctx->current_codec]->stop)
> + dev->dec_ops[ctx->current_codec]->stop(ctx);
> +
> for (;;) {
> spin_lock_irqsave(&ctx->dev->irq_lock, flags);
>
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180621/ce39ae6b/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 6/9] media: cedrus: Add ops structure
2018-06-21 9:49 ` Paul Kocialkowski
@ 2018-06-25 13:29 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-25 13:29 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga,
posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 4262 bytes --]
Hi!
On Thu, Jun 21, 2018 at 11:49:54AM +0200, Paul Kocialkowski wrote:
> Hi,
>
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > In order to increase the number of codecs supported, we need to decouple
> > the MPEG2 only code that was there up until now and turn it into something
> > a bit more generic.
> >
> > Do that by introducing an intermediate ops structure that would need to be
> > filled by each supported codec. Start by implementing in that structure the
> > setup and trigger hooks that are currently the only functions being
> > implemented by codecs support.
> >
> > To do so, we need to store the current codec in use, which we do at
> > start_streaming time.
>
> With the comments below taken in account, this is:
>
> Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Thanks!
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> > .../platform/sunxi/cedrus/sunxi_cedrus.c | 2 ++
> > .../sunxi/cedrus/sunxi_cedrus_common.h | 11 +++++++
> > .../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 10 +++---
> > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 11 +++++--
> > .../sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 -------------------
> > .../sunxi/cedrus/sunxi_cedrus_video.c | 17 +++++++++-
> > 6 files changed, 42 insertions(+), 42 deletions(-)
> > delete mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
> >
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > index ccd41d9a3e41..bc80480f5dfd 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > @@ -244,6 +244,8 @@ static int sunxi_cedrus_probe(struct platform_device *pdev)
> > if (ret)
> > return ret;
> >
> > + dev->dec_ops[SUNXI_CEDRUS_CODEC_MPEG2] = &sunxi_cedrus_dec_ops_mpeg2;
> > +
> > ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > if (ret)
> > goto unreg_media;
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > index a5f83c452006..c2e2c92d103b 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > @@ -75,6 +75,7 @@ struct sunxi_cedrus_ctx {
> > struct v4l2_pix_format_mplane src_fmt;
> > struct sunxi_cedrus_fmt *vpu_dst_fmt;
> > struct v4l2_pix_format_mplane dst_fmt;
> > + enum sunxi_cedrus_codec current_codec;
>
> Nit: for consistency with the way things are named, "codec_current"
> probably makes more sense.
I'm not quite sure what you mean by consitency here. This structure
has 5 other variables with two words: vpu_src_fmt, src_fmt,
vpu_dst_fmt, dst_fmt and dst_bufs. codec_current would be going
against the consistency of that structure.
> IMO using the natural English order is fine for temporary variables, but
> less so for variables used in common parts like structures. This allows
> seeing "_" as a logical hierarchical delimiter that automatically makes
> us end up with consistent prefixes that can easily be grepped for and
> derived.
>
> But that's just my 2 cents, it's really not a big deal, especially in
> this case!
>
> > struct v4l2_ctrl_handler hdl;
> > struct v4l2_ctrl *ctrls[SUNXI_CEDRUS_CTRL_MAX];
> > @@ -107,6 +108,14 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > }
> >
> > +struct sunxi_cedrus_dec_ops {
> > + void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > + struct sunxi_cedrus_run *run);
> > + void (*trigger)(struct sunxi_cedrus_ctx *ctx);
>
> By the way, are we sure that these functions won't ever fail?
> I think this is the case for MPEG2 (there is virtually nothing to check
> for errors) but perhaps it's different for H264.
It won't fail either, and if we need to change it somewhere down the
line, it's quite easy to do.
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 6/9] media: cedrus: Add ops structure
@ 2018-06-25 13:29 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-25 13:29 UTC (permalink / raw)
To: linux-arm-kernel
Hi!
On Thu, Jun 21, 2018 at 11:49:54AM +0200, Paul Kocialkowski wrote:
> Hi,
>
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > In order to increase the number of codecs supported, we need to decouple
> > the MPEG2 only code that was there up until now and turn it into something
> > a bit more generic.
> >
> > Do that by introducing an intermediate ops structure that would need to be
> > filled by each supported codec. Start by implementing in that structure the
> > setup and trigger hooks that are currently the only functions being
> > implemented by codecs support.
> >
> > To do so, we need to store the current codec in use, which we do at
> > start_streaming time.
>
> With the comments below taken in account, this is:
>
> Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Thanks!
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> > .../platform/sunxi/cedrus/sunxi_cedrus.c | 2 ++
> > .../sunxi/cedrus/sunxi_cedrus_common.h | 11 +++++++
> > .../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 10 +++---
> > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 11 +++++--
> > .../sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 -------------------
> > .../sunxi/cedrus/sunxi_cedrus_video.c | 17 +++++++++-
> > 6 files changed, 42 insertions(+), 42 deletions(-)
> > delete mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
> >
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > index ccd41d9a3e41..bc80480f5dfd 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > @@ -244,6 +244,8 @@ static int sunxi_cedrus_probe(struct platform_device *pdev)
> > if (ret)
> > return ret;
> >
> > + dev->dec_ops[SUNXI_CEDRUS_CODEC_MPEG2] = &sunxi_cedrus_dec_ops_mpeg2;
> > +
> > ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > if (ret)
> > goto unreg_media;
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > index a5f83c452006..c2e2c92d103b 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > @@ -75,6 +75,7 @@ struct sunxi_cedrus_ctx {
> > struct v4l2_pix_format_mplane src_fmt;
> > struct sunxi_cedrus_fmt *vpu_dst_fmt;
> > struct v4l2_pix_format_mplane dst_fmt;
> > + enum sunxi_cedrus_codec current_codec;
>
> Nit: for consistency with the way things are named, "codec_current"
> probably makes more sense.
I'm not quite sure what you mean by consitency here. This structure
has 5 other variables with two words: vpu_src_fmt, src_fmt,
vpu_dst_fmt, dst_fmt and dst_bufs. codec_current would be going
against the consistency of that structure.
> IMO using the natural English order is fine for temporary variables, but
> less so for variables used in common parts like structures. This allows
> seeing "_" as a logical hierarchical delimiter that automatically makes
> us end up with consistent prefixes that can easily be grepped for and
> derived.
>
> But that's just my 2 cents, it's really not a big deal, especially in
> this case!
>
> > struct v4l2_ctrl_handler hdl;
> > struct v4l2_ctrl *ctrls[SUNXI_CEDRUS_CTRL_MAX];
> > @@ -107,6 +108,14 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > }
> >
> > +struct sunxi_cedrus_dec_ops {
> > + void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > + struct sunxi_cedrus_run *run);
> > + void (*trigger)(struct sunxi_cedrus_ctx *ctx);
>
> By the way, are we sure that these functions won't ever fail?
> I think this is the case for MPEG2 (there is virtually nothing to check
> for errors) but perhaps it's different for H264.
It won't fail either, and if we need to change it somewhere down the
line, it's quite easy to do.
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180625/e35865d9/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 8/9] media: cedrus: Add start and stop decoder operations
2018-06-21 15:38 ` Paul Kocialkowski
@ 2018-06-25 13:32 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-25 13:32 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga,
posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 3260 bytes --]
On Thu, Jun 21, 2018 at 05:38:23PM +0200, Paul Kocialkowski wrote:
> Hi,
>
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > Some codec needs to perform some additional task when a decoding is started
> > and stopped, and not only at every frame.
> >
> > For example, the H264 decoding support needs to allocate buffers that will
> > be used in the decoding process, but do not need to change over time, or at
> > each frame.
> >
> > In order to allow that for codecs, introduce a start and stop hook that
> > will be called if present at start_streaming and stop_streaming time.
> >
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> > .../platform/sunxi/cedrus/sunxi_cedrus_common.h | 2 ++
> > .../platform/sunxi/cedrus/sunxi_cedrus_video.c | 14 +++++++++++++-
> > 2 files changed, 15 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > index a2a507eb9fc9..20c78ec1f037 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > @@ -120,6 +120,8 @@ struct sunxi_cedrus_dec_ops {
> > enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > struct sunxi_cedrus_run *run);
> > + int (*start)(struct sunxi_cedrus_ctx *ctx);
> > + void (*stop)(struct sunxi_cedrus_ctx *ctx);
> > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > };
> >
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > index fb7b081a5bb7..d93461178857 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > @@ -416,6 +416,8 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
> > static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> > {
> > struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> > + struct sunxi_cedrus_dev *dev = ctx->dev;
> > + int ret = 0;
> >
> > switch (ctx->vpu_src_fmt->fourcc) {
> > case V4L2_PIX_FMT_MPEG2_FRAME:
> > @@ -425,16 +427,26 @@ static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> > return -EINVAL;
> > }
> >
> > - return 0;
> > + if (V4L2_TYPE_IS_OUTPUT(q->type) &&
>
> I suppose this check was put in place to ensure that ->start is only
> called once, but what if start_streaming is called multiple times on
> output? Am I totally unsure about whether the API guarantees that we
> only get one start_streaming call per buffer queue, regardless of how
> many userspace issues.
>
> If we don't have such a guarantee, we probably need an internal
> mechanism to avoid having ->start called more than once.
As far as I understand it, start_streaming can only be called once:
https://elixir.bootlin.com/linux/latest/source/include/media/videobuf2-core.h#L357
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 8/9] media: cedrus: Add start and stop decoder operations
@ 2018-06-25 13:32 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-25 13:32 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Jun 21, 2018 at 05:38:23PM +0200, Paul Kocialkowski wrote:
> Hi,
>
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > Some codec needs to perform some additional task when a decoding is started
> > and stopped, and not only at every frame.
> >
> > For example, the H264 decoding support needs to allocate buffers that will
> > be used in the decoding process, but do not need to change over time, or at
> > each frame.
> >
> > In order to allow that for codecs, introduce a start and stop hook that
> > will be called if present at start_streaming and stop_streaming time.
> >
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> > .../platform/sunxi/cedrus/sunxi_cedrus_common.h | 2 ++
> > .../platform/sunxi/cedrus/sunxi_cedrus_video.c | 14 +++++++++++++-
> > 2 files changed, 15 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > index a2a507eb9fc9..20c78ec1f037 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > @@ -120,6 +120,8 @@ struct sunxi_cedrus_dec_ops {
> > enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > struct sunxi_cedrus_run *run);
> > + int (*start)(struct sunxi_cedrus_ctx *ctx);
> > + void (*stop)(struct sunxi_cedrus_ctx *ctx);
> > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > };
> >
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > index fb7b081a5bb7..d93461178857 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > @@ -416,6 +416,8 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
> > static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> > {
> > struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> > + struct sunxi_cedrus_dev *dev = ctx->dev;
> > + int ret = 0;
> >
> > switch (ctx->vpu_src_fmt->fourcc) {
> > case V4L2_PIX_FMT_MPEG2_FRAME:
> > @@ -425,16 +427,26 @@ static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> > return -EINVAL;
> > }
> >
> > - return 0;
> > + if (V4L2_TYPE_IS_OUTPUT(q->type) &&
>
> I suppose this check was put in place to ensure that ->start is only
> called once, but what if start_streaming is called multiple times on
> output? Am I totally unsure about whether the API guarantees that we
> only get one start_streaming call per buffer queue, regardless of how
> many userspace issues.
>
> If we don't have such a guarantee, we probably need an internal
> mechanism to avoid having ->start called more than once.
As far as I understand it, start_streaming can only be called once:
https://elixir.bootlin.com/linux/latest/source/include/media/videobuf2-core.h#L357
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180625/0c02a316/attachment-0001.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 8/9] media: cedrus: Add start and stop decoder operations
2018-06-25 13:32 ` Maxime Ripard
@ 2018-06-25 13:42 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-25 13:42 UTC (permalink / raw)
To: Maxime Ripard
Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga,
posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 3632 bytes --]
Hi,
On Mon, 2018-06-25 at 15:32 +0200, Maxime Ripard wrote:
> On Thu, Jun 21, 2018 at 05:38:23PM +0200, Paul Kocialkowski wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > Some codec needs to perform some additional task when a decoding is started
> > > and stopped, and not only at every frame.
> > >
> > > For example, the H264 decoding support needs to allocate buffers that will
> > > be used in the decoding process, but do not need to change over time, or at
> > > each frame.
> > >
> > > In order to allow that for codecs, introduce a start and stop hook that
> > > will be called if present at start_streaming and stop_streaming time.
> > >
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > > .../platform/sunxi/cedrus/sunxi_cedrus_common.h | 2 ++
> > > .../platform/sunxi/cedrus/sunxi_cedrus_video.c | 14 +++++++++++++-
> > > 2 files changed, 15 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > index a2a507eb9fc9..20c78ec1f037 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > @@ -120,6 +120,8 @@ struct sunxi_cedrus_dec_ops {
> > > enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > > struct sunxi_cedrus_run *run);
> > > + int (*start)(struct sunxi_cedrus_ctx *ctx);
> > > + void (*stop)(struct sunxi_cedrus_ctx *ctx);
> > > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > > };
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > > index fb7b081a5bb7..d93461178857 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > > @@ -416,6 +416,8 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
> > > static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> > > {
> > > struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> > > + struct sunxi_cedrus_dev *dev = ctx->dev;
> > > + int ret = 0;
> > >
> > > switch (ctx->vpu_src_fmt->fourcc) {
> > > case V4L2_PIX_FMT_MPEG2_FRAME:
> > > @@ -425,16 +427,26 @@ static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> > > return -EINVAL;
> > > }
> > >
> > > - return 0;
> > > + if (V4L2_TYPE_IS_OUTPUT(q->type) &&
> >
> > I suppose this check was put in place to ensure that ->start is only
> > called once, but what if start_streaming is called multiple times on
> > output? Am I totally unsure about whether the API guarantees that we
> > only get one start_streaming call per buffer queue, regardless of how
> > many userspace issues.
> >
> > If we don't have such a guarantee, we probably need an internal
> > mechanism to avoid having ->start called more than once.
>
> As far as I understand it, start_streaming can only be called once:
> https://elixir.bootlin.com/linux/latest/source/include/media/videobuf2-core.h#L357
And there's definitely a check in vb2_core_streamon:
if (q->streaming) {
dprintk(3, "already streaming\n");
return 0;
}
so it looks like we're safe :)
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 8/9] media: cedrus: Add start and stop decoder operations
@ 2018-06-25 13:42 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-25 13:42 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Mon, 2018-06-25 at 15:32 +0200, Maxime Ripard wrote:
> On Thu, Jun 21, 2018 at 05:38:23PM +0200, Paul Kocialkowski wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > Some codec needs to perform some additional task when a decoding is started
> > > and stopped, and not only at every frame.
> > >
> > > For example, the H264 decoding support needs to allocate buffers that will
> > > be used in the decoding process, but do not need to change over time, or at
> > > each frame.
> > >
> > > In order to allow that for codecs, introduce a start and stop hook that
> > > will be called if present at start_streaming and stop_streaming time.
> > >
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > > .../platform/sunxi/cedrus/sunxi_cedrus_common.h | 2 ++
> > > .../platform/sunxi/cedrus/sunxi_cedrus_video.c | 14 +++++++++++++-
> > > 2 files changed, 15 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > index a2a507eb9fc9..20c78ec1f037 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > @@ -120,6 +120,8 @@ struct sunxi_cedrus_dec_ops {
> > > enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > > struct sunxi_cedrus_run *run);
> > > + int (*start)(struct sunxi_cedrus_ctx *ctx);
> > > + void (*stop)(struct sunxi_cedrus_ctx *ctx);
> > > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > > };
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > > index fb7b081a5bb7..d93461178857 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
> > > @@ -416,6 +416,8 @@ static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
> > > static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> > > {
> > > struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
> > > + struct sunxi_cedrus_dev *dev = ctx->dev;
> > > + int ret = 0;
> > >
> > > switch (ctx->vpu_src_fmt->fourcc) {
> > > case V4L2_PIX_FMT_MPEG2_FRAME:
> > > @@ -425,16 +427,26 @@ static int sunxi_cedrus_start_streaming(struct vb2_queue *q, unsigned int count)
> > > return -EINVAL;
> > > }
> > >
> > > - return 0;
> > > + if (V4L2_TYPE_IS_OUTPUT(q->type) &&
> >
> > I suppose this check was put in place to ensure that ->start is only
> > called once, but what if start_streaming is called multiple times on
> > output? Am I totally unsure about whether the API guarantees that we
> > only get one start_streaming call per buffer queue, regardless of how
> > many userspace issues.
> >
> > If we don't have such a guarantee, we probably need an internal
> > mechanism to avoid having ->start called more than once.
>
> As far as I understand it, start_streaming can only be called once:
> https://elixir.bootlin.com/linux/latest/source/include/media/videobuf2-core.h#L357
And there's definitely a check in vb2_core_streamon:
if (q->streaming) {
dprintk(3, "already streaming\n");
return 0;
}
so it looks like we're safe :)
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180625/76ab4b3a/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 6/9] media: cedrus: Add ops structure
2018-06-25 13:29 ` Maxime Ripard
@ 2018-06-25 13:48 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-25 13:48 UTC (permalink / raw)
To: Maxime Ripard
Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga,
posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 4846 bytes --]
Hi,
On Mon, 2018-06-25 at 15:29 +0200, Maxime Ripard wrote:
> Hi!
>
> On Thu, Jun 21, 2018 at 11:49:54AM +0200, Paul Kocialkowski wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > In order to increase the number of codecs supported, we need to decouple
> > > the MPEG2 only code that was there up until now and turn it into something
> > > a bit more generic.
> > >
> > > Do that by introducing an intermediate ops structure that would need to be
> > > filled by each supported codec. Start by implementing in that structure the
> > > setup and trigger hooks that are currently the only functions being
> > > implemented by codecs support.
> > >
> > > To do so, we need to store the current codec in use, which we do at
> > > start_streaming time.
> >
> > With the comments below taken in account, this is:
> >
> > Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
>
> Thanks!
>
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > > .../platform/sunxi/cedrus/sunxi_cedrus.c | 2 ++
> > > .../sunxi/cedrus/sunxi_cedrus_common.h | 11 +++++++
> > > .../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 10 +++---
> > > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 11 +++++--
> > > .../sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 -------------------
> > > .../sunxi/cedrus/sunxi_cedrus_video.c | 17 +++++++++-
> > > 6 files changed, 42 insertions(+), 42 deletions(-)
> > > delete mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > > index ccd41d9a3e41..bc80480f5dfd 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > > @@ -244,6 +244,8 @@ static int sunxi_cedrus_probe(struct platform_device *pdev)
> > > if (ret)
> > > return ret;
> > >
> > > + dev->dec_ops[SUNXI_CEDRUS_CODEC_MPEG2] = &sunxi_cedrus_dec_ops_mpeg2;
> > > +
> > > ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > > if (ret)
> > > goto unreg_media;
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > index a5f83c452006..c2e2c92d103b 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > @@ -75,6 +75,7 @@ struct sunxi_cedrus_ctx {
> > > struct v4l2_pix_format_mplane src_fmt;
> > > struct sunxi_cedrus_fmt *vpu_dst_fmt;
> > > struct v4l2_pix_format_mplane dst_fmt;
> > > + enum sunxi_cedrus_codec current_codec;
> >
> > Nit: for consistency with the way things are named, "codec_current"
> > probably makes more sense.
>
> I'm not quite sure what you mean by consitency here. This structure
> has 5 other variables with two words: vpu_src_fmt, src_fmt,
> vpu_dst_fmt, dst_fmt and dst_bufs. codec_current would be going
> against the consistency of that structure.
Mhh, not sure what I meant after all regarding consistency. I was
thinking in terms of name/qualifier, but it's clear that the structure
for the names of already-existing elements has qualifiers first and name
at the end, so "curent_codec" indeed fits best.
Sorry for the noise!
> > IMO using the natural English order is fine for temporary variables, but
> > less so for variables used in common parts like structures. This allows
> > seeing "_" as a logical hierarchical delimiter that automatically makes
> > us end up with consistent prefixes that can easily be grepped for and
> > derived.
> >
> > But that's just my 2 cents, it's really not a big deal, especially in
> > this case!
> >
> > > struct v4l2_ctrl_handler hdl;
> > > struct v4l2_ctrl *ctrls[SUNXI_CEDRUS_CTRL_MAX];
> > > @@ -107,6 +108,14 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > > }
> > >
> > > +struct sunxi_cedrus_dec_ops {
> > > + void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > > + struct sunxi_cedrus_run *run);
> > > + void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> >
> > By the way, are we sure that these functions won't ever fail?
> > I think this is the case for MPEG2 (there is virtually nothing to check
> > for errors) but perhaps it's different for H264.
>
> It won't fail either, and if we need to change it somewhere down the
> line, it's quite easy to do.
Right, let's keep it that way then.
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 6/9] media: cedrus: Add ops structure
@ 2018-06-25 13:48 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-25 13:48 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Mon, 2018-06-25 at 15:29 +0200, Maxime Ripard wrote:
> Hi!
>
> On Thu, Jun 21, 2018 at 11:49:54AM +0200, Paul Kocialkowski wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > In order to increase the number of codecs supported, we need to decouple
> > > the MPEG2 only code that was there up until now and turn it into something
> > > a bit more generic.
> > >
> > > Do that by introducing an intermediate ops structure that would need to be
> > > filled by each supported codec. Start by implementing in that structure the
> > > setup and trigger hooks that are currently the only functions being
> > > implemented by codecs support.
> > >
> > > To do so, we need to store the current codec in use, which we do at
> > > start_streaming time.
> >
> > With the comments below taken in account, this is:
> >
> > Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
>
> Thanks!
>
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > > .../platform/sunxi/cedrus/sunxi_cedrus.c | 2 ++
> > > .../sunxi/cedrus/sunxi_cedrus_common.h | 11 +++++++
> > > .../platform/sunxi/cedrus/sunxi_cedrus_dec.c | 10 +++---
> > > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 11 +++++--
> > > .../sunxi/cedrus/sunxi_cedrus_mpeg2.h | 33 -------------------
> > > .../sunxi/cedrus/sunxi_cedrus_video.c | 17 +++++++++-
> > > 6 files changed, 42 insertions(+), 42 deletions(-)
> > > delete mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > > index ccd41d9a3e41..bc80480f5dfd 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
> > > @@ -244,6 +244,8 @@ static int sunxi_cedrus_probe(struct platform_device *pdev)
> > > if (ret)
> > > return ret;
> > >
> > > + dev->dec_ops[SUNXI_CEDRUS_CODEC_MPEG2] = &sunxi_cedrus_dec_ops_mpeg2;
> > > +
> > > ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > > if (ret)
> > > goto unreg_media;
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > index a5f83c452006..c2e2c92d103b 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > @@ -75,6 +75,7 @@ struct sunxi_cedrus_ctx {
> > > struct v4l2_pix_format_mplane src_fmt;
> > > struct sunxi_cedrus_fmt *vpu_dst_fmt;
> > > struct v4l2_pix_format_mplane dst_fmt;
> > > + enum sunxi_cedrus_codec current_codec;
> >
> > Nit: for consistency with the way things are named, "codec_current"
> > probably makes more sense.
>
> I'm not quite sure what you mean by consitency here. This structure
> has 5 other variables with two words: vpu_src_fmt, src_fmt,
> vpu_dst_fmt, dst_fmt and dst_bufs. codec_current would be going
> against the consistency of that structure.
Mhh, not sure what I meant after all regarding consistency. I was
thinking in terms of name/qualifier, but it's clear that the structure
for the names of already-existing elements has qualifiers first and name
at the end, so "curent_codec" indeed fits best.
Sorry for the noise!
> > IMO using the natural English order is fine for temporary variables, but
> > less so for variables used in common parts like structures. This allows
> > seeing "_" as a logical hierarchical delimiter that automatically makes
> > us end up with consistent prefixes that can easily be grepped for and
> > derived.
> >
> > But that's just my 2 cents, it's really not a big deal, especially in
> > this case!
> >
> > > struct v4l2_ctrl_handler hdl;
> > > struct v4l2_ctrl *ctrls[SUNXI_CEDRUS_CTRL_MAX];
> > > @@ -107,6 +108,14 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > > }
> > >
> > > +struct sunxi_cedrus_dec_ops {
> > > + void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > > + struct sunxi_cedrus_run *run);
> > > + void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> >
> > By the way, are we sure that these functions won't ever fail?
> > I think this is the case for MPEG2 (there is virtually nothing to check
> > for errors) but perhaps it's different for H264.
>
> It won't fail either, and if we need to change it somewhere down the
> line, it's quite easy to do.
Right, let's keep it that way then.
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180625/d387e14f/attachment-0001.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
2018-06-21 15:35 ` Paul Kocialkowski
@ 2018-06-25 14:18 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-25 14:18 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 4485 bytes --]
Hi,
On Thu, 2018-06-21 at 17:35 +0200, Paul Kocialkowski wrote:
> Hi,
>
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > The IRQ handler up until now was hardcoding the use of the MPEG engine to
> > read the interrupt status, clear it and disable the interrupts.
> >
> > Obviously, that won't work really well with the introduction of new codecs
> > that use a separate engine with a separate register set.
> >
> > In order to make this more future proof, introduce new decodec operations
> > to deal with the interrupt management. The only one missing is the one to
> > enable the interrupts in the first place, but that's taken care of by the
> > trigger hook for now.
>
> Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Scratch that for now, I just thought of something here (see below).
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> > .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> > .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> > 3 files changed, 53 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > index c2e2c92d103b..a2a507eb9fc9 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > }
> >
> > +enum sunxi_cedrus_irq_status {
> > + SUNXI_CEDRUS_IRQ_NONE,
> > + SUNXI_CEDRUS_IRQ_ERROR,
> > + SUNXI_CEDRUS_IRQ_OK,
> > +};
> > +
> > struct sunxi_cedrus_dec_ops {
> > + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> > + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> > + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > struct sunxi_cedrus_run *run);
> > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > index bb46a01214e0..6b97cbd2834e 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> > struct sunxi_cedrus_ctx *ctx;
> > struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> > struct vb2_v4l2_buffer *src_vb, *dst_vb;
> > + enum sunxi_cedrus_irq_status status;
> > unsigned long flags;
> > - unsigned int value, status;
> >
> > spin_lock_irqsave(&dev->irq_lock, flags);
> >
> > - /* Disable MPEG interrupts and stop the MPEG engine */
> > - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> > - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> > -
> > - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> > - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> > - sunxi_cedrus_engine_disable(dev);
> > -
> > ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
> > if (!ctx) {
> > pr_err("Instance released before the end of transaction\n");
> > spin_unlock_irqrestore(&dev->irq_lock, flags);
> >
> > - return IRQ_HANDLED;
> > + return IRQ_NONE;
> > }
> >
> > + status = dev->dec_ops[ctx->current_codec]->irq_status(ctx);
> > + if (status == SUNXI_CEDRUS_IRQ_NONE) {
> > + spin_unlock_irqrestore(&dev->irq_lock, flags);
> > + return IRQ_NONE;
> > + }
> > +
> > + dev->dec_ops[ctx->current_codec]->irq_disable(ctx);
> > + dev->dec_ops[ctx->current_codec]->irq_clear(ctx);
> > +
> > src_vb = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> > dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
Later in that file, there is still some checking that the first bit of
status is set:
/* First bit of MPEG_STATUS indicates success. */
if (ctx->job_abort || !(status & 0x01))
So !(status & 0x01) must be replaced with status != CEDRUS_IRQ_OK.
It seems that was working "by accident", with CEDRUS_IRQ_OK probably
being set by the compiler to 0x03.
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
@ 2018-06-25 14:18 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-25 14:18 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Thu, 2018-06-21 at 17:35 +0200, Paul Kocialkowski wrote:
> Hi,
>
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > The IRQ handler up until now was hardcoding the use of the MPEG engine to
> > read the interrupt status, clear it and disable the interrupts.
> >
> > Obviously, that won't work really well with the introduction of new codecs
> > that use a separate engine with a separate register set.
> >
> > In order to make this more future proof, introduce new decodec operations
> > to deal with the interrupt management. The only one missing is the one to
> > enable the interrupts in the first place, but that's taken care of by the
> > trigger hook for now.
>
> Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Scratch that for now, I just thought of something here (see below).
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> > .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> > .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> > 3 files changed, 53 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > index c2e2c92d103b..a2a507eb9fc9 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > }
> >
> > +enum sunxi_cedrus_irq_status {
> > + SUNXI_CEDRUS_IRQ_NONE,
> > + SUNXI_CEDRUS_IRQ_ERROR,
> > + SUNXI_CEDRUS_IRQ_OK,
> > +};
> > +
> > struct sunxi_cedrus_dec_ops {
> > + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> > + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> > + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > struct sunxi_cedrus_run *run);
> > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > index bb46a01214e0..6b97cbd2834e 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> > struct sunxi_cedrus_ctx *ctx;
> > struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> > struct vb2_v4l2_buffer *src_vb, *dst_vb;
> > + enum sunxi_cedrus_irq_status status;
> > unsigned long flags;
> > - unsigned int value, status;
> >
> > spin_lock_irqsave(&dev->irq_lock, flags);
> >
> > - /* Disable MPEG interrupts and stop the MPEG engine */
> > - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> > - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> > -
> > - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> > - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> > - sunxi_cedrus_engine_disable(dev);
> > -
> > ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
> > if (!ctx) {
> > pr_err("Instance released before the end of transaction\n");
> > spin_unlock_irqrestore(&dev->irq_lock, flags);
> >
> > - return IRQ_HANDLED;
> > + return IRQ_NONE;
> > }
> >
> > + status = dev->dec_ops[ctx->current_codec]->irq_status(ctx);
> > + if (status == SUNXI_CEDRUS_IRQ_NONE) {
> > + spin_unlock_irqrestore(&dev->irq_lock, flags);
> > + return IRQ_NONE;
> > + }
> > +
> > + dev->dec_ops[ctx->current_codec]->irq_disable(ctx);
> > + dev->dec_ops[ctx->current_codec]->irq_clear(ctx);
> > +
> > src_vb = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> > dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
Later in that file, there is still some checking that the first bit of
status is set:
/* First bit of MPEG_STATUS indicates success. */
if (ctx->job_abort || !(status & 0x01))
So !(status & 0x01) must be replaced with status != CEDRUS_IRQ_OK.
It seems that was working "by accident", with CEDRUS_IRQ_OK probably
being set by the compiler to 0x03.
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180625/724e1d27/attachment-0001.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
2018-06-13 14:07 ` Maxime Ripard
@ 2018-06-25 15:38 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-25 15:38 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 3435 bytes --]
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> The IRQ handler up until now was hardcoding the use of the MPEG engine to
> read the interrupt status, clear it and disable the interrupts.
>
> Obviously, that won't work really well with the introduction of new codecs
> that use a separate engine with a separate register set.
>
> In order to make this more future proof, introduce new decodec operations
> to deal with the interrupt management. The only one missing is the one to
> enable the interrupts in the first place, but that's taken care of by the
> trigger hook for now.
Here's another comment about an issue I just figured out.
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> 3 files changed, 53 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index c2e2c92d103b..a2a507eb9fc9 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> }
>
> +enum sunxi_cedrus_irq_status {
> + SUNXI_CEDRUS_IRQ_NONE,
> + SUNXI_CEDRUS_IRQ_ERROR,
> + SUNXI_CEDRUS_IRQ_OK,
> +};
> +
> struct sunxi_cedrus_dec_ops {
> + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> void (*setup)(struct sunxi_cedrus_ctx *ctx,
> struct sunxi_cedrus_run *run);
> void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> index bb46a01214e0..6b97cbd2834e 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> struct sunxi_cedrus_ctx *ctx;
> struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> struct vb2_v4l2_buffer *src_vb, *dst_vb;
> + enum sunxi_cedrus_irq_status status;
> unsigned long flags;
> - unsigned int value, status;
>
> spin_lock_irqsave(&dev->irq_lock, flags);
>
> - /* Disable MPEG interrupts and stop the MPEG engine */
> - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> -
> - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> - sunxi_cedrus_engine_disable(dev);
This call was dropped from the code reorganization. What it intentional?
IMO, it should be brought back to the irq_disable ops for MPEG2 (and the
same probably applies to H264). Things still seem to work without it,
though, but there might be a difference in terms of standby VPU power
consumption.
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
@ 2018-06-25 15:38 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-25 15:38 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> The IRQ handler up until now was hardcoding the use of the MPEG engine to
> read the interrupt status, clear it and disable the interrupts.
>
> Obviously, that won't work really well with the introduction of new codecs
> that use a separate engine with a separate register set.
>
> In order to make this more future proof, introduce new decodec operations
> to deal with the interrupt management. The only one missing is the one to
> enable the interrupts in the first place, but that's taken care of by the
> trigger hook for now.
Here's another comment about an issue I just figured out.
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
> .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> 3 files changed, 53 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> index c2e2c92d103b..a2a507eb9fc9 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> }
>
> +enum sunxi_cedrus_irq_status {
> + SUNXI_CEDRUS_IRQ_NONE,
> + SUNXI_CEDRUS_IRQ_ERROR,
> + SUNXI_CEDRUS_IRQ_OK,
> +};
> +
> struct sunxi_cedrus_dec_ops {
> + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> void (*setup)(struct sunxi_cedrus_ctx *ctx,
> struct sunxi_cedrus_run *run);
> void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> index bb46a01214e0..6b97cbd2834e 100644
> --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> struct sunxi_cedrus_ctx *ctx;
> struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> struct vb2_v4l2_buffer *src_vb, *dst_vb;
> + enum sunxi_cedrus_irq_status status;
> unsigned long flags;
> - unsigned int value, status;
>
> spin_lock_irqsave(&dev->irq_lock, flags);
>
> - /* Disable MPEG interrupts and stop the MPEG engine */
> - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> -
> - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> - sunxi_cedrus_engine_disable(dev);
This call was dropped from the code reorganization. What it intentional?
IMO, it should be brought back to the irq_disable ops for MPEG2 (and the
same probably applies to H264). Things still seem to work without it,
though, but there might be a difference in terms of standby VPU power
consumption.
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180625/dea10102/attachment-0001.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
2018-06-25 15:38 ` Paul Kocialkowski
@ 2018-06-25 15:49 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-25 15:49 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 4261 bytes --]
On Mon, 2018-06-25 at 17:38 +0200, Paul Kocialkowski wrote:
> Hi,
>
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > The IRQ handler up until now was hardcoding the use of the MPEG engine to
> > read the interrupt status, clear it and disable the interrupts.
> >
> > Obviously, that won't work really well with the introduction of new codecs
> > that use a separate engine with a separate register set.
> >
> > In order to make this more future proof, introduce new decodec operations
> > to deal with the interrupt management. The only one missing is the one to
> > enable the interrupts in the first place, but that's taken care of by the
> > trigger hook for now.
>
> Here's another comment about an issue I just figured out.
>
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> > .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> > .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> > 3 files changed, 53 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > index c2e2c92d103b..a2a507eb9fc9 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > }
> >
> > +enum sunxi_cedrus_irq_status {
> > + SUNXI_CEDRUS_IRQ_NONE,
> > + SUNXI_CEDRUS_IRQ_ERROR,
> > + SUNXI_CEDRUS_IRQ_OK,
> > +};
> > +
> > struct sunxi_cedrus_dec_ops {
> > + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> > + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> > + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > struct sunxi_cedrus_run *run);
> > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > index bb46a01214e0..6b97cbd2834e 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> > struct sunxi_cedrus_ctx *ctx;
> > struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> > struct vb2_v4l2_buffer *src_vb, *dst_vb;
> > + enum sunxi_cedrus_irq_status status;
> > unsigned long flags;
> > - unsigned int value, status;
> >
> > spin_lock_irqsave(&dev->irq_lock, flags);
> >
> > - /* Disable MPEG interrupts and stop the MPEG engine */
> > - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> > - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> > -
> > - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> > - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> > - sunxi_cedrus_engine_disable(dev);
>
> This call was dropped from the code reorganization. What it intentional?
>
> IMO, it should be brought back to the irq_disable ops for MPEG2 (and the
> same probably applies to H264).
Actually, this doesn't seem like the right place to put it.
Looking at the sunxi_engine_enable function, explicitly passing the
codec as an argument and calling that function from each codec's code
feels strange.
We should probably break down these functions
(sunxi_engine_enable/sunxi_engine_disable) into codec ops. The
start/stop couple seems like a good fit, but it brings up the following
question: do we want to disable the engine between each frame or not?
I really don't know how significant the power saving benefits are and
what downsides there might be.
What do you think?
> Things still seem to work without it,
> though, but there might be a difference in terms of standby VPU power
> consumption.
>
> Cheers,
>
> Paul
>
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
@ 2018-06-25 15:49 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-06-25 15:49 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, 2018-06-25 at 17:38 +0200, Paul Kocialkowski wrote:
> Hi,
>
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > The IRQ handler up until now was hardcoding the use of the MPEG engine to
> > read the interrupt status, clear it and disable the interrupts.
> >
> > Obviously, that won't work really well with the introduction of new codecs
> > that use a separate engine with a separate register set.
> >
> > In order to make this more future proof, introduce new decodec operations
> > to deal with the interrupt management. The only one missing is the one to
> > enable the interrupts in the first place, but that's taken care of by the
> > trigger hook for now.
>
> Here's another comment about an issue I just figured out.
>
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> > .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> > .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> > 3 files changed, 53 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > index c2e2c92d103b..a2a507eb9fc9 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > }
> >
> > +enum sunxi_cedrus_irq_status {
> > + SUNXI_CEDRUS_IRQ_NONE,
> > + SUNXI_CEDRUS_IRQ_ERROR,
> > + SUNXI_CEDRUS_IRQ_OK,
> > +};
> > +
> > struct sunxi_cedrus_dec_ops {
> > + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> > + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> > + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > struct sunxi_cedrus_run *run);
> > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > index bb46a01214e0..6b97cbd2834e 100644
> > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> > struct sunxi_cedrus_ctx *ctx;
> > struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> > struct vb2_v4l2_buffer *src_vb, *dst_vb;
> > + enum sunxi_cedrus_irq_status status;
> > unsigned long flags;
> > - unsigned int value, status;
> >
> > spin_lock_irqsave(&dev->irq_lock, flags);
> >
> > - /* Disable MPEG interrupts and stop the MPEG engine */
> > - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> > - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> > -
> > - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> > - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> > - sunxi_cedrus_engine_disable(dev);
>
> This call was dropped from the code reorganization. What it intentional?
>
> IMO, it should be brought back to the irq_disable ops for MPEG2 (and the
> same probably applies to H264).
Actually, this doesn't seem like the right place to put it.
Looking at the sunxi_engine_enable function, explicitly passing the
codec as an argument and calling that function from each codec's code
feels strange.
We should probably break down these functions
(sunxi_engine_enable/sunxi_engine_disable) into codec ops. The
start/stop couple seems like a good fit, but it brings up the following
question: do we want to disable the engine between each frame or not?
I really don't know how significant the power saving benefits are and
what downsides there might be.
What do you think?
> Things still seem to work without it,
> though, but there might be a difference in terms of standby VPU power
> consumption.
>
> Cheers,
>
> Paul
>
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180625/c279bd89/attachment-0001.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
2018-06-25 14:18 ` Paul Kocialkowski
@ 2018-06-25 16:15 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-25 16:15 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga,
posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 4811 bytes --]
On Mon, Jun 25, 2018 at 04:18:51PM +0200, Paul Kocialkowski wrote:
> Hi,
>
> On Thu, 2018-06-21 at 17:35 +0200, Paul Kocialkowski wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > The IRQ handler up until now was hardcoding the use of the MPEG engine to
> > > read the interrupt status, clear it and disable the interrupts.
> > >
> > > Obviously, that won't work really well with the introduction of new codecs
> > > that use a separate engine with a separate register set.
> > >
> > > In order to make this more future proof, introduce new decodec operations
> > > to deal with the interrupt management. The only one missing is the one to
> > > enable the interrupts in the first place, but that's taken care of by the
> > > trigger hook for now.
> >
> > Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
>
> Scratch that for now, I just thought of something here (see below).
>
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > > .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> > > .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> > > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> > > 3 files changed, 53 insertions(+), 10 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > index c2e2c92d103b..a2a507eb9fc9 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > > }
> > >
> > > +enum sunxi_cedrus_irq_status {
> > > + SUNXI_CEDRUS_IRQ_NONE,
> > > + SUNXI_CEDRUS_IRQ_ERROR,
> > > + SUNXI_CEDRUS_IRQ_OK,
> > > +};
> > > +
> > > struct sunxi_cedrus_dec_ops {
> > > + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> > > + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> > > + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > > struct sunxi_cedrus_run *run);
> > > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > index bb46a01214e0..6b97cbd2834e 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> > > struct sunxi_cedrus_ctx *ctx;
> > > struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> > > struct vb2_v4l2_buffer *src_vb, *dst_vb;
> > > + enum sunxi_cedrus_irq_status status;
> > > unsigned long flags;
> > > - unsigned int value, status;
> > >
> > > spin_lock_irqsave(&dev->irq_lock, flags);
> > >
> > > - /* Disable MPEG interrupts and stop the MPEG engine */
> > > - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> > > - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> > > -
> > > - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> > > - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> > > - sunxi_cedrus_engine_disable(dev);
> > > -
> > > ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
> > > if (!ctx) {
> > > pr_err("Instance released before the end of transaction\n");
> > > spin_unlock_irqrestore(&dev->irq_lock, flags);
> > >
> > > - return IRQ_HANDLED;
> > > + return IRQ_NONE;
> > > }
> > >
> > > + status = dev->dec_ops[ctx->current_codec]->irq_status(ctx);
> > > + if (status == SUNXI_CEDRUS_IRQ_NONE) {
> > > + spin_unlock_irqrestore(&dev->irq_lock, flags);
> > > + return IRQ_NONE;
> > > + }
> > > +
> > > + dev->dec_ops[ctx->current_codec]->irq_disable(ctx);
> > > + dev->dec_ops[ctx->current_codec]->irq_clear(ctx);
> > > +
> > > src_vb = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> > > dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
>
> Later in that file, there is still some checking that the first bit of
> status is set:
>
> /* First bit of MPEG_STATUS indicates success. */
> if (ctx->job_abort || !(status & 0x01))
>
> So !(status & 0x01) must be replaced with status != CEDRUS_IRQ_OK.
>
> It seems that was working "by accident", with CEDRUS_IRQ_OK probably
> being set by the compiler to 0x03.
Yeah, I noticed this already and this was fixed for the v2 :)
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
@ 2018-06-25 16:15 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-25 16:15 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Jun 25, 2018 at 04:18:51PM +0200, Paul Kocialkowski wrote:
> Hi,
>
> On Thu, 2018-06-21 at 17:35 +0200, Paul Kocialkowski wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > The IRQ handler up until now was hardcoding the use of the MPEG engine to
> > > read the interrupt status, clear it and disable the interrupts.
> > >
> > > Obviously, that won't work really well with the introduction of new codecs
> > > that use a separate engine with a separate register set.
> > >
> > > In order to make this more future proof, introduce new decodec operations
> > > to deal with the interrupt management. The only one missing is the one to
> > > enable the interrupts in the first place, but that's taken care of by the
> > > trigger hook for now.
> >
> > Acked-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
>
> Scratch that for now, I just thought of something here (see below).
>
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > > .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> > > .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> > > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> > > 3 files changed, 53 insertions(+), 10 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > index c2e2c92d103b..a2a507eb9fc9 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > > }
> > >
> > > +enum sunxi_cedrus_irq_status {
> > > + SUNXI_CEDRUS_IRQ_NONE,
> > > + SUNXI_CEDRUS_IRQ_ERROR,
> > > + SUNXI_CEDRUS_IRQ_OK,
> > > +};
> > > +
> > > struct sunxi_cedrus_dec_ops {
> > > + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> > > + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> > > + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > > struct sunxi_cedrus_run *run);
> > > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > index bb46a01214e0..6b97cbd2834e 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> > > struct sunxi_cedrus_ctx *ctx;
> > > struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> > > struct vb2_v4l2_buffer *src_vb, *dst_vb;
> > > + enum sunxi_cedrus_irq_status status;
> > > unsigned long flags;
> > > - unsigned int value, status;
> > >
> > > spin_lock_irqsave(&dev->irq_lock, flags);
> > >
> > > - /* Disable MPEG interrupts and stop the MPEG engine */
> > > - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> > > - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> > > -
> > > - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> > > - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> > > - sunxi_cedrus_engine_disable(dev);
> > > -
> > > ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
> > > if (!ctx) {
> > > pr_err("Instance released before the end of transaction\n");
> > > spin_unlock_irqrestore(&dev->irq_lock, flags);
> > >
> > > - return IRQ_HANDLED;
> > > + return IRQ_NONE;
> > > }
> > >
> > > + status = dev->dec_ops[ctx->current_codec]->irq_status(ctx);
> > > + if (status == SUNXI_CEDRUS_IRQ_NONE) {
> > > + spin_unlock_irqrestore(&dev->irq_lock, flags);
> > > + return IRQ_NONE;
> > > + }
> > > +
> > > + dev->dec_ops[ctx->current_codec]->irq_disable(ctx);
> > > + dev->dec_ops[ctx->current_codec]->irq_clear(ctx);
> > > +
> > > src_vb = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
> > > dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
>
> Later in that file, there is still some checking that the first bit of
> status is set:
>
> /* First bit of MPEG_STATUS indicates success. */
> if (ctx->job_abort || !(status & 0x01))
>
> So !(status & 0x01) must be replaced with status != CEDRUS_IRQ_OK.
>
> It seems that was working "by accident", with CEDRUS_IRQ_OK probably
> being set by the compiler to 0x03.
Yeah, I noticed this already and this was fixed for the v2 :)
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180625/f59de79d/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
2018-06-25 15:49 ` Paul Kocialkowski
@ 2018-06-25 19:01 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-25 19:01 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga,
posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 4645 bytes --]
On Mon, Jun 25, 2018 at 05:49:58PM +0200, Paul Kocialkowski wrote:
> On Mon, 2018-06-25 at 17:38 +0200, Paul Kocialkowski wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > The IRQ handler up until now was hardcoding the use of the MPEG engine to
> > > read the interrupt status, clear it and disable the interrupts.
> > >
> > > Obviously, that won't work really well with the introduction of new codecs
> > > that use a separate engine with a separate register set.
> > >
> > > In order to make this more future proof, introduce new decodec operations
> > > to deal with the interrupt management. The only one missing is the one to
> > > enable the interrupts in the first place, but that's taken care of by the
> > > trigger hook for now.
> >
> > Here's another comment about an issue I just figured out.
> >
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > > .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> > > .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> > > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> > > 3 files changed, 53 insertions(+), 10 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > index c2e2c92d103b..a2a507eb9fc9 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > > }
> > >
> > > +enum sunxi_cedrus_irq_status {
> > > + SUNXI_CEDRUS_IRQ_NONE,
> > > + SUNXI_CEDRUS_IRQ_ERROR,
> > > + SUNXI_CEDRUS_IRQ_OK,
> > > +};
> > > +
> > > struct sunxi_cedrus_dec_ops {
> > > + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> > > + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> > > + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > > struct sunxi_cedrus_run *run);
> > > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > index bb46a01214e0..6b97cbd2834e 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> > > struct sunxi_cedrus_ctx *ctx;
> > > struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> > > struct vb2_v4l2_buffer *src_vb, *dst_vb;
> > > + enum sunxi_cedrus_irq_status status;
> > > unsigned long flags;
> > > - unsigned int value, status;
> > >
> > > spin_lock_irqsave(&dev->irq_lock, flags);
> > >
> > > - /* Disable MPEG interrupts and stop the MPEG engine */
> > > - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> > > - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> > > -
> > > - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> > > - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> > > - sunxi_cedrus_engine_disable(dev);
> >
> > This call was dropped from the code reorganization. What it intentional?
> >
> > IMO, it should be brought back to the irq_disable ops for MPEG2 (and the
> > same probably applies to H264).
>
> Actually, this doesn't seem like the right place to put it.
>
> Looking at the sunxi_engine_enable function, explicitly passing the
> codec as an argument and calling that function from each codec's code
> feels strange.
>
> We should probably break down these functions
> (sunxi_engine_enable/sunxi_engine_disable) into codec ops. The
> start/stop couple seems like a good fit, but it brings up the following
> question: do we want to disable the engine between each frame or not?
>
> I really don't know how significant the power saving benefits are and
> what downsides there might be.
>
> What do you think?
If we don't need to disable it between each frames, then let's keep it
enabled.
Without any number, and considering the current state of power
management (which is inexistent on this driver, and close to for the
whole platform), wasting a bit of power if we do isn't really a big
deal.
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
@ 2018-06-25 19:01 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-25 19:01 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Jun 25, 2018 at 05:49:58PM +0200, Paul Kocialkowski wrote:
> On Mon, 2018-06-25 at 17:38 +0200, Paul Kocialkowski wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > The IRQ handler up until now was hardcoding the use of the MPEG engine to
> > > read the interrupt status, clear it and disable the interrupts.
> > >
> > > Obviously, that won't work really well with the introduction of new codecs
> > > that use a separate engine with a separate register set.
> > >
> > > In order to make this more future proof, introduce new decodec operations
> > > to deal with the interrupt management. The only one missing is the one to
> > > enable the interrupts in the first place, but that's taken care of by the
> > > trigger hook for now.
> >
> > Here's another comment about an issue I just figured out.
> >
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > > .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> > > .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> > > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> > > 3 files changed, 53 insertions(+), 10 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > index c2e2c92d103b..a2a507eb9fc9 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > > }
> > >
> > > +enum sunxi_cedrus_irq_status {
> > > + SUNXI_CEDRUS_IRQ_NONE,
> > > + SUNXI_CEDRUS_IRQ_ERROR,
> > > + SUNXI_CEDRUS_IRQ_OK,
> > > +};
> > > +
> > > struct sunxi_cedrus_dec_ops {
> > > + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> > > + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> > > + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > > struct sunxi_cedrus_run *run);
> > > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > index bb46a01214e0..6b97cbd2834e 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> > > struct sunxi_cedrus_ctx *ctx;
> > > struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> > > struct vb2_v4l2_buffer *src_vb, *dst_vb;
> > > + enum sunxi_cedrus_irq_status status;
> > > unsigned long flags;
> > > - unsigned int value, status;
> > >
> > > spin_lock_irqsave(&dev->irq_lock, flags);
> > >
> > > - /* Disable MPEG interrupts and stop the MPEG engine */
> > > - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> > > - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> > > -
> > > - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> > > - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> > > - sunxi_cedrus_engine_disable(dev);
> >
> > This call was dropped from the code reorganization. What it intentional?
> >
> > IMO, it should be brought back to the irq_disable ops for MPEG2 (and the
> > same probably applies to H264).
>
> Actually, this doesn't seem like the right place to put it.
>
> Looking at the sunxi_engine_enable function, explicitly passing the
> codec as an argument and calling that function from each codec's code
> feels strange.
>
> We should probably break down these functions
> (sunxi_engine_enable/sunxi_engine_disable) into codec ops. The
> start/stop couple seems like a good fit, but it brings up the following
> question: do we want to disable the engine between each frame or not?
>
> I really don't know how significant the power saving benefits are and
> what downsides there might be.
>
> What do you think?
If we don't need to disable it between each frames, then let's keep it
enabled.
Without any number, and considering the current state of power
management (which is inexistent on this driver, and close to for the
whole platform), wasting a bit of power if we do isn't really a big
deal.
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180625/ede6afda/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
2018-06-25 15:49 ` Paul Kocialkowski
@ 2018-06-27 17:58 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-27 17:58 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga,
posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 4233 bytes --]
On Mon, Jun 25, 2018 at 05:49:58PM +0200, Paul Kocialkowski wrote:
> On Mon, 2018-06-25 at 17:38 +0200, Paul Kocialkowski wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > The IRQ handler up until now was hardcoding the use of the MPEG engine to
> > > read the interrupt status, clear it and disable the interrupts.
> > >
> > > Obviously, that won't work really well with the introduction of new codecs
> > > that use a separate engine with a separate register set.
> > >
> > > In order to make this more future proof, introduce new decodec operations
> > > to deal with the interrupt management. The only one missing is the one to
> > > enable the interrupts in the first place, but that's taken care of by the
> > > trigger hook for now.
> >
> > Here's another comment about an issue I just figured out.
> >
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > > .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> > > .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> > > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> > > 3 files changed, 53 insertions(+), 10 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > index c2e2c92d103b..a2a507eb9fc9 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > > }
> > >
> > > +enum sunxi_cedrus_irq_status {
> > > + SUNXI_CEDRUS_IRQ_NONE,
> > > + SUNXI_CEDRUS_IRQ_ERROR,
> > > + SUNXI_CEDRUS_IRQ_OK,
> > > +};
> > > +
> > > struct sunxi_cedrus_dec_ops {
> > > + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> > > + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> > > + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > > struct sunxi_cedrus_run *run);
> > > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > index bb46a01214e0..6b97cbd2834e 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> > > struct sunxi_cedrus_ctx *ctx;
> > > struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> > > struct vb2_v4l2_buffer *src_vb, *dst_vb;
> > > + enum sunxi_cedrus_irq_status status;
> > > unsigned long flags;
> > > - unsigned int value, status;
> > >
> > > spin_lock_irqsave(&dev->irq_lock, flags);
> > >
> > > - /* Disable MPEG interrupts and stop the MPEG engine */
> > > - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> > > - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> > > -
> > > - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> > > - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> > > - sunxi_cedrus_engine_disable(dev);
> >
> > This call was dropped from the code reorganization. What it intentional?
> >
> > IMO, it should be brought back to the irq_disable ops for MPEG2 (and the
> > same probably applies to H264).
>
> Actually, this doesn't seem like the right place to put it.
>
> Looking at the sunxi_engine_enable function, explicitly passing the
> codec as an argument and calling that function from each codec's code
> feels strange.
This isn't what this code is doing. You have three engines, each of
them implementing a different set of codec. The JPEG engine implements
MPEG1, MPEG2 and MPEG4, the H264 engine H264 and VP8, and the H265
H265 and VP9.
So there is no direct match between a codec and an engine.
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops
@ 2018-06-27 17:58 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-06-27 17:58 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Jun 25, 2018 at 05:49:58PM +0200, Paul Kocialkowski wrote:
> On Mon, 2018-06-25 at 17:38 +0200, Paul Kocialkowski wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > The IRQ handler up until now was hardcoding the use of the MPEG engine to
> > > read the interrupt status, clear it and disable the interrupts.
> > >
> > > Obviously, that won't work really well with the introduction of new codecs
> > > that use a separate engine with a separate register set.
> > >
> > > In order to make this more future proof, introduce new decodec operations
> > > to deal with the interrupt management. The only one missing is the one to
> > > enable the interrupts in the first place, but that's taken care of by the
> > > trigger hook for now.
> >
> > Here's another comment about an issue I just figured out.
> >
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > > .../sunxi/cedrus/sunxi_cedrus_common.h | 9 +++++
> > > .../platform/sunxi/cedrus/sunxi_cedrus_hw.c | 21 ++++++------
> > > .../sunxi/cedrus/sunxi_cedrus_mpeg2.c | 33 +++++++++++++++++++
> > > 3 files changed, 53 insertions(+), 10 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > index c2e2c92d103b..a2a507eb9fc9 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
> > > @@ -108,7 +108,16 @@ struct sunxi_cedrus_buffer *vb2_to_cedrus_buffer(const struct vb2_buffer *p)
> > > return vb2_v4l2_to_cedrus_buffer(to_vb2_v4l2_buffer(p));
> > > }
> > >
> > > +enum sunxi_cedrus_irq_status {
> > > + SUNXI_CEDRUS_IRQ_NONE,
> > > + SUNXI_CEDRUS_IRQ_ERROR,
> > > + SUNXI_CEDRUS_IRQ_OK,
> > > +};
> > > +
> > > struct sunxi_cedrus_dec_ops {
> > > + void (*irq_clear)(struct sunxi_cedrus_ctx *ctx);
> > > + void (*irq_disable)(struct sunxi_cedrus_ctx *ctx);
> > > + enum sunxi_cedrus_irq_status (*irq_status)(struct sunxi_cedrus_ctx *ctx);
> > > void (*setup)(struct sunxi_cedrus_ctx *ctx,
> > > struct sunxi_cedrus_run *run);
> > > void (*trigger)(struct sunxi_cedrus_ctx *ctx);
> > > diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > index bb46a01214e0..6b97cbd2834e 100644
> > > --- a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > +++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
> > > @@ -77,27 +77,28 @@ static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
> > > struct sunxi_cedrus_ctx *ctx;
> > > struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
> > > struct vb2_v4l2_buffer *src_vb, *dst_vb;
> > > + enum sunxi_cedrus_irq_status status;
> > > unsigned long flags;
> > > - unsigned int value, status;
> > >
> > > spin_lock_irqsave(&dev->irq_lock, flags);
> > >
> > > - /* Disable MPEG interrupts and stop the MPEG engine */
> > > - value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
> > > - sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
> > > -
> > > - status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
> > > - sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
> > > - sunxi_cedrus_engine_disable(dev);
> >
> > This call was dropped from the code reorganization. What it intentional?
> >
> > IMO, it should be brought back to the irq_disable ops for MPEG2 (and the
> > same probably applies to H264).
>
> Actually, this doesn't seem like the right place to put it.
>
> Looking at the sunxi_engine_enable function, explicitly passing the
> codec as an argument and calling that function from each codec's code
> feels strange.
This isn't what this code is doing. You have three engines, each of
them implementing a different set of codec. The JPEG engine implements
MPEG1, MPEG2 and MPEG4, the H264 engine H264 and VP8, and the H265
H265 and VP9.
So there is no direct match between a codec and an engine.
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180627/d175bf3b/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-06-15 11:59 ` Hans Verkuil
@ 2018-07-12 16:38 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-07-12 16:38 UTC (permalink / raw)
To: Hans Verkuil
Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga,
posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Guenter Roeck
Hi Hans,
Thanks for your feedback, I have a few things I'm not really sure
about though.
On Fri, Jun 15, 2018 at 01:59:58PM +0200, Hans Verkuil wrote:
> > +struct v4l2_ctrl_h264_sps {
> > + __u8 profile_idc;
> > + __u8 constraint_set_flags;
> > + __u8 level_idc;
> > + __u8 seq_parameter_set_id;
> > + __u8 chroma_format_idc;
> > + __u8 bit_depth_luma_minus8;
> > + __u8 bit_depth_chroma_minus8;
> > + __u8 log2_max_frame_num_minus4;
> > + __u8 pic_order_cnt_type;
> > + __u8 log2_max_pic_order_cnt_lsb_minus4;
>
> There is a hole in the struct here. Is that OK? Are there alignment
> requirements?
This structure represents an equivalent structure in the H264
bitstream, but it's not a 1:1 mapping, so I don't think there's any
alignment issues.
As of the padding, is it an issue? Isn't it defined by the ABI, and
therefore the userspace will have the same padding rules?
>
> > + __s32 offset_for_non_ref_pic;
> > + __s32 offset_for_top_to_bottom_field;
> > + __u8 num_ref_frames_in_pic_order_cnt_cycle;
> > + __s32 offset_for_ref_frame[255];
>
> Perhaps use a define instead of hardcoding 255? Not sure if that makes sense.
> Same for other arrays below.
>
> > + __u8 max_num_ref_frames;
> > + __u16 pic_width_in_mbs_minus1;
> > + __u16 pic_height_in_map_units_minus1;
> > + __u8 flags;
> > +};
>
> You have to test the struct layout for 32 bit and 64 bit systems
> (the latter for both 64 bit arm and Intel). The layout should be the
> same for all of them since the control framework does not support
> compat32 conversions for compound controls.
I'm not really sure how to test that though? Should I write a program
doing a bunch of offset_of calls to make sure it matches by hand, or
is there anything smarter?
Thanks!
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-07-12 16:38 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-07-12 16:38 UTC (permalink / raw)
To: linux-arm-kernel
Hi Hans,
Thanks for your feedback, I have a few things I'm not really sure
about though.
On Fri, Jun 15, 2018 at 01:59:58PM +0200, Hans Verkuil wrote:
> > +struct v4l2_ctrl_h264_sps {
> > + __u8 profile_idc;
> > + __u8 constraint_set_flags;
> > + __u8 level_idc;
> > + __u8 seq_parameter_set_id;
> > + __u8 chroma_format_idc;
> > + __u8 bit_depth_luma_minus8;
> > + __u8 bit_depth_chroma_minus8;
> > + __u8 log2_max_frame_num_minus4;
> > + __u8 pic_order_cnt_type;
> > + __u8 log2_max_pic_order_cnt_lsb_minus4;
>
> There is a hole in the struct here. Is that OK? Are there alignment
> requirements?
This structure represents an equivalent structure in the H264
bitstream, but it's not a 1:1 mapping, so I don't think there's any
alignment issues.
As of the padding, is it an issue? Isn't it defined by the ABI, and
therefore the userspace will have the same padding rules?
>
> > + __s32 offset_for_non_ref_pic;
> > + __s32 offset_for_top_to_bottom_field;
> > + __u8 num_ref_frames_in_pic_order_cnt_cycle;
> > + __s32 offset_for_ref_frame[255];
>
> Perhaps use a define instead of hardcoding 255? Not sure if that makes sense.
> Same for other arrays below.
>
> > + __u8 max_num_ref_frames;
> > + __u16 pic_width_in_mbs_minus1;
> > + __u16 pic_height_in_map_units_minus1;
> > + __u8 flags;
> > +};
>
> You have to test the struct layout for 32 bit and 64 bit systems
> (the latter for both 64 bit arm and Intel). The layout should be the
> same for all of them since the control framework does not support
> compat32 conversions for compound controls.
I'm not really sure how to test that though? Should I write a program
doing a bunch of offset_of calls to make sure it matches by hand, or
is there anything smarter?
Thanks!
Maxime
--
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-07-12 16:38 ` Maxime Ripard
@ 2018-07-12 16:47 ` Andrew Lunn
-1 siblings, 0 replies; 102+ messages in thread
From: Andrew Lunn @ 2018-07-12 16:47 UTC (permalink / raw)
To: Maxime Ripard
Cc: Hans Verkuil, linux-arm-kernel, Thomas Petazzoni, acourbot,
jenskuske, linux-sunxi, linux-kernel, tfiga, Paul Kocialkowski,
Chen-Yu Tsai, hans.verkuil, Laurent Pinchart, sakari.ailus,
Guenter Roeck, nicolas.dufresne, posciak, linux-media
On Thu, Jul 12, 2018 at 06:38:21PM +0200, Maxime Ripard wrote:
> Hi Hans,
>
> Thanks for your feedback, I have a few things I'm not really sure
> about though.
>
> On Fri, Jun 15, 2018 at 01:59:58PM +0200, Hans Verkuil wrote:
> > > +struct v4l2_ctrl_h264_sps {
> > > + __u8 profile_idc;
> > > + __u8 constraint_set_flags;
> > > + __u8 level_idc;
> > > + __u8 seq_parameter_set_id;
> > > + __u8 chroma_format_idc;
> > > + __u8 bit_depth_luma_minus8;
> > > + __u8 bit_depth_chroma_minus8;
> > > + __u8 log2_max_frame_num_minus4;
> > > + __u8 pic_order_cnt_type;
> > > + __u8 log2_max_pic_order_cnt_lsb_minus4;
> >
> > There is a hole in the struct here. Is that OK? Are there alignment
> > requirements?
>
> This structure represents an equivalent structure in the H264
> bitstream, but it's not a 1:1 mapping, so I don't think there's any
> alignment issues.
>
> As of the padding, is it an issue? Isn't it defined by the ABI, and
> therefore the userspace will have the same padding rules?
Hi Maxime
It gets interesting when you have a 64 bit kernel and a 32 bit userspace.
Andrew
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-07-12 16:47 ` Andrew Lunn
0 siblings, 0 replies; 102+ messages in thread
From: Andrew Lunn @ 2018-07-12 16:47 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Jul 12, 2018 at 06:38:21PM +0200, Maxime Ripard wrote:
> Hi Hans,
>
> Thanks for your feedback, I have a few things I'm not really sure
> about though.
>
> On Fri, Jun 15, 2018 at 01:59:58PM +0200, Hans Verkuil wrote:
> > > +struct v4l2_ctrl_h264_sps {
> > > + __u8 profile_idc;
> > > + __u8 constraint_set_flags;
> > > + __u8 level_idc;
> > > + __u8 seq_parameter_set_id;
> > > + __u8 chroma_format_idc;
> > > + __u8 bit_depth_luma_minus8;
> > > + __u8 bit_depth_chroma_minus8;
> > > + __u8 log2_max_frame_num_minus4;
> > > + __u8 pic_order_cnt_type;
> > > + __u8 log2_max_pic_order_cnt_lsb_minus4;
> >
> > There is a hole in the struct here. Is that OK? Are there alignment
> > requirements?
>
> This structure represents an equivalent structure in the H264
> bitstream, but it's not a 1:1 mapping, so I don't think there's any
> alignment issues.
>
> As of the padding, is it an issue? Isn't it defined by the ABI, and
> therefore the userspace will have the same padding rules?
Hi Maxime
It gets interesting when you have a 64 bit kernel and a 32 bit userspace.
Andrew
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 9/9] media: cedrus: Add H264 decoding support
2018-06-13 14:07 ` Maxime Ripard
@ 2018-07-27 13:56 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-07-27 13:56 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 1903 bytes --]
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> Introduce some basic H264 decoding support in cedrus. So far, only the
> baseline profile videos have been tested, and some more advanced features
> used in higher profiles are not even implemented.
Here are two specific comments about things I noticed when going through
the h264 code.
[...]
> @@ -88,12 +101,37 @@ struct sunxi_cedrus_ctx {
> struct work_struct run_work;
> struct list_head src_list;
> struct list_head dst_list;
> +
> + union {
> + struct {
> + void *mv_col_buf;
> + dma_addr_t mv_col_buf_dma;
> + ssize_t mv_col_buf_size;
> + void *neighbor_info_buf;
> + dma_addr_t neighbor_info_buf_dma;
Should be "neighbour" instead of "neighbor" and the same applies to most
variables related to this, as well as the register description.
[...]
> +static int sunxi_cedrus_h264_start(struct sunxi_cedrus_ctx *ctx)
> +{
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + int ret;
> +
> + ctx->codec.h264.pic_info_buf =
> + dma_alloc_coherent(dev->dev, SUNXI_CEDRUS_PIC_INFO_BUF_SIZE,
> + &ctx->codec.h264.pic_info_buf_dma,
> + GFP_KERNEL);
> + if (!ctx->codec.h264.pic_info_buf)
> + return -ENOMEM;
> +
> + ctx->codec.h264.neighbor_info_buf =
> + dma_alloc_coherent(dev->dev, SUNXI_CEDRUS_NEIGHBOR_INFO_BUF_SIZE,
> + &ctx->codec.h264.neighbor_info_buf_dma,
> + GFP_KERNEL);
> + if (!ctx->codec.h264.neighbor_info_buf) {
> + ret = -ENOMEM;
> + goto err_pic_buf;
> + }
Although this buffer is allocated here, the resulting address is
apparently never written to the appropriate VPU register (0x54).
Perhaps a write to the aforementioned register was lost along the
development process?
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 9/9] media: cedrus: Add H264 decoding support
@ 2018-07-27 13:56 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-07-27 13:56 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> Introduce some basic H264 decoding support in cedrus. So far, only the
> baseline profile videos have been tested, and some more advanced features
> used in higher profiles are not even implemented.
Here are two specific comments about things I noticed when going through
the h264 code.
[...]
> @@ -88,12 +101,37 @@ struct sunxi_cedrus_ctx {
> struct work_struct run_work;
> struct list_head src_list;
> struct list_head dst_list;
> +
> + union {
> + struct {
> + void *mv_col_buf;
> + dma_addr_t mv_col_buf_dma;
> + ssize_t mv_col_buf_size;
> + void *neighbor_info_buf;
> + dma_addr_t neighbor_info_buf_dma;
Should be "neighbour" instead of "neighbor" and the same applies to most
variables related to this, as well as the register description.
[...]
> +static int sunxi_cedrus_h264_start(struct sunxi_cedrus_ctx *ctx)
> +{
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + int ret;
> +
> + ctx->codec.h264.pic_info_buf =
> + dma_alloc_coherent(dev->dev, SUNXI_CEDRUS_PIC_INFO_BUF_SIZE,
> + &ctx->codec.h264.pic_info_buf_dma,
> + GFP_KERNEL);
> + if (!ctx->codec.h264.pic_info_buf)
> + return -ENOMEM;
> +
> + ctx->codec.h264.neighbor_info_buf =
> + dma_alloc_coherent(dev->dev, SUNXI_CEDRUS_NEIGHBOR_INFO_BUF_SIZE,
> + &ctx->codec.h264.neighbor_info_buf_dma,
> + GFP_KERNEL);
> + if (!ctx->codec.h264.neighbor_info_buf) {
> + ret = -ENOMEM;
> + goto err_pic_buf;
> + }
Although this buffer is allocated here, the resulting address is
apparently never written to the appropriate VPU register (0x54).
Perhaps a write to the aforementioned register was lost along the
development process?
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180727/e39f3bdc/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 9/9] media: cedrus: Add H264 decoding support
2018-07-27 13:56 ` Paul Kocialkowski
@ 2018-07-27 14:01 ` Chen-Yu Tsai
-1 siblings, 0 replies; 102+ messages in thread
From: Chen-Yu Tsai @ 2018-07-27 14:01 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: Maxime Ripard, Hans Verkuil, Alexandre Courbot, Sakari Ailus,
Laurent Pinchart, Tomasz Figa, Pawel Osciak, linux-kernel,
linux-arm-kernel, Linux Media Mailing List, nicolas.dufresne,
Jens Kuske, linux-sunxi, Thomas Petazzoni
On Fri, Jul 27, 2018 at 9:56 PM, Paul Kocialkowski
<paul.kocialkowski@bootlin.com> wrote:
> Hi,
>
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
>> Introduce some basic H264 decoding support in cedrus. So far, only the
>> baseline profile videos have been tested, and some more advanced features
>> used in higher profiles are not even implemented.
>
> Here are two specific comments about things I noticed when going through
> the h264 code.
>
> [...]
>
>> @@ -88,12 +101,37 @@ struct sunxi_cedrus_ctx {
>> struct work_struct run_work;
>> struct list_head src_list;
>> struct list_head dst_list;
>> +
>> + union {
>> + struct {
>> + void *mv_col_buf;
>> + dma_addr_t mv_col_buf_dma;
>> + ssize_t mv_col_buf_size;
>> + void *neighbor_info_buf;
>> + dma_addr_t neighbor_info_buf_dma;
>
> Should be "neighbour" instead of "neighbor" and the same applies to most
> variables related to this, as well as the register description.
This just means you've been hanging out with people who use American
English. :)
ChenYu
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 9/9] media: cedrus: Add H264 decoding support
@ 2018-07-27 14:01 ` Chen-Yu Tsai
0 siblings, 0 replies; 102+ messages in thread
From: Chen-Yu Tsai @ 2018-07-27 14:01 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Jul 27, 2018 at 9:56 PM, Paul Kocialkowski
<paul.kocialkowski@bootlin.com> wrote:
> Hi,
>
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
>> Introduce some basic H264 decoding support in cedrus. So far, only the
>> baseline profile videos have been tested, and some more advanced features
>> used in higher profiles are not even implemented.
>
> Here are two specific comments about things I noticed when going through
> the h264 code.
>
> [...]
>
>> @@ -88,12 +101,37 @@ struct sunxi_cedrus_ctx {
>> struct work_struct run_work;
>> struct list_head src_list;
>> struct list_head dst_list;
>> +
>> + union {
>> + struct {
>> + void *mv_col_buf;
>> + dma_addr_t mv_col_buf_dma;
>> + ssize_t mv_col_buf_size;
>> + void *neighbor_info_buf;
>> + dma_addr_t neighbor_info_buf_dma;
>
> Should be "neighbour" instead of "neighbor" and the same applies to most
> variables related to this, as well as the register description.
This just means you've been hanging out with people who use American
English. :)
ChenYu
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 9/9] media: cedrus: Add H264 decoding support
2018-07-27 14:01 ` Chen-Yu Tsai
@ 2018-07-27 14:03 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-07-27 14:03 UTC (permalink / raw)
To: Chen-Yu Tsai
Cc: Maxime Ripard, Hans Verkuil, Alexandre Courbot, Sakari Ailus,
Laurent Pinchart, Tomasz Figa, Pawel Osciak, linux-kernel,
linux-arm-kernel, Linux Media Mailing List, nicolas.dufresne,
Jens Kuske, linux-sunxi, Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 1612 bytes --]
Hi Chen-Yu,
On Fri, 2018-07-27 at 22:01 +0800, Chen-Yu Tsai wrote:
> On Fri, Jul 27, 2018 at 9:56 PM, Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > Introduce some basic H264 decoding support in cedrus. So far, only the
> > > baseline profile videos have been tested, and some more advanced features
> > > used in higher profiles are not even implemented.
> >
> > Here are two specific comments about things I noticed when going through
> > the h264 code.
> >
> > [...]
> >
> > > @@ -88,12 +101,37 @@ struct sunxi_cedrus_ctx {
> > > struct work_struct run_work;
> > > struct list_head src_list;
> > > struct list_head dst_list;
> > > +
> > > + union {
> > > + struct {
> > > + void *mv_col_buf;
> > > + dma_addr_t mv_col_buf_dma;
> > > + ssize_t mv_col_buf_size;
> > > + void *neighbor_info_buf;
> > > + dma_addr_t neighbor_info_buf_dma;
> >
> > Should be "neighbour" instead of "neighbor" and the same applies to most
> > variables related to this, as well as the register description.
>
> This just means you've been hanging out with people who use American
> English. :)
Oh wow, I honestly thought this was a typo. My mistake then!
Thanks for the heads-up!
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 9/9] media: cedrus: Add H264 decoding support
@ 2018-07-27 14:03 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-07-27 14:03 UTC (permalink / raw)
To: linux-arm-kernel
Hi Chen-Yu,
On Fri, 2018-07-27 at 22:01 +0800, Chen-Yu Tsai wrote:
> On Fri, Jul 27, 2018 at 9:56 PM, Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> > Hi,
> >
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > Introduce some basic H264 decoding support in cedrus. So far, only the
> > > baseline profile videos have been tested, and some more advanced features
> > > used in higher profiles are not even implemented.
> >
> > Here are two specific comments about things I noticed when going through
> > the h264 code.
> >
> > [...]
> >
> > > @@ -88,12 +101,37 @@ struct sunxi_cedrus_ctx {
> > > struct work_struct run_work;
> > > struct list_head src_list;
> > > struct list_head dst_list;
> > > +
> > > + union {
> > > + struct {
> > > + void *mv_col_buf;
> > > + dma_addr_t mv_col_buf_dma;
> > > + ssize_t mv_col_buf_size;
> > > + void *neighbor_info_buf;
> > > + dma_addr_t neighbor_info_buf_dma;
> >
> > Should be "neighbour" instead of "neighbor" and the same applies to most
> > variables related to this, as well as the register description.
>
> This just means you've been hanging out with people who use American
> English. :)
Oh wow, I honestly thought this was a typo. My mistake then!
Thanks for the heads-up!
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180727/88ea9303/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 9/9] media: cedrus: Add H264 decoding support
2018-06-13 14:07 ` Maxime Ripard
@ 2018-07-30 12:54 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-07-30 12:54 UTC (permalink / raw)
To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Chen-Yu Tsai, linux-kernel, linux-arm-kernel,
linux-media, nicolas.dufresne, jenskuske, linux-sunxi,
Thomas Petazzoni
[-- Attachment #1: Type: text/plain, Size: 4980 bytes --]
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> Introduce some basic H264 decoding support in cedrus. So far, only the
> baseline profile videos have been tested, and some more advanced features
> used in higher profiles are not even implemented.
While working on H265 support, I noticed a few things that should apply
to H264 as well.
[...]
> +struct sunxi_cedrus_h264_sram_ref_pic {
> + __le32 top_field_order_cnt;
> + __le32 bottom_field_order_cnt;
> + __le32 frame_info;
> + __le32 luma_ptr;
> + __le32 chroma_ptr;
> + __le32 extra_data_ptr;
> + __le32 extra_data_end;
These two previous fields represent the top and bottom (field) motion
vector column buffer addresses, so the second field is not the end of
the first one. These fields should be frame-specific and they are called
topmv_coladdr, botmv_coladdr by Allwinner.
> + __le32 reserved;
> +} __packed;
> +
[...]
> +static void sunxi_cedrus_fill_ref_pic(struct sunxi_cedrus_h264_sram_ref_pic *pic,
> + struct vb2_buffer *buf,
> + dma_addr_t extra_buf,
> + size_t extra_buf_len,
> + unsigned int top_field_order_cnt,
> + unsigned int bottom_field_order_cnt,
> + enum sunxi_cedrus_h264_pic_type pic_type)
> +{
> + pic->top_field_order_cnt = top_field_order_cnt;
> + pic->bottom_field_order_cnt = bottom_field_order_cnt;
> + pic->frame_info = pic_type << 8;
> + pic->luma_ptr = vb2_dma_contig_plane_dma_addr(buf, 0) - PHYS_OFFSET;
> + pic->chroma_ptr = vb2_dma_contig_plane_dma_addr(buf, 1) - PHYS_OFFSET;
> + pic->extra_data_ptr = extra_buf - PHYS_OFFSET;
> + pic->extra_data_end = (extra_buf - PHYS_OFFSET) + extra_buf_len;
> +}
> +
> +static void sunxi_cedrus_write_frame_list(struct sunxi_cedrus_ctx *ctx,
> + struct sunxi_cedrus_run *run)
> +{
> + struct sunxi_cedrus_h264_sram_ref_pic pic_list[SUNXI_CEDRUS_H264_FRAME_NUM];
> + const struct v4l2_ctrl_h264_decode_param *dec_param = run->h264.decode_param;
> + const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
> + const struct v4l2_ctrl_h264_sps *sps = run->h264.sps;
> + struct sunxi_cedrus_buffer *output_buf;
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + unsigned long used_dpbs = 0;
> + unsigned int position;
> + unsigned int output = 0;
> + unsigned int i;
> +
> + memset(pic_list, 0, sizeof(pic_list));
> +
> + for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) {
> + const struct v4l2_h264_dpb_entry *dpb = &dec_param->dpb[i];
> + const struct sunxi_cedrus_buffer *cedrus_buf;
> + struct vb2_buffer *ref_buf;
> +
> + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
> + continue;
> +
> + ref_buf = ctx->dst_bufs[dpb->buf_index];
> + cedrus_buf = vb2_to_cedrus_buffer(ref_buf);
> + position = cedrus_buf->codec.h264.position;
> + used_dpbs |= BIT(position);
> +
> + sunxi_cedrus_fill_ref_pic(&pic_list[position], ref_buf,
> + ctx->codec.h264.mv_col_buf_dma,
> + ctx->codec.h264.mv_col_buf_size,
Following up on my previous comment, this should be specific to each
frame, with 2 buffer chunks per frame (top and bottom fields) as done in
Allwinner's H264MallocBuffer function.
[...]
> +static void sunxi_cedrus_set_params(struct sunxi_cedrus_ctx *ctx,
> + struct sunxi_cedrus_run *run)
> +{
> + const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
> + const struct v4l2_ctrl_h264_pps *pps = run->h264.pps;
> + const struct v4l2_ctrl_h264_sps *sps = run->h264.sps;
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + dma_addr_t src_buf_addr;
> + u32 offset = slice->header_bit_size;
> + u32 len = (slice->size * 8) - offset;
> + u32 reg;
> +
> + sunxi_cedrus_write(dev, ctx->codec.h264.pic_info_buf_dma - PHYS_OFFSET, 0x250);
> + sunxi_cedrus_write(dev, (ctx->codec.h264.pic_info_buf_dma - PHYS_OFFSET) + 0x48000, 0x254);
> +
> + sunxi_cedrus_write(dev, len, VE_H264_VLD_LEN);
> + sunxi_cedrus_write(dev, offset, VE_H264_VLD_OFFSET);
> +
> + src_buf_addr = vb2_dma_contig_plane_dma_addr(&run->src->vb2_buf, 0);
> + src_buf_addr -= PHYS_OFFSET;
> + sunxi_cedrus_write(dev, VE_H264_VLD_ADDR_VAL(src_buf_addr) |
> + VE_H264_VLD_ADDR_FIRST | VE_H264_VLD_ADDR_VALID | VE_H264_VLD_ADDR_LAST,
> + VE_H264_VLD_ADDR);
> + sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_H264_VLD_END);
> +
> + sunxi_cedrus_write(dev, VE_H264_TRIGGER_TYPE_INIT_SWDEC,
> + VE_H264_TRIGGER_TYPE);
It seems that this trigger type is only useful when trying
to subsequently access the bitstream data from the VPU (for easier
parsing, as done in libvdpau-sunxi), but it should not be required when
all the parsing was done already and no such access is necessary.
I haven't tested without it so far, but I have a hunch we can spare this
call.
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 9/9] media: cedrus: Add H264 decoding support
@ 2018-07-30 12:54 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-07-30 12:54 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> Introduce some basic H264 decoding support in cedrus. So far, only the
> baseline profile videos have been tested, and some more advanced features
> used in higher profiles are not even implemented.
While working on H265 support, I noticed a few things that should apply
to H264 as well.
[...]
> +struct sunxi_cedrus_h264_sram_ref_pic {
> + __le32 top_field_order_cnt;
> + __le32 bottom_field_order_cnt;
> + __le32 frame_info;
> + __le32 luma_ptr;
> + __le32 chroma_ptr;
> + __le32 extra_data_ptr;
> + __le32 extra_data_end;
These two previous fields represent the top and bottom (field) motion
vector column buffer addresses, so the second field is not the end of
the first one. These fields should be frame-specific and they are called
topmv_coladdr, botmv_coladdr by Allwinner.
> + __le32 reserved;
> +} __packed;
> +
[...]
> +static void sunxi_cedrus_fill_ref_pic(struct sunxi_cedrus_h264_sram_ref_pic *pic,
> + struct vb2_buffer *buf,
> + dma_addr_t extra_buf,
> + size_t extra_buf_len,
> + unsigned int top_field_order_cnt,
> + unsigned int bottom_field_order_cnt,
> + enum sunxi_cedrus_h264_pic_type pic_type)
> +{
> + pic->top_field_order_cnt = top_field_order_cnt;
> + pic->bottom_field_order_cnt = bottom_field_order_cnt;
> + pic->frame_info = pic_type << 8;
> + pic->luma_ptr = vb2_dma_contig_plane_dma_addr(buf, 0) - PHYS_OFFSET;
> + pic->chroma_ptr = vb2_dma_contig_plane_dma_addr(buf, 1) - PHYS_OFFSET;
> + pic->extra_data_ptr = extra_buf - PHYS_OFFSET;
> + pic->extra_data_end = (extra_buf - PHYS_OFFSET) + extra_buf_len;
> +}
> +
> +static void sunxi_cedrus_write_frame_list(struct sunxi_cedrus_ctx *ctx,
> + struct sunxi_cedrus_run *run)
> +{
> + struct sunxi_cedrus_h264_sram_ref_pic pic_list[SUNXI_CEDRUS_H264_FRAME_NUM];
> + const struct v4l2_ctrl_h264_decode_param *dec_param = run->h264.decode_param;
> + const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
> + const struct v4l2_ctrl_h264_sps *sps = run->h264.sps;
> + struct sunxi_cedrus_buffer *output_buf;
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + unsigned long used_dpbs = 0;
> + unsigned int position;
> + unsigned int output = 0;
> + unsigned int i;
> +
> + memset(pic_list, 0, sizeof(pic_list));
> +
> + for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) {
> + const struct v4l2_h264_dpb_entry *dpb = &dec_param->dpb[i];
> + const struct sunxi_cedrus_buffer *cedrus_buf;
> + struct vb2_buffer *ref_buf;
> +
> + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
> + continue;
> +
> + ref_buf = ctx->dst_bufs[dpb->buf_index];
> + cedrus_buf = vb2_to_cedrus_buffer(ref_buf);
> + position = cedrus_buf->codec.h264.position;
> + used_dpbs |= BIT(position);
> +
> + sunxi_cedrus_fill_ref_pic(&pic_list[position], ref_buf,
> + ctx->codec.h264.mv_col_buf_dma,
> + ctx->codec.h264.mv_col_buf_size,
Following up on my previous comment, this should be specific to each
frame, with 2 buffer chunks per frame (top and bottom fields) as done in
Allwinner's H264MallocBuffer function.
[...]
> +static void sunxi_cedrus_set_params(struct sunxi_cedrus_ctx *ctx,
> + struct sunxi_cedrus_run *run)
> +{
> + const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param;
> + const struct v4l2_ctrl_h264_pps *pps = run->h264.pps;
> + const struct v4l2_ctrl_h264_sps *sps = run->h264.sps;
> + struct sunxi_cedrus_dev *dev = ctx->dev;
> + dma_addr_t src_buf_addr;
> + u32 offset = slice->header_bit_size;
> + u32 len = (slice->size * 8) - offset;
> + u32 reg;
> +
> + sunxi_cedrus_write(dev, ctx->codec.h264.pic_info_buf_dma - PHYS_OFFSET, 0x250);
> + sunxi_cedrus_write(dev, (ctx->codec.h264.pic_info_buf_dma - PHYS_OFFSET) + 0x48000, 0x254);
> +
> + sunxi_cedrus_write(dev, len, VE_H264_VLD_LEN);
> + sunxi_cedrus_write(dev, offset, VE_H264_VLD_OFFSET);
> +
> + src_buf_addr = vb2_dma_contig_plane_dma_addr(&run->src->vb2_buf, 0);
> + src_buf_addr -= PHYS_OFFSET;
> + sunxi_cedrus_write(dev, VE_H264_VLD_ADDR_VAL(src_buf_addr) |
> + VE_H264_VLD_ADDR_FIRST | VE_H264_VLD_ADDR_VALID | VE_H264_VLD_ADDR_LAST,
> + VE_H264_VLD_ADDR);
> + sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_H264_VLD_END);
> +
> + sunxi_cedrus_write(dev, VE_H264_TRIGGER_TYPE_INIT_SWDEC,
> + VE_H264_TRIGGER_TYPE);
It seems that this trigger type is only useful when trying
to subsequently access the bitstream data from the VPU (for easier
parsing, as done in libvdpau-sunxi), but it should not be required when
all the parsing was done already and no such access is necessary.
I haven't tested without it so far, but I have a hunch we can spare this
call.
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180730/d5a59cac/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-06-13 14:07 ` Maxime Ripard
@ 2018-08-21 16:58 ` Ezequiel Garcia
-1 siblings, 0 replies; 102+ messages in thread
From: Ezequiel Garcia @ 2018-08-21 16:58 UTC (permalink / raw)
To: Maxime Ripard, tfiga, posciak, Paul Kocialkowski, hans.verkuil,
acourbot, sakari.ailus, Laurent Pinchart
Cc: Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media,
nicolas.dufresne, jenskuske, linux-sunxi, Thomas Petazzoni,
Guenter Roeck
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> From: Pawel Osciak <posciak@chromium.org>
>
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> Tested-by: Tomasz Figa <tfiga@chromium.org>
> [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> Signed-off-by: Guenter Roeck <groeck@chromium.org>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
>
[..]
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 242a6bfa1440..4b4a1b25a0db 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
should be aware of it. Perhaps we could document this pixel format better as:
#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
And introduce another pixel format:
#define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264 parsed slices without start codes */
For cedrus to use, as it seems it doesn't need start codes.
How does it sound?
[1] https://cs.chromium.org/chromium/src/media/gpu/v4l2/v4l2_slice_video_decode_accelerator.cc?rcl=63129434aeacf0f54bbae96814f10cf64e3e6c35&l=2438
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-21 16:58 ` Ezequiel Garcia
0 siblings, 0 replies; 102+ messages in thread
From: Ezequiel Garcia @ 2018-08-21 16:58 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> From: Pawel Osciak <posciak@chromium.org>
>
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> Tested-by: Tomasz Figa <tfiga@chromium.org>
> [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> Signed-off-by: Guenter Roeck <groeck@chromium.org>
> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> ---
>
[..]
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 242a6bfa1440..4b4a1b25a0db 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
should be aware of it. Perhaps we could document this pixel format better as:
#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
And introduce another pixel format:
#define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264 parsed slices without start codes */
For cedrus to use, as it seems it doesn't need start codes.
How does it sound?
[1] https://cs.chromium.org/chromium/src/media/gpu/v4l2/v4l2_slice_video_decode_accelerator.cc?rcl=63129434aeacf0f54bbae96814f10cf64e3e6c35&l=2438
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-21 16:58 ` Ezequiel Garcia
@ 2018-08-21 17:07 ` Nicolas Dufresne
-1 siblings, 0 replies; 102+ messages in thread
From: Nicolas Dufresne @ 2018-08-21 17:07 UTC (permalink / raw)
To: Ezequiel Garcia, Maxime Ripard, tfiga, posciak,
Paul Kocialkowski, hans.verkuil, acourbot, sakari.ailus,
Laurent Pinchart
Cc: Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media,
jenskuske, linux-sunxi, Thomas Petazzoni, Guenter Roeck
[-- Attachment #1: Type: text/plain, Size: 2432 bytes --]
Le mardi 21 août 2018 à 13:58 -0300, Ezequiel Garcia a écrit :
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > From: Pawel Osciak <posciak@chromium.org>
> >
> > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> >
>
> [..]
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 242a6bfa1440..4b4a1b25a0db 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > H264 with start codes */
> > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > H264 without start codes */
> > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > H264 MVC */
> > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > H264 parsed slices */
>
> As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> [1], so the userspace
> should be aware of it. Perhaps we could document this pixel format
> better as:
>
> #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> H264 parsed slices with start codes */
>
> And introduce another pixel format:
>
> #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> parsed slices without start codes */
>
> For cedrus to use, as it seems it doesn't need start codes.
I must admit that this RK requirement is a bit weird for slice data.
Though, userspace wise, always adding start-code would be compatible,
as the driver can just offset to remove it.
Another option, because I'm not fan of adding dedicated formats for
this, the RK driver could use data_offset (in mplane v4l2 buffers),
just write a start code there. I like this solution because I would not
be surprise if some drivers requires in fact an HW specific header,
that the driver can generate as needed.
>
> How does it sound?
>
> [1]
> https://cs.chromium.org/chromium/src/media/gpu/v4l2/v4l2_slice_video_decode_accelerator.cc?rcl=63129434aeacf0f54bbae96814f10cf64e3e6c35&l=2438
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-21 17:07 ` Nicolas Dufresne
0 siblings, 0 replies; 102+ messages in thread
From: Nicolas Dufresne @ 2018-08-21 17:07 UTC (permalink / raw)
To: linux-arm-kernel
Le mardi 21 ao?t 2018 ? 13:58 -0300, Ezequiel Garcia a ?crit :
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > From: Pawel Osciak <posciak@chromium.org>
> >
> > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> >
>
> [..]
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 242a6bfa1440..4b4a1b25a0db 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > H264 with start codes */
> > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > H264 without start codes */
> > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > H264 MVC */
> > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > H264 parsed slices */
>
> As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> [1], so the userspace
> should be aware of it. Perhaps we could document this pixel format
> better as:
>
> #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> H264 parsed slices with start codes */
>
> And introduce another pixel format:
>
> #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> parsed slices without start codes */
>
> For cedrus to use, as it seems it doesn't need start codes.
I must admit that this RK requirement is a bit weird for slice data.
Though, userspace wise, always adding start-code would be compatible,
as the driver can just offset to remove it.
Another option, because I'm not fan of adding dedicated formats for
this, the RK driver could use data_offset (in mplane v4l2 buffers),
just write a start code there. I like this solution because I would not
be surprise if some drivers requires in fact an HW specific header,
that the driver can generate as needed.
>
> How does it sound?
>
> [1]
> https://cs.chromium.org/chromium/src/media/gpu/v4l2/v4l2_slice_video_decode_accelerator.cc?rcl=63129434aeacf0f54bbae96814f10cf64e3e6c35&l=2438
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180821/de2e3094/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-21 16:58 ` Ezequiel Garcia
@ 2018-08-22 9:15 ` Maxime Ripard
-1 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-08-22 9:15 UTC (permalink / raw)
To: Ezequiel Garcia
Cc: tfiga, posciak, Paul Kocialkowski, hans.verkuil, acourbot,
sakari.ailus, Laurent Pinchart, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
linux-sunxi, Thomas Petazzoni, Guenter Roeck
[-- Attachment #1: Type: text/plain, Size: 1853 bytes --]
Hi,
On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > From: Pawel Osciak <posciak@chromium.org>
> >
> > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> >
> [..]
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 242a6bfa1440..4b4a1b25a0db 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
>
> As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> should be aware of it. Perhaps we could document this pixel format better as:
>
> #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
I'm not sure this is something we want to do at that point. libva
doesn't give the start code, so this is only going to make the life of
the sane controllers more difficult. And if you need to have the start
code and parse it, then you're not so stateless anymore.
Maxime
--
Maxime Ripard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-22 9:15 ` Maxime Ripard
0 siblings, 0 replies; 102+ messages in thread
From: Maxime Ripard @ 2018-08-22 9:15 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > From: Pawel Osciak <posciak@chromium.org>
> >
> > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > ---
> >
> [..]
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 242a6bfa1440..4b4a1b25a0db 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
>
> As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> should be aware of it. Perhaps we could document this pixel format better as:
>
> #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
I'm not sure this is something we want to do at that point. libva
doesn't give the start code, so this is only going to make the life of
the sane controllers more difficult. And if you need to have the start
code and parse it, then you're not so stateless anymore.
Maxime
--
Maxime Ripard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180822/07de23d0/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-22 9:15 ` Maxime Ripard
@ 2018-08-22 9:54 ` Tomasz Figa
-1 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-08-22 9:54 UTC (permalink / raw)
To: Maxime Ripard
Cc: Ezequiel Garcia, Pawel Osciak, Paul Kocialkowski, Hans Verkuil,
Alexandre Courbot, Sakari Ailus, Laurent Pinchart, Chen-Yu Tsai,
Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, Nicolas Dufresne, jenskuske,
linux-sunxi, thomas.petazzoni, groeck
On Wed, Aug 22, 2018 at 6:16 PM Maxime Ripard <maxime.ripard@bootlin.com> wrote:
>
> Hi,
>
> On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > From: Pawel Osciak <posciak@chromium.org>
> > >
> > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > >
> > [..]
> > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > --- a/include/uapi/linux/videodev2.h
> > > +++ b/include/uapi/linux/videodev2.h
> > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> >
> > As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> > should be aware of it. Perhaps we could document this pixel format better as:
> >
> > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
>
> I'm not sure this is something we want to do at that point. libva
> doesn't give the start code, so this is only going to make the life of
> the sane controllers more difficult. And if you need to have the start
> code and parse it, then you're not so stateless anymore.
I might not remember correctly, but Rockchip decoder does some slice
parsing on its own (despite not doing any higher level parsing).
Probably that's why it needs those start codes.
I wonder if libva is the best reference here. It's been designed
almost entirely by Intel for Intel video hardware. We want something
that could work with a wide range of devices and avoid something like
a need to create a semi-stateless API few months later. In fact,
hardware from another vendor, we're working with, also does parsing of
slice headers internally. Moreover, we have some weird
kind-of-stateful decoders, which cannot fully deal with bitstream on
its own, e.g. cannot parse formats, cannot handle resolution changes,
need H264 bitstream NALUs split into separate buffers, etc.
As I suggested some time ago, having the full bitstream in the buffer,
with offsets of particular units included in respective controls,
would be the most scalable thing. If really needed, we could add flags
telling the driver that particular units are present, so one's
implementation of libva could put only raw slice data in the buffers.
But perhaps it's libva which needs some amendment?
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-22 9:54 ` Tomasz Figa
0 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-08-22 9:54 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Aug 22, 2018 at 6:16 PM Maxime Ripard <maxime.ripard@bootlin.com> wrote:
>
> Hi,
>
> On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > From: Pawel Osciak <posciak@chromium.org>
> > >
> > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > >
> > [..]
> > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > --- a/include/uapi/linux/videodev2.h
> > > +++ b/include/uapi/linux/videodev2.h
> > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> >
> > As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> > should be aware of it. Perhaps we could document this pixel format better as:
> >
> > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
>
> I'm not sure this is something we want to do at that point. libva
> doesn't give the start code, so this is only going to make the life of
> the sane controllers more difficult. And if you need to have the start
> code and parse it, then you're not so stateless anymore.
I might not remember correctly, but Rockchip decoder does some slice
parsing on its own (despite not doing any higher level parsing).
Probably that's why it needs those start codes.
I wonder if libva is the best reference here. It's been designed
almost entirely by Intel for Intel video hardware. We want something
that could work with a wide range of devices and avoid something like
a need to create a semi-stateless API few months later. In fact,
hardware from another vendor, we're working with, also does parsing of
slice headers internally. Moreover, we have some weird
kind-of-stateful decoders, which cannot fully deal with bitstream on
its own, e.g. cannot parse formats, cannot handle resolution changes,
need H264 bitstream NALUs split into separate buffers, etc.
As I suggested some time ago, having the full bitstream in the buffer,
with offsets of particular units included in respective controls,
would be the most scalable thing. If really needed, we could add flags
telling the driver that particular units are present, so one's
implementation of libva could put only raw slice data in the buffers.
But perhaps it's libva which needs some amendment?
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-22 9:54 ` Tomasz Figa
@ 2018-08-22 13:03 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-08-22 13:03 UTC (permalink / raw)
To: Tomasz Figa, Maxime Ripard
Cc: Ezequiel Garcia, Pawel Osciak, Hans Verkuil, Alexandre Courbot,
Sakari Ailus, Laurent Pinchart, Chen-Yu Tsai,
Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, Nicolas Dufresne, jenskuske,
linux-sunxi, thomas.petazzoni, groeck
[-- Attachment #1: Type: text/plain, Size: 4725 bytes --]
Hi,
On Wed, 2018-08-22 at 18:54 +0900, Tomasz Figa wrote:
> On Wed, Aug 22, 2018 at 6:16 PM Maxime Ripard <maxime.ripard@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > From: Pawel Osciak <posciak@chromium.org>
> > > >
> > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > ---
> > > >
> > >
> > > [..]
> > > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > --- a/include/uapi/linux/videodev2.h
> > > > +++ b/include/uapi/linux/videodev2.h
> > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> > >
> > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> > > should be aware of it. Perhaps we could document this pixel format better as:
> > >
> > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
> >
> > I'm not sure this is something we want to do at that point. libva
> > doesn't give the start code, so this is only going to make the life of
> > the sane controllers more difficult. And if you need to have the start
> > code and parse it, then you're not so stateless anymore.
>
> I might not remember correctly, but Rockchip decoder does some slice
> parsing on its own (despite not doing any higher level parsing).
> Probably that's why it needs those start codes.
The VPU found on Allwinner platforms also provides a mechanism to parse
the bitstream data via a dedicated interface through the VPU registers.
It is used in libvdpau-sunxi but not in our driver, because we don't
want to be doing bitstream parsing in the kernel.
It would be good to know if this is just a feature of the Rockchip VPU
hardware that can be skipped (like on Allwinner) or if it's a hard
requirement in its decoding pipeline. Also, maybe it only concerns the
slice header? It is already part of the slice data (provided by VAAPI)
for H.264/H.265 and an offset is provided to the beginning of the coded
video data.
> I wonder if libva is the best reference here. It's been designed
> almost entirely by Intel for Intel video hardware. We want something
> that could work with a wide range of devices and avoid something like
> a need to create a semi-stateless API few months later. In fact,
> hardware from another vendor, we're working with, also does parsing of
> slice headers internally. Moreover, we have some weird
> kind-of-stateful decoders, which cannot fully deal with bitstream on
> its own, e.g. cannot parse formats, cannot handle resolution changes,
> need H264 bitstream NALUs split into separate buffers, etc.
>
> As I suggested some time ago, having the full bitstream in the buffer,
> with offsets of particular units included in respective controls,
> would be the most scalable thing. If really needed, we could add flags
> telling the driver that particular units are present, so one's
> implementation of libva could put only raw slice data in the buffers.
> But perhaps it's libva which needs some amendment?
If the raw bitstream is needed, I think it would make more sense to use
the already-existing formats for stateful VPUs along with the controls
for stateless ones instead of having the full bitstream in the
V4L2_PIX_FMT_*_SLICE formats.
I would also be tempted to say that reconstructing the needed parts of
the bitstream in-driver for these half-way VPUs would be a better
approach than blurrying the line between how (and what) data should be
passed for stateful and stateless VPUs at the API level. Stateless
should only cover what's in the slice NAL unit RBSP, which excludes the
start code detection bytes. It is no longer parsed data otherwise.
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-22 13:03 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-08-22 13:03 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-08-22 at 18:54 +0900, Tomasz Figa wrote:
> On Wed, Aug 22, 2018 at 6:16 PM Maxime Ripard <maxime.ripard@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > From: Pawel Osciak <posciak@chromium.org>
> > > >
> > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > ---
> > > >
> > >
> > > [..]
> > > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > --- a/include/uapi/linux/videodev2.h
> > > > +++ b/include/uapi/linux/videodev2.h
> > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> > >
> > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> > > should be aware of it. Perhaps we could document this pixel format better as:
> > >
> > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
> >
> > I'm not sure this is something we want to do at that point. libva
> > doesn't give the start code, so this is only going to make the life of
> > the sane controllers more difficult. And if you need to have the start
> > code and parse it, then you're not so stateless anymore.
>
> I might not remember correctly, but Rockchip decoder does some slice
> parsing on its own (despite not doing any higher level parsing).
> Probably that's why it needs those start codes.
The VPU found on Allwinner platforms also provides a mechanism to parse
the bitstream data via a dedicated interface through the VPU registers.
It is used in libvdpau-sunxi but not in our driver, because we don't
want to be doing bitstream parsing in the kernel.
It would be good to know if this is just a feature of the Rockchip VPU
hardware that can be skipped (like on Allwinner) or if it's a hard
requirement in its decoding pipeline. Also, maybe it only concerns the
slice header? It is already part of the slice data (provided by VAAPI)
for H.264/H.265 and an offset is provided to the beginning of the coded
video data.
> I wonder if libva is the best reference here. It's been designed
> almost entirely by Intel for Intel video hardware. We want something
> that could work with a wide range of devices and avoid something like
> a need to create a semi-stateless API few months later. In fact,
> hardware from another vendor, we're working with, also does parsing of
> slice headers internally. Moreover, we have some weird
> kind-of-stateful decoders, which cannot fully deal with bitstream on
> its own, e.g. cannot parse formats, cannot handle resolution changes,
> need H264 bitstream NALUs split into separate buffers, etc.
>
> As I suggested some time ago, having the full bitstream in the buffer,
> with offsets of particular units included in respective controls,
> would be the most scalable thing. If really needed, we could add flags
> telling the driver that particular units are present, so one's
> implementation of libva could put only raw slice data in the buffers.
> But perhaps it's libva which needs some amendment?
If the raw bitstream is needed, I think it would make more sense to use
the already-existing formats for stateful VPUs along with the controls
for stateless ones instead of having the full bitstream in the
V4L2_PIX_FMT_*_SLICE formats.
I would also be tempted to say that reconstructing the needed parts of
the bitstream in-driver for these half-way VPUs would be a better
approach than blurrying the line between how (and what) data should be
passed for stateful and stateless VPUs at the API level. Stateless
should only cover what's in the slice NAL unit RBSP, which excludes the
start code detection bytes. It is no longer parsed data otherwise.
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180822/65c5896b/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-21 17:07 ` Nicolas Dufresne
@ 2018-08-22 13:07 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-08-22 13:07 UTC (permalink / raw)
To: Nicolas Dufresne, Ezequiel Garcia, Maxime Ripard, tfiga, posciak,
hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media,
jenskuske, linux-sunxi, Thomas Petazzoni, Guenter Roeck
[-- Attachment #1: Type: text/plain, Size: 3100 bytes --]
Hi,
On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> Le mardi 21 août 2018 à 13:58 -0300, Ezequiel Garcia a écrit :
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > From: Pawel Osciak <posciak@chromium.org>
> > >
> > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > >
> >
> > [..]
> > > diff --git a/include/uapi/linux/videodev2.h
> > > b/include/uapi/linux/videodev2.h
> > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > --- a/include/uapi/linux/videodev2.h
> > > +++ b/include/uapi/linux/videodev2.h
> > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > H264 with start codes */
> > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > H264 without start codes */
> > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > H264 MVC */
> > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > H264 parsed slices */
> >
> > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > [1], so the userspace
> > should be aware of it. Perhaps we could document this pixel format
> > better as:
> >
> > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > H264 parsed slices with start codes */
> >
> > And introduce another pixel format:
> >
> > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > parsed slices without start codes */
> >
> > For cedrus to use, as it seems it doesn't need start codes.
>
> I must admit that this RK requirement is a bit weird for slice data.
> Though, userspace wise, always adding start-code would be compatible,
> as the driver can just offset to remove it.
This would mean that the stateless API no longer takes parsed bitstream
data but effectively the full bitstream, which defeats the purpose of
the _SLICE pixel formats.
> Another option, because I'm not fan of adding dedicated formats for
> this, the RK driver could use data_offset (in mplane v4l2 buffers),
> just write a start code there. I like this solution because I would not
> be surprise if some drivers requires in fact an HW specific header,
> that the driver can generate as needed.
I like this idea, because it implies that the driver should deal with
the specificities of the hardware, instead of making the blurrying the
lines of stateless API for covering these cases.
> >
> > How does it sound?
> >
> > [1]
> > https://cs.chromium.org/chromium/src/media/gpu/v4l2/v4l2_slice_video_decode_accelerator.cc?rcl=63129434aeacf0f54bbae96814f10cf64e3e6c35&l=2438
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-22 13:07 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-08-22 13:07 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> Le mardi 21 ao?t 2018 ? 13:58 -0300, Ezequiel Garcia a ?crit :
> > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > From: Pawel Osciak <posciak@chromium.org>
> > >
> > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > ---
> > >
> >
> > [..]
> > > diff --git a/include/uapi/linux/videodev2.h
> > > b/include/uapi/linux/videodev2.h
> > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > --- a/include/uapi/linux/videodev2.h
> > > +++ b/include/uapi/linux/videodev2.h
> > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > H264 with start codes */
> > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > H264 without start codes */
> > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > H264 MVC */
> > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > H264 parsed slices */
> >
> > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > [1], so the userspace
> > should be aware of it. Perhaps we could document this pixel format
> > better as:
> >
> > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > H264 parsed slices with start codes */
> >
> > And introduce another pixel format:
> >
> > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > parsed slices without start codes */
> >
> > For cedrus to use, as it seems it doesn't need start codes.
>
> I must admit that this RK requirement is a bit weird for slice data.
> Though, userspace wise, always adding start-code would be compatible,
> as the driver can just offset to remove it.
This would mean that the stateless API no longer takes parsed bitstream
data but effectively the full bitstream, which defeats the purpose of
the _SLICE pixel formats.
> Another option, because I'm not fan of adding dedicated formats for
> this, the RK driver could use data_offset (in mplane v4l2 buffers),
> just write a start code there. I like this solution because I would not
> be surprise if some drivers requires in fact an HW specific header,
> that the driver can generate as needed.
I like this idea, because it implies that the driver should deal with
the specificities of the hardware, instead of making the blurrying the
lines of stateless API for covering these cases.
> >
> > How does it sound?
> >
> > [1]
> > https://cs.chromium.org/chromium/src/media/gpu/v4l2/v4l2_slice_video_decode_accelerator.cc?rcl=63129434aeacf0f54bbae96814f10cf64e3e6c35&l=2438
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180822/b16d18ac/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-22 13:03 ` Paul Kocialkowski
@ 2018-08-22 13:24 ` Tomasz Figa
-1 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-08-22 13:24 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: Maxime Ripard, Ezequiel Garcia, Pawel Osciak, Hans Verkuil,
Alexandre Courbot, Sakari Ailus, Laurent Pinchart, Chen-Yu Tsai,
Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, Nicolas Dufresne, jenskuske,
linux-sunxi, thomas.petazzoni, groeck
On Wed, Aug 22, 2018 at 10:03 PM Paul Kocialkowski
<paul.kocialkowski@bootlin.com> wrote:
>
> Hi,
>
> On Wed, 2018-08-22 at 18:54 +0900, Tomasz Figa wrote:
> > On Wed, Aug 22, 2018 at 6:16 PM Maxime Ripard <maxime.ripard@bootlin.com> wrote:
> > >
> > > Hi,
> > >
> > > On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > >
> > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > ---
> > > > >
> > > >
> > > > [..]
> > > > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > --- a/include/uapi/linux/videodev2.h
> > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> > > >
> > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> > > > should be aware of it. Perhaps we could document this pixel format better as:
> > > >
> > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
> > >
> > > I'm not sure this is something we want to do at that point. libva
> > > doesn't give the start code, so this is only going to make the life of
> > > the sane controllers more difficult. And if you need to have the start
> > > code and parse it, then you're not so stateless anymore.
> >
> > I might not remember correctly, but Rockchip decoder does some slice
> > parsing on its own (despite not doing any higher level parsing).
> > Probably that's why it needs those start codes.
>
> The VPU found on Allwinner platforms also provides a mechanism to parse
> the bitstream data via a dedicated interface through the VPU registers.
> It is used in libvdpau-sunxi but not in our driver, because we don't
> want to be doing bitstream parsing in the kernel.
>
> It would be good to know if this is just a feature of the Rockchip VPU
> hardware that can be skipped (like on Allwinner) or if it's a hard
> requirement in its decoding pipeline.
It's a hard requirement for its decoding pipeline, but...
> Also, maybe it only concerns the
> slice header? It is already part of the slice data (provided by VAAPI)
> for H.264/H.265 and an offset is provided to the beginning of the coded
> video data.
Yes, it seems to be only the slice header.
>
> > I wonder if libva is the best reference here. It's been designed
> > almost entirely by Intel for Intel video hardware. We want something
> > that could work with a wide range of devices and avoid something like
> > a need to create a semi-stateless API few months later. In fact,
> > hardware from another vendor, we're working with, also does parsing of
> > slice headers internally. Moreover, we have some weird
> > kind-of-stateful decoders, which cannot fully deal with bitstream on
> > its own, e.g. cannot parse formats, cannot handle resolution changes,
> > need H264 bitstream NALUs split into separate buffers, etc.
> >
> > As I suggested some time ago, having the full bitstream in the buffer,
> > with offsets of particular units included in respective controls,
> > would be the most scalable thing. If really needed, we could add flags
> > telling the driver that particular units are present, so one's
> > implementation of libva could put only raw slice data in the buffers.
> > But perhaps it's libva which needs some amendment?
>
> If the raw bitstream is needed, I think it would make more sense to use
> the already-existing formats for stateful VPUs along with the controls
> for stateless ones instead of having the full bitstream in the
> V4L2_PIX_FMT_*_SLICE formats.
It may indeed make sense to separate this based on pixel format.
However, how do we tell the client that it needs to provide those
controls? Current concept was based entirely on pixel format, so I
guess that would mean creating something like
V4L2_PIX_FMT_*_NOT_REALLY_SLICE (_PARSED, _STATELESS?). Might be okay,
though...
>
> I would also be tempted to say that reconstructing the needed parts of
> the bitstream in-driver for these half-way VPUs would be a better
> approach than blurrying the line between how (and what) data should be
> passed for stateful and stateless VPUs at the API level. Stateless
> should only cover what's in the slice NAL unit RBSP, which excludes the
> start code detection bytes. It is no longer parsed data otherwise.
I'm not sure where such decision comes from. In particular, Chromium,
from which this code originates, includes start codes in
V4L2_PIX_FMT_H264_SLICE. As I mentioned earlier, we can't design this
API based only on 1 type of hardware semantics. The stateless API
should cover any kind of codec that needs user space assistance in
processing the stream, which in practice would be almost everything
for which stateful API doesn't work.
That said, since pixel format essentially specifies the buffer
contents, having such cases differentiated based on the pixel format
doesn't sound insane.
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-22 13:24 ` Tomasz Figa
0 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-08-22 13:24 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Aug 22, 2018 at 10:03 PM Paul Kocialkowski
<paul.kocialkowski@bootlin.com> wrote:
>
> Hi,
>
> On Wed, 2018-08-22 at 18:54 +0900, Tomasz Figa wrote:
> > On Wed, Aug 22, 2018 at 6:16 PM Maxime Ripard <maxime.ripard@bootlin.com> wrote:
> > >
> > > Hi,
> > >
> > > On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > >
> > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > ---
> > > > >
> > > >
> > > > [..]
> > > > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > --- a/include/uapi/linux/videodev2.h
> > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> > > >
> > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> > > > should be aware of it. Perhaps we could document this pixel format better as:
> > > >
> > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
> > >
> > > I'm not sure this is something we want to do at that point. libva
> > > doesn't give the start code, so this is only going to make the life of
> > > the sane controllers more difficult. And if you need to have the start
> > > code and parse it, then you're not so stateless anymore.
> >
> > I might not remember correctly, but Rockchip decoder does some slice
> > parsing on its own (despite not doing any higher level parsing).
> > Probably that's why it needs those start codes.
>
> The VPU found on Allwinner platforms also provides a mechanism to parse
> the bitstream data via a dedicated interface through the VPU registers.
> It is used in libvdpau-sunxi but not in our driver, because we don't
> want to be doing bitstream parsing in the kernel.
>
> It would be good to know if this is just a feature of the Rockchip VPU
> hardware that can be skipped (like on Allwinner) or if it's a hard
> requirement in its decoding pipeline.
It's a hard requirement for its decoding pipeline, but...
> Also, maybe it only concerns the
> slice header? It is already part of the slice data (provided by VAAPI)
> for H.264/H.265 and an offset is provided to the beginning of the coded
> video data.
Yes, it seems to be only the slice header.
>
> > I wonder if libva is the best reference here. It's been designed
> > almost entirely by Intel for Intel video hardware. We want something
> > that could work with a wide range of devices and avoid something like
> > a need to create a semi-stateless API few months later. In fact,
> > hardware from another vendor, we're working with, also does parsing of
> > slice headers internally. Moreover, we have some weird
> > kind-of-stateful decoders, which cannot fully deal with bitstream on
> > its own, e.g. cannot parse formats, cannot handle resolution changes,
> > need H264 bitstream NALUs split into separate buffers, etc.
> >
> > As I suggested some time ago, having the full bitstream in the buffer,
> > with offsets of particular units included in respective controls,
> > would be the most scalable thing. If really needed, we could add flags
> > telling the driver that particular units are present, so one's
> > implementation of libva could put only raw slice data in the buffers.
> > But perhaps it's libva which needs some amendment?
>
> If the raw bitstream is needed, I think it would make more sense to use
> the already-existing formats for stateful VPUs along with the controls
> for stateless ones instead of having the full bitstream in the
> V4L2_PIX_FMT_*_SLICE formats.
It may indeed make sense to separate this based on pixel format.
However, how do we tell the client that it needs to provide those
controls? Current concept was based entirely on pixel format, so I
guess that would mean creating something like
V4L2_PIX_FMT_*_NOT_REALLY_SLICE (_PARSED, _STATELESS?). Might be okay,
though...
>
> I would also be tempted to say that reconstructing the needed parts of
> the bitstream in-driver for these half-way VPUs would be a better
> approach than blurrying the line between how (and what) data should be
> passed for stateful and stateless VPUs at the API level. Stateless
> should only cover what's in the slice NAL unit RBSP, which excludes the
> start code detection bytes. It is no longer parsed data otherwise.
I'm not sure where such decision comes from. In particular, Chromium,
from which this code originates, includes start codes in
V4L2_PIX_FMT_H264_SLICE. As I mentioned earlier, we can't design this
API based only on 1 type of hardware semantics. The stateless API
should cover any kind of codec that needs user space assistance in
processing the stream, which in practice would be almost everything
for which stateful API doesn't work.
That said, since pixel format essentially specifies the buffer
contents, having such cases differentiated based on the pixel format
doesn't sound insane.
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-22 13:07 ` Paul Kocialkowski
@ 2018-08-22 13:38 ` Tomasz Figa
-1 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-08-22 13:38 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: Nicolas Dufresne, Ezequiel Garcia, Maxime Ripard, Pawel Osciak,
Hans Verkuil, Alexandre Courbot, Sakari Ailus, Laurent Pinchart,
Chen-Yu Tsai, Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, jenskuske, linux-sunxi,
thomas.petazzoni, groeck
On Wed, Aug 22, 2018 at 10:07 PM Paul Kocialkowski
<paul.kocialkowski@bootlin.com> wrote:
>
> Hi,
>
> On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> > Le mardi 21 août 2018 à 13:58 -0300, Ezequiel Garcia a écrit :
> > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > From: Pawel Osciak <posciak@chromium.org>
> > > >
> > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > ---
> > > >
> > >
> > > [..]
> > > > diff --git a/include/uapi/linux/videodev2.h
> > > > b/include/uapi/linux/videodev2.h
> > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > --- a/include/uapi/linux/videodev2.h
> > > > +++ b/include/uapi/linux/videodev2.h
> > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > > H264 with start codes */
> > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > > H264 without start codes */
> > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > > H264 MVC */
> > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > H264 parsed slices */
> > >
> > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > > [1], so the userspace
> > > should be aware of it. Perhaps we could document this pixel format
> > > better as:
> > >
> > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > H264 parsed slices with start codes */
> > >
> > > And introduce another pixel format:
> > >
> > > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > > parsed slices without start codes */
> > >
> > > For cedrus to use, as it seems it doesn't need start codes.
> >
> > I must admit that this RK requirement is a bit weird for slice data.
> > Though, userspace wise, always adding start-code would be compatible,
> > as the driver can just offset to remove it.
>
> This would mean that the stateless API no longer takes parsed bitstream
> data but effectively the full bitstream, which defeats the purpose of
> the _SLICE pixel formats.
>
Not entirely. One of the purposes of the _SLICE pixel format was to
specify it in a way that adds a requirement of providing the required
controls by the client.
> > Another option, because I'm not fan of adding dedicated formats for
> > this, the RK driver could use data_offset (in mplane v4l2 buffers),
> > just write a start code there. I like this solution because I would not
> > be surprise if some drivers requires in fact an HW specific header,
> > that the driver can generate as needed.
>
> I like this idea, because it implies that the driver should deal with
> the specificities of the hardware, instead of making the blurrying the
> lines of stateless API for covering these cases.
The spec says
"Offset in bytes to video data in the plane. Drivers must set this
field when type refers to a capture stream, applications when it
refers to an output stream."
which would mean that user space would have to know to reserve some
bytes at the beginning for the driver to add the start code there. (Or
the driver memmove()ing the data forward when the buffer is queued,
assuming that there is enough space in the buffer, but it should
normally be the case.)
Sounds like a pixel format with full bitstream data and some offsets
to particular parts inside given inside a control might be the most
flexible and cleanest solution.
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-22 13:38 ` Tomasz Figa
0 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-08-22 13:38 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Aug 22, 2018 at 10:07 PM Paul Kocialkowski
<paul.kocialkowski@bootlin.com> wrote:
>
> Hi,
>
> On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> > Le mardi 21 ao?t 2018 ? 13:58 -0300, Ezequiel Garcia a ?crit :
> > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > From: Pawel Osciak <posciak@chromium.org>
> > > >
> > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > ---
> > > >
> > >
> > > [..]
> > > > diff --git a/include/uapi/linux/videodev2.h
> > > > b/include/uapi/linux/videodev2.h
> > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > --- a/include/uapi/linux/videodev2.h
> > > > +++ b/include/uapi/linux/videodev2.h
> > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > > H264 with start codes */
> > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > > H264 without start codes */
> > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > > H264 MVC */
> > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > H264 parsed slices */
> > >
> > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > > [1], so the userspace
> > > should be aware of it. Perhaps we could document this pixel format
> > > better as:
> > >
> > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > H264 parsed slices with start codes */
> > >
> > > And introduce another pixel format:
> > >
> > > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > > parsed slices without start codes */
> > >
> > > For cedrus to use, as it seems it doesn't need start codes.
> >
> > I must admit that this RK requirement is a bit weird for slice data.
> > Though, userspace wise, always adding start-code would be compatible,
> > as the driver can just offset to remove it.
>
> This would mean that the stateless API no longer takes parsed bitstream
> data but effectively the full bitstream, which defeats the purpose of
> the _SLICE pixel formats.
>
Not entirely. One of the purposes of the _SLICE pixel format was to
specify it in a way that adds a requirement of providing the required
controls by the client.
> > Another option, because I'm not fan of adding dedicated formats for
> > this, the RK driver could use data_offset (in mplane v4l2 buffers),
> > just write a start code there. I like this solution because I would not
> > be surprise if some drivers requires in fact an HW specific header,
> > that the driver can generate as needed.
>
> I like this idea, because it implies that the driver should deal with
> the specificities of the hardware, instead of making the blurrying the
> lines of stateless API for covering these cases.
The spec says
"Offset in bytes to video data in the plane. Drivers must set this
field when type refers to a capture stream, applications when it
refers to an output stream."
which would mean that user space would have to know to reserve some
bytes at the beginning for the driver to add the start code there. (Or
the driver memmove()ing the data forward when the buffer is queued,
assuming that there is enough space in the buffer, but it should
normally be the case.)
Sounds like a pixel format with full bitstream data and some offsets
to particular parts inside given inside a control might be the most
flexible and cleanest solution.
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-22 13:38 ` Tomasz Figa
@ 2018-08-22 13:52 ` Nicolas Dufresne
-1 siblings, 0 replies; 102+ messages in thread
From: Nicolas Dufresne @ 2018-08-22 13:52 UTC (permalink / raw)
To: Tomasz Figa, Paul Kocialkowski
Cc: Ezequiel Garcia, Maxime Ripard, Pawel Osciak, Hans Verkuil,
Alexandre Courbot, Sakari Ailus, Laurent Pinchart, Chen-Yu Tsai,
Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, jenskuske, linux-sunxi,
thomas.petazzoni, groeck
[-- Attachment #1: Type: text/plain, Size: 5316 bytes --]
Le mercredi 22 août 2018 à 22:38 +0900, Tomasz Figa a écrit :
> On Wed, Aug 22, 2018 at 10:07 PM Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> > > Le mardi 21 août 2018 à 13:58 -0300, Ezequiel Garcia a écrit :
> > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > >
> > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > ---
> > > > >
> > > >
> > > > [..]
> > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > b/include/uapi/linux/videodev2.h
> > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > --- a/include/uapi/linux/videodev2.h
> > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > > > H264 with start codes */
> > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > > > H264 without start codes */
> > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > > > H264 MVC */
> > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > H264 parsed slices */
> > > >
> > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > > > [1], so the userspace
> > > > should be aware of it. Perhaps we could document this pixel format
> > > > better as:
> > > >
> > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > H264 parsed slices with start codes */
> > > >
> > > > And introduce another pixel format:
> > > >
> > > > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > > > parsed slices without start codes */
> > > >
> > > > For cedrus to use, as it seems it doesn't need start codes.
> > >
> > > I must admit that this RK requirement is a bit weird for slice data.
> > > Though, userspace wise, always adding start-code would be compatible,
> > > as the driver can just offset to remove it.
> >
> > This would mean that the stateless API no longer takes parsed bitstream
> > data but effectively the full bitstream, which defeats the purpose of
> > the _SLICE pixel formats.
> >
>
> Not entirely. One of the purposes of the _SLICE pixel format was to
> specify it in a way that adds a requirement of providing the required
> controls by the client.
Slice is also a bitstream alignment requirement. In the default _H264
format we pass an complete AU, while in _SLICE we will pass single NAL
(along with the tables through controls of course). Though, H264 does
not only support start codes, it also supports AVC headers (as stored
in ISOMP4). That makes the sentence "passing the full bitstream" quite
ambiguous, as whatever we require, userspace may endup having to
replace that part of the NAL.
>
> > > Another option, because I'm not fan of adding dedicated formats for
> > > this, the RK driver could use data_offset (in mplane v4l2 buffers),
> > > just write a start code there. I like this solution because I would not
> > > be surprise if some drivers requires in fact an HW specific header,
> > > that the driver can generate as needed.
> >
> > I like this idea, because it implies that the driver should deal with
> > the specificities of the hardware, instead of making the blurrying the
> > lines of stateless API for covering these cases.
>
> The spec says
>
> "Offset in bytes to video data in the plane. Drivers must set this
> field when type refers to a capture stream, applications when it
> refers to an output stream."
>
> which would mean that user space would have to know to reserve some
> bytes at the beginning for the driver to add the start code there. (Or
> the driver memmove()ing the data forward when the buffer is queued,
> assuming that there is enough space in the buffer, but it should
> normally be the case.)
>
> Sounds like a pixel format with full bitstream data and some offsets
> to particular parts inside given inside a control might be the most
> flexible and cleanest solution.
Yeah, that comment was miss-informed. But it's simpler then that, the
prefix can be added internally to the driver, there is no need to
communicate this prefix to userspace.
Again, if you go the pixel format way, you'll need to resolve the large
ambiguity of saying "the full bitstream data". While dealing with a
prefix in the driver, is just about writing couple of bytes, and then
you can support HW that wants start-codes, AVC/AVC3 headers or HW
specific headers. I've seen a totally custom header on Panasonic HW
accelerator on some smart TV.
The other important aspect, is that VAAPI won't provide any headers,
just the raw NAL. So you'd be forcing userspace to handle a HW specific
requirement, which is trivial to adapt.
>
> Best regards,
> Tomasz
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-22 13:52 ` Nicolas Dufresne
0 siblings, 0 replies; 102+ messages in thread
From: Nicolas Dufresne @ 2018-08-22 13:52 UTC (permalink / raw)
To: linux-arm-kernel
Le mercredi 22 ao?t 2018 ? 22:38 +0900, Tomasz Figa a ?crit :
> On Wed, Aug 22, 2018 at 10:07 PM Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> > > Le mardi 21 ao?t 2018 ? 13:58 -0300, Ezequiel Garcia a ?crit :
> > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > >
> > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > ---
> > > > >
> > > >
> > > > [..]
> > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > b/include/uapi/linux/videodev2.h
> > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > --- a/include/uapi/linux/videodev2.h
> > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > > > H264 with start codes */
> > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > > > H264 without start codes */
> > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > > > H264 MVC */
> > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > H264 parsed slices */
> > > >
> > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > > > [1], so the userspace
> > > > should be aware of it. Perhaps we could document this pixel format
> > > > better as:
> > > >
> > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > H264 parsed slices with start codes */
> > > >
> > > > And introduce another pixel format:
> > > >
> > > > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > > > parsed slices without start codes */
> > > >
> > > > For cedrus to use, as it seems it doesn't need start codes.
> > >
> > > I must admit that this RK requirement is a bit weird for slice data.
> > > Though, userspace wise, always adding start-code would be compatible,
> > > as the driver can just offset to remove it.
> >
> > This would mean that the stateless API no longer takes parsed bitstream
> > data but effectively the full bitstream, which defeats the purpose of
> > the _SLICE pixel formats.
> >
>
> Not entirely. One of the purposes of the _SLICE pixel format was to
> specify it in a way that adds a requirement of providing the required
> controls by the client.
Slice is also a bitstream alignment requirement. In the default _H264
format we pass an complete AU, while in _SLICE we will pass single NAL
(along with the tables through controls of course). Though, H264 does
not only support start codes, it also supports AVC headers (as stored
in ISOMP4). That makes the sentence "passing the full bitstream" quite
ambiguous, as whatever we require, userspace may endup having to
replace that part of the NAL.
>
> > > Another option, because I'm not fan of adding dedicated formats for
> > > this, the RK driver could use data_offset (in mplane v4l2 buffers),
> > > just write a start code there. I like this solution because I would not
> > > be surprise if some drivers requires in fact an HW specific header,
> > > that the driver can generate as needed.
> >
> > I like this idea, because it implies that the driver should deal with
> > the specificities of the hardware, instead of making the blurrying the
> > lines of stateless API for covering these cases.
>
> The spec says
>
> "Offset in bytes to video data in the plane. Drivers must set this
> field when type refers to a capture stream, applications when it
> refers to an output stream."
>
> which would mean that user space would have to know to reserve some
> bytes at the beginning for the driver to add the start code there. (Or
> the driver memmove()ing the data forward when the buffer is queued,
> assuming that there is enough space in the buffer, but it should
> normally be the case.)
>
> Sounds like a pixel format with full bitstream data and some offsets
> to particular parts inside given inside a control might be the most
> flexible and cleanest solution.
Yeah, that comment was miss-informed. But it's simpler then that, the
prefix can be added internally to the driver, there is no need to
communicate this prefix to userspace.
Again, if you go the pixel format way, you'll need to resolve the large
ambiguity of saying "the full bitstream data". While dealing with a
prefix in the driver, is just about writing couple of bytes, and then
you can support HW that wants start-codes, AVC/AVC3 headers or HW
specific headers. I've seen a totally custom header on Panasonic HW
accelerator on some smart TV.
The other important aspect, is that VAAPI won't provide any headers,
just the raw NAL. So you'd be forcing userspace to handle a HW specific
requirement, which is trivial to adapt.
>
> Best regards,
> Tomasz
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180822/ab0c6cf5/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-22 13:24 ` Tomasz Figa
@ 2018-08-22 14:03 ` Nicolas Dufresne
-1 siblings, 0 replies; 102+ messages in thread
From: Nicolas Dufresne @ 2018-08-22 14:03 UTC (permalink / raw)
To: Tomasz Figa, Paul Kocialkowski
Cc: Maxime Ripard, Ezequiel Garcia, Pawel Osciak, Hans Verkuil,
Alexandre Courbot, Sakari Ailus, Laurent Pinchart, Chen-Yu Tsai,
Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, jenskuske, linux-sunxi,
thomas.petazzoni, groeck
[-- Attachment #1: Type: text/plain, Size: 7017 bytes --]
Le mercredi 22 août 2018 à 22:24 +0900, Tomasz Figa a écrit :
> On Wed, Aug 22, 2018 at 10:03 PM Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Wed, 2018-08-22 at 18:54 +0900, Tomasz Figa wrote:
> > > On Wed, Aug 22, 2018 at 6:16 PM Maxime Ripard <maxime.ripard@bootlin.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> > > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > > >
> > > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > > ---
> > > > > >
> > > > >
> > > > > [..]
> > > > > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > > --- a/include/uapi/linux/videodev2.h
> > > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> > > > >
> > > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> > > > > should be aware of it. Perhaps we could document this pixel format better as:
> > > > >
> > > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
> > > >
> > > > I'm not sure this is something we want to do at that point. libva
> > > > doesn't give the start code, so this is only going to make the life of
> > > > the sane controllers more difficult. And if you need to have the start
> > > > code and parse it, then you're not so stateless anymore.
> > >
> > > I might not remember correctly, but Rockchip decoder does some slice
> > > parsing on its own (despite not doing any higher level parsing).
> > > Probably that's why it needs those start codes.
> >
> > The VPU found on Allwinner platforms also provides a mechanism to parse
> > the bitstream data via a dedicated interface through the VPU registers.
> > It is used in libvdpau-sunxi but not in our driver, because we don't
> > want to be doing bitstream parsing in the kernel.
> >
> > It would be good to know if this is just a feature of the Rockchip VPU
> > hardware that can be skipped (like on Allwinner) or if it's a hard
> > requirement in its decoding pipeline.
>
> It's a hard requirement for its decoding pipeline, but...
>
> > Also, maybe it only concerns the
> > slice header? It is already part of the slice data (provided by VAAPI)
> > for H.264/H.265 and an offset is provided to the beginning of the coded
> > video data.
>
> Yes, it seems to be only the slice header.
>
> >
> > > I wonder if libva is the best reference here. It's been designed
> > > almost entirely by Intel for Intel video hardware. We want something
> > > that could work with a wide range of devices and avoid something like
> > > a need to create a semi-stateless API few months later. In fact,
> > > hardware from another vendor, we're working with, also does parsing of
> > > slice headers internally. Moreover, we have some weird
> > > kind-of-stateful decoders, which cannot fully deal with bitstream on
> > > its own, e.g. cannot parse formats, cannot handle resolution changes,
> > > need H264 bitstream NALUs split into separate buffers, etc.
> > >
> > > As I suggested some time ago, having the full bitstream in the buffer,
> > > with offsets of particular units included in respective controls,
> > > would be the most scalable thing. If really needed, we could add flags
> > > telling the driver that particular units are present, so one's
> > > implementation of libva could put only raw slice data in the buffers.
> > > But perhaps it's libva which needs some amendment?
> >
> > If the raw bitstream is needed, I think it would make more sense to use
> > the already-existing formats for stateful VPUs along with the controls
> > for stateless ones instead of having the full bitstream in the
> > V4L2_PIX_FMT_*_SLICE formats.
>
> It may indeed make sense to separate this based on pixel format.
> However, how do we tell the client that it needs to provide those
> controls? Current concept was based entirely on pixel format, so I
> guess that would mean creating something like
> V4L2_PIX_FMT_*_NOT_REALLY_SLICE (_PARSED, _STATELESS?). Might be okay,
> though...
>
> >
> > I would also be tempted to say that reconstructing the needed parts of
> > the bitstream in-driver for these half-way VPUs would be a better
> > approach than blurrying the line between how (and what) data should be
> > passed for stateful and stateless VPUs at the API level. Stateless
> > should only cover what's in the slice NAL unit RBSP, which excludes the
> > start code detection bytes. It is no longer parsed data otherwise.
>
> I'm not sure where such decision comes from. In particular, Chromium,
> from which this code originates, includes start codes in
> V4L2_PIX_FMT_H264_SLICE. As I mentioned earlier, we can't design this
> API based only on 1 type of hardware semantics. The stateless API
> should cover any kind of codec that needs user space assistance in
> processing the stream, which in practice would be almost everything
> for which stateful API doesn't work.
Well, be aware that in most of today's use case, Chromium bitstream
handling is suboptimal since the startcode are crafted by Chromium.
Most of today's use case uses ISOMP4 container, which does not hold
start codes but AVC headers.
If we step back a little, the start codes are designed for randomly
accessing bytes. It's a mark made to be able to randomly scan and find
the start of a NAL. In a well framed stream, they are useless. Then AVC
header is designed to announce the size of the following block, if we
pass only one NAL by buffer, this is not very useful either. Both seems
trivial to reconstruct if the HW requires it.
>
> That said, since pixel format essentially specifies the buffer
> contents, having such cases differentiated based on the pixel format
> doesn't sound insane.
As of today, that's what the pixel formats tries to do with these
_NO_SC postfix (the current naming is generally ambiguous though).
>
> Best regards,
> Tomasz
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-22 14:03 ` Nicolas Dufresne
0 siblings, 0 replies; 102+ messages in thread
From: Nicolas Dufresne @ 2018-08-22 14:03 UTC (permalink / raw)
To: linux-arm-kernel
Le mercredi 22 ao?t 2018 ? 22:24 +0900, Tomasz Figa a ?crit :
> On Wed, Aug 22, 2018 at 10:03 PM Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Wed, 2018-08-22 at 18:54 +0900, Tomasz Figa wrote:
> > > On Wed, Aug 22, 2018 at 6:16 PM Maxime Ripard <maxime.ripard@bootlin.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> > > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > > >
> > > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > > ---
> > > > > >
> > > > >
> > > > > [..]
> > > > > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > > --- a/include/uapi/linux/videodev2.h
> > > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> > > > >
> > > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> > > > > should be aware of it. Perhaps we could document this pixel format better as:
> > > > >
> > > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
> > > >
> > > > I'm not sure this is something we want to do at that point. libva
> > > > doesn't give the start code, so this is only going to make the life of
> > > > the sane controllers more difficult. And if you need to have the start
> > > > code and parse it, then you're not so stateless anymore.
> > >
> > > I might not remember correctly, but Rockchip decoder does some slice
> > > parsing on its own (despite not doing any higher level parsing).
> > > Probably that's why it needs those start codes.
> >
> > The VPU found on Allwinner platforms also provides a mechanism to parse
> > the bitstream data via a dedicated interface through the VPU registers.
> > It is used in libvdpau-sunxi but not in our driver, because we don't
> > want to be doing bitstream parsing in the kernel.
> >
> > It would be good to know if this is just a feature of the Rockchip VPU
> > hardware that can be skipped (like on Allwinner) or if it's a hard
> > requirement in its decoding pipeline.
>
> It's a hard requirement for its decoding pipeline, but...
>
> > Also, maybe it only concerns the
> > slice header? It is already part of the slice data (provided by VAAPI)
> > for H.264/H.265 and an offset is provided to the beginning of the coded
> > video data.
>
> Yes, it seems to be only the slice header.
>
> >
> > > I wonder if libva is the best reference here. It's been designed
> > > almost entirely by Intel for Intel video hardware. We want something
> > > that could work with a wide range of devices and avoid something like
> > > a need to create a semi-stateless API few months later. In fact,
> > > hardware from another vendor, we're working with, also does parsing of
> > > slice headers internally. Moreover, we have some weird
> > > kind-of-stateful decoders, which cannot fully deal with bitstream on
> > > its own, e.g. cannot parse formats, cannot handle resolution changes,
> > > need H264 bitstream NALUs split into separate buffers, etc.
> > >
> > > As I suggested some time ago, having the full bitstream in the buffer,
> > > with offsets of particular units included in respective controls,
> > > would be the most scalable thing. If really needed, we could add flags
> > > telling the driver that particular units are present, so one's
> > > implementation of libva could put only raw slice data in the buffers.
> > > But perhaps it's libva which needs some amendment?
> >
> > If the raw bitstream is needed, I think it would make more sense to use
> > the already-existing formats for stateful VPUs along with the controls
> > for stateless ones instead of having the full bitstream in the
> > V4L2_PIX_FMT_*_SLICE formats.
>
> It may indeed make sense to separate this based on pixel format.
> However, how do we tell the client that it needs to provide those
> controls? Current concept was based entirely on pixel format, so I
> guess that would mean creating something like
> V4L2_PIX_FMT_*_NOT_REALLY_SLICE (_PARSED, _STATELESS?). Might be okay,
> though...
>
> >
> > I would also be tempted to say that reconstructing the needed parts of
> > the bitstream in-driver for these half-way VPUs would be a better
> > approach than blurrying the line between how (and what) data should be
> > passed for stateful and stateless VPUs at the API level. Stateless
> > should only cover what's in the slice NAL unit RBSP, which excludes the
> > start code detection bytes. It is no longer parsed data otherwise.
>
> I'm not sure where such decision comes from. In particular, Chromium,
> from which this code originates, includes start codes in
> V4L2_PIX_FMT_H264_SLICE. As I mentioned earlier, we can't design this
> API based only on 1 type of hardware semantics. The stateless API
> should cover any kind of codec that needs user space assistance in
> processing the stream, which in practice would be almost everything
> for which stateful API doesn't work.
Well, be aware that in most of today's use case, Chromium bitstream
handling is suboptimal since the startcode are crafted by Chromium.
Most of today's use case uses ISOMP4 container, which does not hold
start codes but AVC headers.
If we step back a little, the start codes are designed for randomly
accessing bytes. It's a mark made to be able to randomly scan and find
the start of a NAL. In a well framed stream, they are useless. Then AVC
header is designed to announce the size of the following block, if we
pass only one NAL by buffer, this is not very useful either. Both seems
trivial to reconstruct if the HW requires it.
>
> That said, since pixel format essentially specifies the buffer
> contents, having such cases differentiated based on the pixel format
> doesn't sound insane.
As of today, that's what the pixel formats tries to do with these
_NO_SC postfix (the current naming is generally ambiguous though).
>
> Best regards,
> Tomasz
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180822/7791baf8/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-22 13:24 ` Tomasz Figa
@ 2018-08-22 14:30 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-08-22 14:30 UTC (permalink / raw)
To: Tomasz Figa
Cc: Maxime Ripard, Ezequiel Garcia, Pawel Osciak, Hans Verkuil,
Alexandre Courbot, Sakari Ailus, Laurent Pinchart, Chen-Yu Tsai,
Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, Nicolas Dufresne, jenskuske,
linux-sunxi, thomas.petazzoni, groeck
[-- Attachment #1: Type: text/plain, Size: 7161 bytes --]
Hi,
On Wed, 2018-08-22 at 22:24 +0900, Tomasz Figa wrote:
> On Wed, Aug 22, 2018 at 10:03 PM Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Wed, 2018-08-22 at 18:54 +0900, Tomasz Figa wrote:
> > > On Wed, Aug 22, 2018 at 6:16 PM Maxime Ripard <maxime.ripard@bootlin.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> > > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > > >
> > > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > > ---
> > > > > >
> > > > >
> > > > > [..]
> > > > > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > > --- a/include/uapi/linux/videodev2.h
> > > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> > > > >
> > > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> > > > > should be aware of it. Perhaps we could document this pixel format better as:
> > > > >
> > > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
> > > >
> > > > I'm not sure this is something we want to do at that point. libva
> > > > doesn't give the start code, so this is only going to make the life of
> > > > the sane controllers more difficult. And if you need to have the start
> > > > code and parse it, then you're not so stateless anymore.
> > >
> > > I might not remember correctly, but Rockchip decoder does some slice
> > > parsing on its own (despite not doing any higher level parsing).
> > > Probably that's why it needs those start codes.
> >
> > The VPU found on Allwinner platforms also provides a mechanism to parse
> > the bitstream data via a dedicated interface through the VPU registers.
> > It is used in libvdpau-sunxi but not in our driver, because we don't
> > want to be doing bitstream parsing in the kernel.
> >
> > It would be good to know if this is just a feature of the Rockchip VPU
> > hardware that can be skipped (like on Allwinner) or if it's a hard
> > requirement in its decoding pipeline.
>
> It's a hard requirement for its decoding pipeline, but...
>
> > Also, maybe it only concerns the
> > slice header? It is already part of the slice data (provided by VAAPI)
> > for H.264/H.265 and an offset is provided to the beginning of the coded
> > video data.
>
> Yes, it seems to be only the slice header.
Sounds good, then I don't have any problem with that.
> >
> > > I wonder if libva is the best reference here. It's been designed
> > > almost entirely by Intel for Intel video hardware. We want something
> > > that could work with a wide range of devices and avoid something like
> > > a need to create a semi-stateless API few months later. In fact,
> > > hardware from another vendor, we're working with, also does parsing of
> > > slice headers internally. Moreover, we have some weird
> > > kind-of-stateful decoders, which cannot fully deal with bitstream on
> > > its own, e.g. cannot parse formats, cannot handle resolution changes,
> > > need H264 bitstream NALUs split into separate buffers, etc.
> > >
> > > As I suggested some time ago, having the full bitstream in the buffer,
> > > with offsets of particular units included in respective controls,
> > > would be the most scalable thing. If really needed, we could add flags
> > > telling the driver that particular units are present, so one's
> > > implementation of libva could put only raw slice data in the buffers.
> > > But perhaps it's libva which needs some amendment?
> >
> > If the raw bitstream is needed, I think it would make more sense to use
> > the already-existing formats for stateful VPUs along with the controls
> > for stateless ones instead of having the full bitstream in the
> > V4L2_PIX_FMT_*_SLICE formats.
>
> It may indeed make sense to separate this based on pixel format.
> However, how do we tell the client that it needs to provide those
> controls? Current concept was based entirely on pixel format, so I
> guess that would mean creating something like
> V4L2_PIX_FMT_*_NOT_REALLY_SLICE (_PARSED, _STATELESS?). Might be okay,
> though...
How about declaring support for the request API (through the associated
CAPs) and only having the non-_SLICE formats listed in ENUM_FMT?
> > I would also be tempted to say that reconstructing the needed parts of
> > the bitstream in-driver for these half-way VPUs would be a better
> > approach than blurrying the line between how (and what) data should be
> > passed for stateful and stateless VPUs at the API level. Stateless
> > should only cover what's in the slice NAL unit RBSP, which excludes the
> > start code detection bytes. It is no longer parsed data otherwise.
>
> I'm not sure where such decision comes from. In particular, Chromium,
> from which this code originates, includes start codes in
> V4L2_PIX_FMT_H264_SLICE. As I mentioned earlier, we can't design this
> API based only on 1 type of hardware semantics. The stateless API
> should cover any kind of codec that needs user space assistance in
> processing the stream, which in practice would be almost everything
> for which stateful API doesn't work.
Maybe we need to formalize what the stateless API aims to support and
what the formats really entail. I was under the impression that it was
synonymouse with providing parsed bitstream to the kernel. The way I
understand "parsed bitstream" means the slice NALU RBSP as raw data and
the metadata from other NALUs as controls (with the overlap of the slice
header in recent formats, that is in both).
I don't see any other sane boundary that could be conceptually attached
to something like "providing parsed bitstream". Covering all that might
be required for != stateful seems hard to formalize conceptually and to
delimit in general.
> That said, since pixel format essentially specifies the buffer
> contents, having such cases differentiated based on the pixel format
> doesn't sound insane.
Great!
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-22 14:30 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-08-22 14:30 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-08-22 at 22:24 +0900, Tomasz Figa wrote:
> On Wed, Aug 22, 2018 at 10:03 PM Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Wed, 2018-08-22 at 18:54 +0900, Tomasz Figa wrote:
> > > On Wed, Aug 22, 2018 at 6:16 PM Maxime Ripard <maxime.ripard@bootlin.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > On Tue, Aug 21, 2018 at 01:58:38PM -0300, Ezequiel Garcia wrote:
> > > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > > >
> > > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > > ---
> > > > > >
> > > > >
> > > > > [..]
> > > > > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > > --- a/include/uapi/linux/videodev2.h
> > > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */
> > > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */
> > > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */
> > > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> > > > >
> > > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes [1], so the userspace
> > > > > should be aware of it. Perhaps we could document this pixel format better as:
> > > > >
> > > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices with start codes */
> > > >
> > > > I'm not sure this is something we want to do at that point. libva
> > > > doesn't give the start code, so this is only going to make the life of
> > > > the sane controllers more difficult. And if you need to have the start
> > > > code and parse it, then you're not so stateless anymore.
> > >
> > > I might not remember correctly, but Rockchip decoder does some slice
> > > parsing on its own (despite not doing any higher level parsing).
> > > Probably that's why it needs those start codes.
> >
> > The VPU found on Allwinner platforms also provides a mechanism to parse
> > the bitstream data via a dedicated interface through the VPU registers.
> > It is used in libvdpau-sunxi but not in our driver, because we don't
> > want to be doing bitstream parsing in the kernel.
> >
> > It would be good to know if this is just a feature of the Rockchip VPU
> > hardware that can be skipped (like on Allwinner) or if it's a hard
> > requirement in its decoding pipeline.
>
> It's a hard requirement for its decoding pipeline, but...
>
> > Also, maybe it only concerns the
> > slice header? It is already part of the slice data (provided by VAAPI)
> > for H.264/H.265 and an offset is provided to the beginning of the coded
> > video data.
>
> Yes, it seems to be only the slice header.
Sounds good, then I don't have any problem with that.
> >
> > > I wonder if libva is the best reference here. It's been designed
> > > almost entirely by Intel for Intel video hardware. We want something
> > > that could work with a wide range of devices and avoid something like
> > > a need to create a semi-stateless API few months later. In fact,
> > > hardware from another vendor, we're working with, also does parsing of
> > > slice headers internally. Moreover, we have some weird
> > > kind-of-stateful decoders, which cannot fully deal with bitstream on
> > > its own, e.g. cannot parse formats, cannot handle resolution changes,
> > > need H264 bitstream NALUs split into separate buffers, etc.
> > >
> > > As I suggested some time ago, having the full bitstream in the buffer,
> > > with offsets of particular units included in respective controls,
> > > would be the most scalable thing. If really needed, we could add flags
> > > telling the driver that particular units are present, so one's
> > > implementation of libva could put only raw slice data in the buffers.
> > > But perhaps it's libva which needs some amendment?
> >
> > If the raw bitstream is needed, I think it would make more sense to use
> > the already-existing formats for stateful VPUs along with the controls
> > for stateless ones instead of having the full bitstream in the
> > V4L2_PIX_FMT_*_SLICE formats.
>
> It may indeed make sense to separate this based on pixel format.
> However, how do we tell the client that it needs to provide those
> controls? Current concept was based entirely on pixel format, so I
> guess that would mean creating something like
> V4L2_PIX_FMT_*_NOT_REALLY_SLICE (_PARSED, _STATELESS?). Might be okay,
> though...
How about declaring support for the request API (through the associated
CAPs) and only having the non-_SLICE formats listed in ENUM_FMT?
> > I would also be tempted to say that reconstructing the needed parts of
> > the bitstream in-driver for these half-way VPUs would be a better
> > approach than blurrying the line between how (and what) data should be
> > passed for stateful and stateless VPUs at the API level. Stateless
> > should only cover what's in the slice NAL unit RBSP, which excludes the
> > start code detection bytes. It is no longer parsed data otherwise.
>
> I'm not sure where such decision comes from. In particular, Chromium,
> from which this code originates, includes start codes in
> V4L2_PIX_FMT_H264_SLICE. As I mentioned earlier, we can't design this
> API based only on 1 type of hardware semantics. The stateless API
> should cover any kind of codec that needs user space assistance in
> processing the stream, which in practice would be almost everything
> for which stateful API doesn't work.
Maybe we need to formalize what the stateless API aims to support and
what the formats really entail. I was under the impression that it was
synonymouse with providing parsed bitstream to the kernel. The way I
understand "parsed bitstream" means the slice NALU RBSP as raw data and
the metadata from other NALUs as controls (with the overlap of the slice
header in recent formats, that is in both).
I don't see any other sane boundary that could be conceptually attached
to something like "providing parsed bitstream". Covering all that might
be required for != stateful seems hard to formalize conceptually and to
delimit in general.
> That said, since pixel format essentially specifies the buffer
> contents, having such cases differentiated based on the pixel format
> doesn't sound insane.
Great!
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180822/2dc3f39f/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-22 13:38 ` Tomasz Figa
@ 2018-08-22 14:45 ` Paul Kocialkowski
-1 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-08-22 14:45 UTC (permalink / raw)
To: Tomasz Figa
Cc: Nicolas Dufresne, Ezequiel Garcia, Maxime Ripard, Pawel Osciak,
Hans Verkuil, Alexandre Courbot, Sakari Ailus, Laurent Pinchart,
Chen-Yu Tsai, Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, jenskuske, linux-sunxi,
thomas.petazzoni, groeck
[-- Attachment #1: Type: text/plain, Size: 4878 bytes --]
Hi,
On Wed, 2018-08-22 at 22:38 +0900, Tomasz Figa wrote:
> On Wed, Aug 22, 2018 at 10:07 PM Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> > > Le mardi 21 août 2018 à 13:58 -0300, Ezequiel Garcia a écrit :
> > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > >
> > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > ---
> > > > >
> > > >
> > > > [..]
> > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > b/include/uapi/linux/videodev2.h
> > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > --- a/include/uapi/linux/videodev2.h
> > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > > > H264 with start codes */
> > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > > > H264 without start codes */
> > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > > > H264 MVC */
> > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > H264 parsed slices */
> > > >
> > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > > > [1], so the userspace
> > > > should be aware of it. Perhaps we could document this pixel format
> > > > better as:
> > > >
> > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > H264 parsed slices with start codes */
> > > >
> > > > And introduce another pixel format:
> > > >
> > > > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > > > parsed slices without start codes */
> > > >
> > > > For cedrus to use, as it seems it doesn't need start codes.
> > >
> > > I must admit that this RK requirement is a bit weird for slice data.
> > > Though, userspace wise, always adding start-code would be compatible,
> > > as the driver can just offset to remove it.
> >
> > This would mean that the stateless API no longer takes parsed bitstream
> > data but effectively the full bitstream, which defeats the purpose of
> > the _SLICE pixel formats.
> >
>
> Not entirely. One of the purposes of the _SLICE pixel format was to
> specify it in a way that adds a requirement of providing the required
> controls by the client.
I think we need to define what we want the stateless APIs (and these
formats) to precisely reflect conceptually. I've started discussing this
in the Request API and V4L2 capabilities thread.
> > > Another option, because I'm not fan of adding dedicated formats for
> > > this, the RK driver could use data_offset (in mplane v4l2 buffers),
> > > just write a start code there. I like this solution because I would not
> > > be surprise if some drivers requires in fact an HW specific header,
> > > that the driver can generate as needed.
> >
> > I like this idea, because it implies that the driver should deal with
> > the specificities of the hardware, instead of making the blurrying the
> > lines of stateless API for covering these cases.
>
> The spec says
>
> "Offset in bytes to video data in the plane. Drivers must set this
> field when type refers to a capture stream, applications when it
> refers to an output stream."
>
> which would mean that user space would have to know to reserve some
> bytes at the beginning for the driver to add the start code there. (Or
> the driver memmove()ing the data forward when the buffer is queued,
> assuming that there is enough space in the buffer, but it should
> normally be the case.)
>
> Sounds like a pixel format with full bitstream data and some offsets
> to particular parts inside given inside a control might be the most
> flexible and cleanest solution.
I can't help but think that bringing the whole bitstream over to the
kernel with a dedicated pix fmt just for the sake of having 3 start code
bytes is rather overkill anyway.
I believe moving the data around to be the best call for this situation.
Or maybe there's a way to alloc more data *before* the bufer that is
exposed to userspace, so userspace can fill it normally and the driver
can bring-in the necessary heading start code bytes before the buffer?
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-22 14:45 ` Paul Kocialkowski
0 siblings, 0 replies; 102+ messages in thread
From: Paul Kocialkowski @ 2018-08-22 14:45 UTC (permalink / raw)
To: linux-arm-kernel
Hi,
On Wed, 2018-08-22 at 22:38 +0900, Tomasz Figa wrote:
> On Wed, Aug 22, 2018 at 10:07 PM Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> > > Le mardi 21 ao?t 2018 ? 13:58 -0300, Ezequiel Garcia a ?crit :
> > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > >
> > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > ---
> > > > >
> > > >
> > > > [..]
> > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > b/include/uapi/linux/videodev2.h
> > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > --- a/include/uapi/linux/videodev2.h
> > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > > > H264 with start codes */
> > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > > > H264 without start codes */
> > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > > > H264 MVC */
> > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > H264 parsed slices */
> > > >
> > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > > > [1], so the userspace
> > > > should be aware of it. Perhaps we could document this pixel format
> > > > better as:
> > > >
> > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > H264 parsed slices with start codes */
> > > >
> > > > And introduce another pixel format:
> > > >
> > > > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > > > parsed slices without start codes */
> > > >
> > > > For cedrus to use, as it seems it doesn't need start codes.
> > >
> > > I must admit that this RK requirement is a bit weird for slice data.
> > > Though, userspace wise, always adding start-code would be compatible,
> > > as the driver can just offset to remove it.
> >
> > This would mean that the stateless API no longer takes parsed bitstream
> > data but effectively the full bitstream, which defeats the purpose of
> > the _SLICE pixel formats.
> >
>
> Not entirely. One of the purposes of the _SLICE pixel format was to
> specify it in a way that adds a requirement of providing the required
> controls by the client.
I think we need to define what we want the stateless APIs (and these
formats) to precisely reflect conceptually. I've started discussing this
in the Request API and V4L2 capabilities thread.
> > > Another option, because I'm not fan of adding dedicated formats for
> > > this, the RK driver could use data_offset (in mplane v4l2 buffers),
> > > just write a start code there. I like this solution because I would not
> > > be surprise if some drivers requires in fact an HW specific header,
> > > that the driver can generate as needed.
> >
> > I like this idea, because it implies that the driver should deal with
> > the specificities of the hardware, instead of making the blurrying the
> > lines of stateless API for covering these cases.
>
> The spec says
>
> "Offset in bytes to video data in the plane. Drivers must set this
> field when type refers to a capture stream, applications when it
> refers to an output stream."
>
> which would mean that user space would have to know to reserve some
> bytes at the beginning for the driver to add the start code there. (Or
> the driver memmove()ing the data forward when the buffer is queued,
> assuming that there is enough space in the buffer, but it should
> normally be the case.)
>
> Sounds like a pixel format with full bitstream data and some offsets
> to particular parts inside given inside a control might be the most
> flexible and cleanest solution.
I can't help but think that bringing the whole bitstream over to the
kernel with a dedicated pix fmt just for the sake of having 3 start code
bytes is rather overkill anyway.
I believe moving the data around to be the best call for this situation.
Or maybe there's a way to alloc more data *before* the bufer that is
exposed to userspace, so userspace can fill it normally and the driver
can bring-in the necessary heading start code bytes before the buffer?
Cheers,
Paul
--
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180822/c635d458/attachment.sig>
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-22 14:45 ` Paul Kocialkowski
@ 2018-08-28 8:11 ` Tomasz Figa
-1 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-08-28 8:11 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: Nicolas Dufresne, Ezequiel Garcia, Maxime Ripard, Pawel Osciak,
Hans Verkuil, Alexandre Courbot, Sakari Ailus, Laurent Pinchart,
Chen-Yu Tsai, Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, jenskuske, linux-sunxi,
thomas.petazzoni, groeck
On Wed, Aug 22, 2018 at 11:45 PM Paul Kocialkowski
<paul.kocialkowski@bootlin.com> wrote:
>
> Hi,
>
> On Wed, 2018-08-22 at 22:38 +0900, Tomasz Figa wrote:
> > On Wed, Aug 22, 2018 at 10:07 PM Paul Kocialkowski
> > <paul.kocialkowski@bootlin.com> wrote:
> > >
> > > Hi,
> > >
> > > On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> > > > Le mardi 21 août 2018 à 13:58 -0300, Ezequiel Garcia a écrit :
> > > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > > >
> > > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > > ---
> > > > > >
> > > > >
> > > > > [..]
> > > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > > b/include/uapi/linux/videodev2.h
> > > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > > --- a/include/uapi/linux/videodev2.h
> > > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > > > > H264 with start codes */
> > > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > > > > H264 without start codes */
> > > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > > > > H264 MVC */
> > > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > > H264 parsed slices */
> > > > >
> > > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > > > > [1], so the userspace
> > > > > should be aware of it. Perhaps we could document this pixel format
> > > > > better as:
> > > > >
> > > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > H264 parsed slices with start codes */
> > > > >
> > > > > And introduce another pixel format:
> > > > >
> > > > > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > > > > parsed slices without start codes */
> > > > >
> > > > > For cedrus to use, as it seems it doesn't need start codes.
> > > >
> > > > I must admit that this RK requirement is a bit weird for slice data.
> > > > Though, userspace wise, always adding start-code would be compatible,
> > > > as the driver can just offset to remove it.
> > >
> > > This would mean that the stateless API no longer takes parsed bitstream
> > > data but effectively the full bitstream, which defeats the purpose of
> > > the _SLICE pixel formats.
> > >
> >
> > Not entirely. One of the purposes of the _SLICE pixel format was to
> > specify it in a way that adds a requirement of providing the required
> > controls by the client.
>
> I think we need to define what we want the stateless APIs (and these
> formats) to precisely reflect conceptually. I've started discussing this
> in the Request API and V4L2 capabilities thread.
>
> > > > Another option, because I'm not fan of adding dedicated formats for
> > > > this, the RK driver could use data_offset (in mplane v4l2 buffers),
> > > > just write a start code there. I like this solution because I would not
> > > > be surprise if some drivers requires in fact an HW specific header,
> > > > that the driver can generate as needed.
> > >
> > > I like this idea, because it implies that the driver should deal with
> > > the specificities of the hardware, instead of making the blurrying the
> > > lines of stateless API for covering these cases.
> >
> > The spec says
> >
> > "Offset in bytes to video data in the plane. Drivers must set this
> > field when type refers to a capture stream, applications when it
> > refers to an output stream."
> >
> > which would mean that user space would have to know to reserve some
> > bytes at the beginning for the driver to add the start code there. (Or
> > the driver memmove()ing the data forward when the buffer is queued,
> > assuming that there is enough space in the buffer, but it should
> > normally be the case.)
> >
> > Sounds like a pixel format with full bitstream data and some offsets
> > to particular parts inside given inside a control might be the most
> > flexible and cleanest solution.
>
> I can't help but think that bringing the whole bitstream over to the
> kernel with a dedicated pix fmt just for the sake of having 3 start code
> bytes is rather overkill anyway.
>
> I believe moving the data around to be the best call for this situation.
> Or maybe there's a way to alloc more data *before* the bufer that is
> exposed to userspace, so userspace can fill it normally and the driver
> can bring-in the necessary heading start code bytes before the buffer?
After thinking this over for some time, I believe it boils down to
whether we can have an in-kernel library for turning H264 (and other
codec) header structs back into a bitstream, if we end up with more
than one driver need to do it. If that's fine, I think we're okay with
having just the parsed pixel format around.
Note that I didn't think about this with the Rockchip driver in mind,
since it indeed only needs few bytes.
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-08-28 8:11 ` Tomasz Figa
0 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-08-28 8:11 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Aug 22, 2018 at 11:45 PM Paul Kocialkowski
<paul.kocialkowski@bootlin.com> wrote:
>
> Hi,
>
> On Wed, 2018-08-22 at 22:38 +0900, Tomasz Figa wrote:
> > On Wed, Aug 22, 2018 at 10:07 PM Paul Kocialkowski
> > <paul.kocialkowski@bootlin.com> wrote:
> > >
> > > Hi,
> > >
> > > On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> > > > Le mardi 21 ao?t 2018 ? 13:58 -0300, Ezequiel Garcia a ?crit :
> > > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > > >
> > > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > > ---
> > > > > >
> > > > >
> > > > > [..]
> > > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > > b/include/uapi/linux/videodev2.h
> > > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > > --- a/include/uapi/linux/videodev2.h
> > > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > > > > H264 with start codes */
> > > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > > > > H264 without start codes */
> > > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > > > > H264 MVC */
> > > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > > H264 parsed slices */
> > > > >
> > > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > > > > [1], so the userspace
> > > > > should be aware of it. Perhaps we could document this pixel format
> > > > > better as:
> > > > >
> > > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > H264 parsed slices with start codes */
> > > > >
> > > > > And introduce another pixel format:
> > > > >
> > > > > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > > > > parsed slices without start codes */
> > > > >
> > > > > For cedrus to use, as it seems it doesn't need start codes.
> > > >
> > > > I must admit that this RK requirement is a bit weird for slice data.
> > > > Though, userspace wise, always adding start-code would be compatible,
> > > > as the driver can just offset to remove it.
> > >
> > > This would mean that the stateless API no longer takes parsed bitstream
> > > data but effectively the full bitstream, which defeats the purpose of
> > > the _SLICE pixel formats.
> > >
> >
> > Not entirely. One of the purposes of the _SLICE pixel format was to
> > specify it in a way that adds a requirement of providing the required
> > controls by the client.
>
> I think we need to define what we want the stateless APIs (and these
> formats) to precisely reflect conceptually. I've started discussing this
> in the Request API and V4L2 capabilities thread.
>
> > > > Another option, because I'm not fan of adding dedicated formats for
> > > > this, the RK driver could use data_offset (in mplane v4l2 buffers),
> > > > just write a start code there. I like this solution because I would not
> > > > be surprise if some drivers requires in fact an HW specific header,
> > > > that the driver can generate as needed.
> > >
> > > I like this idea, because it implies that the driver should deal with
> > > the specificities of the hardware, instead of making the blurrying the
> > > lines of stateless API for covering these cases.
> >
> > The spec says
> >
> > "Offset in bytes to video data in the plane. Drivers must set this
> > field when type refers to a capture stream, applications when it
> > refers to an output stream."
> >
> > which would mean that user space would have to know to reserve some
> > bytes at the beginning for the driver to add the start code there. (Or
> > the driver memmove()ing the data forward when the buffer is queued,
> > assuming that there is enough space in the buffer, but it should
> > normally be the case.)
> >
> > Sounds like a pixel format with full bitstream data and some offsets
> > to particular parts inside given inside a control might be the most
> > flexible and cleanest solution.
>
> I can't help but think that bringing the whole bitstream over to the
> kernel with a dedicated pix fmt just for the sake of having 3 start code
> bytes is rather overkill anyway.
>
> I believe moving the data around to be the best call for this situation.
> Or maybe there's a way to alloc more data *before* the bufer that is
> exposed to userspace, so userspace can fill it normally and the driver
> can bring-in the necessary heading start code bytes before the buffer?
After thinking this over for some time, I believe it boils down to
whether we can have an in-kernel library for turning H264 (and other
codec) header structs back into a bitstream, if we end up with more
than one driver need to do it. If that's fine, I think we're okay with
having just the parsed pixel format around.
Note that I didn't think about this with the Rockchip driver in mind,
since it indeed only needs few bytes.
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
* Re: [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
2018-08-28 8:11 ` Tomasz Figa
@ 2018-09-07 7:54 ` Tomasz Figa
-1 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-09-07 7:54 UTC (permalink / raw)
To: Paul Kocialkowski
Cc: Nicolas Dufresne, Ezequiel Garcia, Maxime Ripard, Pawel Osciak,
Hans Verkuil, Alexandre Courbot, Sakari Ailus, Laurent Pinchart,
Chen-Yu Tsai, Linux Kernel Mailing List,
list@263.net:IOMMU DRIVERS
<iommu@lists.linux-foundation.org>,
Joerg Roedel <joro@8bytes.org>,,
Linux Media Mailing List, jenskuske, linux-sunxi,
thomas.petazzoni, groeck
On Tue, Aug 28, 2018 at 5:11 PM Tomasz Figa <tfiga@chromium.org> wrote:
>
> On Wed, Aug 22, 2018 at 11:45 PM Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Wed, 2018-08-22 at 22:38 +0900, Tomasz Figa wrote:
> > > On Wed, Aug 22, 2018 at 10:07 PM Paul Kocialkowski
> > > <paul.kocialkowski@bootlin.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> > > > > Le mardi 21 août 2018 à 13:58 -0300, Ezequiel Garcia a écrit :
> > > > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > > > >
> > > > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > > > ---
> > > > > > >
> > > > > >
> > > > > > [..]
> > > > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > > > b/include/uapi/linux/videodev2.h
> > > > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > > > --- a/include/uapi/linux/videodev2.h
> > > > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > > > > > H264 with start codes */
> > > > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > > > > > H264 without start codes */
> > > > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > > > > > H264 MVC */
> > > > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > > > H264 parsed slices */
> > > > > >
> > > > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > > > > > [1], so the userspace
> > > > > > should be aware of it. Perhaps we could document this pixel format
> > > > > > better as:
> > > > > >
> > > > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > > H264 parsed slices with start codes */
> > > > > >
> > > > > > And introduce another pixel format:
> > > > > >
> > > > > > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > > > > > parsed slices without start codes */
> > > > > >
> > > > > > For cedrus to use, as it seems it doesn't need start codes.
> > > > >
> > > > > I must admit that this RK requirement is a bit weird for slice data.
> > > > > Though, userspace wise, always adding start-code would be compatible,
> > > > > as the driver can just offset to remove it.
> > > >
> > > > This would mean that the stateless API no longer takes parsed bitstream
> > > > data but effectively the full bitstream, which defeats the purpose of
> > > > the _SLICE pixel formats.
> > > >
> > >
> > > Not entirely. One of the purposes of the _SLICE pixel format was to
> > > specify it in a way that adds a requirement of providing the required
> > > controls by the client.
> >
> > I think we need to define what we want the stateless APIs (and these
> > formats) to precisely reflect conceptually. I've started discussing this
> > in the Request API and V4L2 capabilities thread.
> >
> > > > > Another option, because I'm not fan of adding dedicated formats for
> > > > > this, the RK driver could use data_offset (in mplane v4l2 buffers),
> > > > > just write a start code there. I like this solution because I would not
> > > > > be surprise if some drivers requires in fact an HW specific header,
> > > > > that the driver can generate as needed.
> > > >
> > > > I like this idea, because it implies that the driver should deal with
> > > > the specificities of the hardware, instead of making the blurrying the
> > > > lines of stateless API for covering these cases.
> > >
> > > The spec says
> > >
> > > "Offset in bytes to video data in the plane. Drivers must set this
> > > field when type refers to a capture stream, applications when it
> > > refers to an output stream."
> > >
> > > which would mean that user space would have to know to reserve some
> > > bytes at the beginning for the driver to add the start code there. (Or
> > > the driver memmove()ing the data forward when the buffer is queued,
> > > assuming that there is enough space in the buffer, but it should
> > > normally be the case.)
> > >
> > > Sounds like a pixel format with full bitstream data and some offsets
> > > to particular parts inside given inside a control might be the most
> > > flexible and cleanest solution.
> >
> > I can't help but think that bringing the whole bitstream over to the
> > kernel with a dedicated pix fmt just for the sake of having 3 start code
> > bytes is rather overkill anyway.
> >
> > I believe moving the data around to be the best call for this situation.
> > Or maybe there's a way to alloc more data *before* the bufer that is
> > exposed to userspace, so userspace can fill it normally and the driver
> > can bring-in the necessary heading start code bytes before the buffer?
>
> After thinking this over for some time, I believe it boils down to
> whether we can have an in-kernel library for turning H264 (and other
> codec) header structs back into a bitstream, if we end up with more
> than one driver need to do it. If that's fine, I think we're okay with
> having just the parsed pixel format around.
>
> Note that I didn't think about this with the Rockchip driver in mind,
> since it indeed only needs few bytes.
By the way, Alex posted an RFC with stateless codec interface documentation:
https://patchwork.kernel.org/patch/10583233/
I think we should move any discussion there, to have everything in one place.
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
* [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls.
@ 2018-09-07 7:54 ` Tomasz Figa
0 siblings, 0 replies; 102+ messages in thread
From: Tomasz Figa @ 2018-09-07 7:54 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Aug 28, 2018 at 5:11 PM Tomasz Figa <tfiga@chromium.org> wrote:
>
> On Wed, Aug 22, 2018 at 11:45 PM Paul Kocialkowski
> <paul.kocialkowski@bootlin.com> wrote:
> >
> > Hi,
> >
> > On Wed, 2018-08-22 at 22:38 +0900, Tomasz Figa wrote:
> > > On Wed, Aug 22, 2018 at 10:07 PM Paul Kocialkowski
> > > <paul.kocialkowski@bootlin.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > On Tue, 2018-08-21 at 13:07 -0400, Nicolas Dufresne wrote:
> > > > > Le mardi 21 ao?t 2018 ? 13:58 -0300, Ezequiel Garcia a ?crit :
> > > > > > On Wed, 2018-06-13 at 16:07 +0200, Maxime Ripard wrote:
> > > > > > > From: Pawel Osciak <posciak@chromium.org>
> > > > > > >
> > > > > > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> > > > > > > Reviewed-by: Wu-cheng Li <wuchengli@chromium.org>
> > > > > > > Tested-by: Tomasz Figa <tfiga@chromium.org>
> > > > > > > [rebase44(groeck): include linux/types.h in v4l2-controls.h]
> > > > > > > Signed-off-by: Guenter Roeck <groeck@chromium.org>
> > > > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com>
> > > > > > > ---
> > > > > > >
> > > > > >
> > > > > > [..]
> > > > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > > > b/include/uapi/linux/videodev2.h
> > > > > > > index 242a6bfa1440..4b4a1b25a0db 100644
> > > > > > > --- a/include/uapi/linux/videodev2.h
> > > > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > > > @@ -626,6 +626,7 @@ struct v4l2_pix_format {
> > > > > > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /*
> > > > > > > H264 with start codes */
> > > > > > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /*
> > > > > > > H264 without start codes */
> > > > > > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /*
> > > > > > > H264 MVC */
> > > > > > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > > > H264 parsed slices */
> > > > > >
> > > > > > As pointed out by Tomasz, the Rockchip VPU driver expects start codes
> > > > > > [1], so the userspace
> > > > > > should be aware of it. Perhaps we could document this pixel format
> > > > > > better as:
> > > > > >
> > > > > > #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /*
> > > > > > H264 parsed slices with start codes */
> > > > > >
> > > > > > And introduce another pixel format:
> > > > > >
> > > > > > #define V4L2_PIX_FMT_H264_SLICE_NO_SC v4l2_fourcc(TODO) /* H264
> > > > > > parsed slices without start codes */
> > > > > >
> > > > > > For cedrus to use, as it seems it doesn't need start codes.
> > > > >
> > > > > I must admit that this RK requirement is a bit weird for slice data.
> > > > > Though, userspace wise, always adding start-code would be compatible,
> > > > > as the driver can just offset to remove it.
> > > >
> > > > This would mean that the stateless API no longer takes parsed bitstream
> > > > data but effectively the full bitstream, which defeats the purpose of
> > > > the _SLICE pixel formats.
> > > >
> > >
> > > Not entirely. One of the purposes of the _SLICE pixel format was to
> > > specify it in a way that adds a requirement of providing the required
> > > controls by the client.
> >
> > I think we need to define what we want the stateless APIs (and these
> > formats) to precisely reflect conceptually. I've started discussing this
> > in the Request API and V4L2 capabilities thread.
> >
> > > > > Another option, because I'm not fan of adding dedicated formats for
> > > > > this, the RK driver could use data_offset (in mplane v4l2 buffers),
> > > > > just write a start code there. I like this solution because I would not
> > > > > be surprise if some drivers requires in fact an HW specific header,
> > > > > that the driver can generate as needed.
> > > >
> > > > I like this idea, because it implies that the driver should deal with
> > > > the specificities of the hardware, instead of making the blurrying the
> > > > lines of stateless API for covering these cases.
> > >
> > > The spec says
> > >
> > > "Offset in bytes to video data in the plane. Drivers must set this
> > > field when type refers to a capture stream, applications when it
> > > refers to an output stream."
> > >
> > > which would mean that user space would have to know to reserve some
> > > bytes at the beginning for the driver to add the start code there. (Or
> > > the driver memmove()ing the data forward when the buffer is queued,
> > > assuming that there is enough space in the buffer, but it should
> > > normally be the case.)
> > >
> > > Sounds like a pixel format with full bitstream data and some offsets
> > > to particular parts inside given inside a control might be the most
> > > flexible and cleanest solution.
> >
> > I can't help but think that bringing the whole bitstream over to the
> > kernel with a dedicated pix fmt just for the sake of having 3 start code
> > bytes is rather overkill anyway.
> >
> > I believe moving the data around to be the best call for this situation.
> > Or maybe there's a way to alloc more data *before* the bufer that is
> > exposed to userspace, so userspace can fill it normally and the driver
> > can bring-in the necessary heading start code bytes before the buffer?
>
> After thinking this over for some time, I believe it boils down to
> whether we can have an in-kernel library for turning H264 (and other
> codec) header structs back into a bitstream, if we end up with more
> than one driver need to do it. If that's fine, I think we're okay with
> having just the parsed pixel format around.
>
> Note that I didn't think about this with the Rockchip driver in mind,
> since it indeed only needs few bytes.
By the way, Alex posted an RFC with stateless codec interface documentation:
https://patchwork.kernel.org/patch/10583233/
I think we should move any discussion there, to have everything in one place.
Best regards,
Tomasz
^ permalink raw reply [flat|nested] 102+ messages in thread
end of thread, other threads:[~2018-09-07 7:54 UTC | newest]
Thread overview: 102+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-13 14:07 [PATCH 0/9] media: cedrus: Add H264 decoding support Maxime Ripard
2018-06-13 14:07 ` Maxime Ripard
2018-06-13 14:07 ` [PATCH 1/9] CHROMIUM: v4l: Add H264 low-level decoder API compound controls Maxime Ripard
2018-06-13 14:07 ` Maxime Ripard
2018-06-15 11:59 ` Hans Verkuil
2018-06-15 11:59 ` Hans Verkuil
2018-06-15 13:01 ` Guenter Roeck
2018-06-15 13:01 ` Guenter Roeck
2018-07-12 16:38 ` Maxime Ripard
2018-07-12 16:38 ` Maxime Ripard
2018-07-12 16:47 ` Andrew Lunn
2018-07-12 16:47 ` Andrew Lunn
2018-06-21 8:58 ` Paul Kocialkowski
2018-06-21 8:58 ` Paul Kocialkowski
2018-08-21 16:58 ` Ezequiel Garcia
2018-08-21 16:58 ` Ezequiel Garcia
2018-08-21 17:07 ` Nicolas Dufresne
2018-08-21 17:07 ` Nicolas Dufresne
2018-08-22 13:07 ` Paul Kocialkowski
2018-08-22 13:07 ` Paul Kocialkowski
2018-08-22 13:38 ` Tomasz Figa
2018-08-22 13:38 ` Tomasz Figa
2018-08-22 13:52 ` Nicolas Dufresne
2018-08-22 13:52 ` Nicolas Dufresne
2018-08-22 14:45 ` Paul Kocialkowski
2018-08-22 14:45 ` Paul Kocialkowski
2018-08-28 8:11 ` Tomasz Figa
2018-08-28 8:11 ` Tomasz Figa
2018-09-07 7:54 ` Tomasz Figa
2018-09-07 7:54 ` Tomasz Figa
2018-08-22 9:15 ` Maxime Ripard
2018-08-22 9:15 ` Maxime Ripard
2018-08-22 9:54 ` Tomasz Figa
2018-08-22 9:54 ` Tomasz Figa
2018-08-22 13:03 ` Paul Kocialkowski
2018-08-22 13:03 ` Paul Kocialkowski
2018-08-22 13:24 ` Tomasz Figa
2018-08-22 13:24 ` Tomasz Figa
2018-08-22 14:03 ` Nicolas Dufresne
2018-08-22 14:03 ` Nicolas Dufresne
2018-08-22 14:30 ` Paul Kocialkowski
2018-08-22 14:30 ` Paul Kocialkowski
2018-06-13 14:07 ` [PATCH 2/9] media: cedrus: Add wrappers around container_of for our buffers Maxime Ripard
2018-06-13 14:07 ` Maxime Ripard
2018-06-21 9:03 ` Paul Kocialkowski
2018-06-21 9:03 ` Paul Kocialkowski
2018-06-13 14:07 ` [PATCH 3/9] media: cedrus: Add a macro to check for the validity of a control Maxime Ripard
2018-06-13 14:07 ` Maxime Ripard
2018-06-21 9:13 ` Paul Kocialkowski
2018-06-21 9:13 ` Paul Kocialkowski
2018-06-13 14:07 ` [PATCH 4/9] media: cedrus: make engine type more generic Maxime Ripard
2018-06-13 14:07 ` Maxime Ripard
2018-06-21 9:33 ` Paul Kocialkowski
2018-06-21 9:33 ` Paul Kocialkowski
2018-06-13 14:07 ` [PATCH 5/9] media: cedrus: Remove MPEG1 support Maxime Ripard
2018-06-13 14:07 ` Maxime Ripard
2018-06-13 14:07 ` [PATCH 6/9] media: cedrus: Add ops structure Maxime Ripard
2018-06-13 14:07 ` Maxime Ripard
2018-06-21 9:49 ` Paul Kocialkowski
2018-06-21 9:49 ` Paul Kocialkowski
2018-06-25 13:29 ` Maxime Ripard
2018-06-25 13:29 ` Maxime Ripard
2018-06-25 13:48 ` Paul Kocialkowski
2018-06-25 13:48 ` Paul Kocialkowski
2018-06-13 14:07 ` [PATCH 7/9] media: cedrus: Move IRQ maintainance to cedrus_dec_ops Maxime Ripard
2018-06-13 14:07 ` Maxime Ripard
2018-06-21 15:35 ` Paul Kocialkowski
2018-06-21 15:35 ` Paul Kocialkowski
2018-06-25 14:18 ` Paul Kocialkowski
2018-06-25 14:18 ` Paul Kocialkowski
2018-06-25 16:15 ` Maxime Ripard
2018-06-25 16:15 ` Maxime Ripard
2018-06-25 15:38 ` Paul Kocialkowski
2018-06-25 15:38 ` Paul Kocialkowski
2018-06-25 15:49 ` Paul Kocialkowski
2018-06-25 15:49 ` Paul Kocialkowski
2018-06-25 19:01 ` Maxime Ripard
2018-06-25 19:01 ` Maxime Ripard
2018-06-27 17:58 ` Maxime Ripard
2018-06-27 17:58 ` Maxime Ripard
2018-06-13 14:07 ` [PATCH 8/9] media: cedrus: Add start and stop decoder operations Maxime Ripard
2018-06-13 14:07 ` Maxime Ripard
2018-06-21 15:38 ` Paul Kocialkowski
2018-06-21 15:38 ` Paul Kocialkowski
2018-06-25 13:32 ` Maxime Ripard
2018-06-25 13:32 ` Maxime Ripard
2018-06-25 13:42 ` Paul Kocialkowski
2018-06-25 13:42 ` Paul Kocialkowski
2018-06-13 14:07 ` [PATCH 9/9] media: cedrus: Add H264 decoding support Maxime Ripard
2018-06-13 14:07 ` Maxime Ripard
2018-07-27 13:56 ` Paul Kocialkowski
2018-07-27 13:56 ` Paul Kocialkowski
2018-07-27 14:01 ` Chen-Yu Tsai
2018-07-27 14:01 ` Chen-Yu Tsai
2018-07-27 14:03 ` Paul Kocialkowski
2018-07-27 14:03 ` Paul Kocialkowski
2018-07-30 12:54 ` Paul Kocialkowski
2018-07-30 12:54 ` Paul Kocialkowski
2018-06-14 13:00 ` [PATCH 0/9] " Tomasz Figa
2018-06-14 13:00 ` Tomasz Figa
2018-06-14 16:37 ` Maxime Ripard
2018-06-14 16:37 ` Maxime Ripard
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.