[PATCH 0/3] media: mediatek: vcodec: Add driver to support 10bit

[PATCH 0/3] media: mediatek: vcodec: Add driver to support 10bit

[Yunfei Dong]

Define two capture formats V4L2_PIX_FMT_MT2110R and
V4L2_PIX_FMT_MT2110T to support 10bit in mt8195, mt8199 and more.
Getting the size of each plane again when userspace set 10bit
syntax to driver.

V4L2_PIX_FMT_MT2110R is used for H264, and V4L2_PIX_FMT_MT2110T
is used for AV1/VP9/HEVC.

patch 1 Add driver to support 10bit
patch 2 Add capture format V4L2_PIX_FMT_MT2110T to support 10bit tile mode
patch 3 Add capture format V4L2_PIX_FMT_MT2110R to support 10bit raster mode
---
- send the first version v1:
- Run 10bit VP9/AV1 fluster test pass.
- Will return error when the 10bit parameter no correctly in function mtk_vdec_s_ctrl.
---

Reference series:
[1]: this series depends on v6 which is send by Yunfei Dong.
     message-id: 20230704131349.8354-1-yunfei.dong@mediatek.com

Mingjia Zhang (3):
  media: mediatek: vcodec: Add capture format to support 10bit tile mode
  media: mediatek: vcodec: Add capture format to support 10bit raster
    mode
  media: mediatek: vcodec: Add driver to support 10bit

 .../media/v4l/pixfmt-reserved.rst             |  15 ++
 .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
 .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
 .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140 +++++++++++++++++-
 drivers/media/v4l2-core/v4l2-common.c         |   4 +
 drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
 include/uapi/linux/videodev2.h                |   2 +
 7 files changed, 186 insertions(+), 4 deletions(-)

-- 
2.18.0

[PATCH 1/3] media: mediatek: vcodec: Add capture format to support 10bit tile mode

[Yunfei Dong]

From: Mingjia Zhang <mingjia.zhang@mediatek.com>

Define one uncompressed capture format V4L2_PIX_FMT_MT2110T in order to
support 10bit for AV1/VP9/HEVC in mt8195.

Signed-off-by: Mingjia Zhang <mingjia.zhang@mediatek.com>
Co-developed-by: Yunfei Dong <yunfei.dong@mediatek.com>
Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 Documentation/userspace-api/media/v4l/pixfmt-reserved.rst | 8 ++++++++
 drivers/media/v4l2-core/v4l2-common.c                     | 2 ++
 drivers/media/v4l2-core/v4l2-ioctl.c                      | 1 +
 include/uapi/linux/videodev2.h                            | 1 +
 4 files changed, 12 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
index 58f6ae25b2e7..b16a7257580c 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
@@ -275,6 +275,14 @@ please make a proposal on the linux-media mailing list.
 
         Decoder's implementation can be found here,
         `aspeed_codec <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
+    * .. _V4L2-PIX-FMT-MT2110T:
+
+      - ``V4L2_PIX_FMT_MT2110T``
+      - 'MT2110T'
+      - Two-planar 10-Bit tile mode YVU420 format used by Mediatek MT8195, MT8188
+        and more. This format have similitude with ``V4L2_PIX_FMT_MM21``.
+        It remains an opaque intermediate format and it is used for VP9, AV1
+        and HEVC.
 .. raw:: latex
 
     \normalsize
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index bee1535b04d3..869fc09a210b 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -262,6 +262,8 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
 		{ .format = V4L2_PIX_FMT_VYUY,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
 		{ .format = V4L2_PIX_FMT_Y212,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 4, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
 		{ .format = V4L2_PIX_FMT_YUV48_12, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 6, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_MT2110T, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div = { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 2,
+		  .block_w = { 16, 8, 0, 0 }, .block_h = { 32, 16, 0, 0 }},
 
 		/* YUV planar formats */
 		{ .format = V4L2_PIX_FMT_NV12,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 2 },
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 01ba27f2ef87..f465c0e3d6e3 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1508,6 +1508,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_QC10C:	descr = "QCOM Compressed 10-bit Format"; break;
 		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
 		case V4L2_PIX_FMT_AV1_FRAME:	descr = "AV1 Frame"; break;
+		case V4L2_PIX_FMT_MT2110T:	descr = "Mediatek 10bit Tile Mode"; break;
 		default:
 			if (fmt->description[0])
 				return;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 3af6a82d0cad..8c7d71afbdc7 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -796,6 +796,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /* Depth data 16-bit */
 #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /* Mediatek compressed block mode  */
 #define V4L2_PIX_FMT_MM21     v4l2_fourcc('M', 'M', '2', '1') /* Mediatek 8-bit block mode, two non-contiguous planes */
+#define V4L2_PIX_FMT_MT2110T  v4l2_fourcc('M', 'T', '2', 'T') /* Mediatek 10-bit block tile mode */
 #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
 #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
 #define V4L2_PIX_FMT_HI240    v4l2_fourcc('H', 'I', '2', '4') /* BTTV 8-bit dithered RGB */
-- 
2.18.0

[PATCH 2/3] media: mediatek: vcodec: Add capture format to support 10bit raster mode

[Yunfei Dong]

From: Mingjia Zhang <mingjia.zhang@mediatek.com>

Define one uncompressed capture format V4L2_PIX_FMT_MT2110R in order to
support 10bit for H264 in mt8195.

Signed-off-by: Mingjia Zhang <mingjia.zhang@mediatek.com>
Co-developed-by: Yunfei Dong <yunfei.dong@mediatek.com>
Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 Documentation/userspace-api/media/v4l/pixfmt-reserved.rst | 7 +++++++
 drivers/media/v4l2-core/v4l2-common.c                     | 2 ++
 drivers/media/v4l2-core/v4l2-ioctl.c                      | 1 +
 include/uapi/linux/videodev2.h                            | 1 +
 4 files changed, 11 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
index b16a7257580c..175dc790ae17 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
@@ -283,6 +283,13 @@ please make a proposal on the linux-media mailing list.
         and more. This format have similitude with ``V4L2_PIX_FMT_MM21``.
         It remains an opaque intermediate format and it is used for VP9, AV1
         and HEVC.
+    * .. _V4L2-PIX-FMT-MT2110R:
+
+      - ``V4L2_PIX_FMT_MT2110R``
+      - 'MT2110R'
+      - Two-planar 10-Bit raster mode YVU420 format used by Mediatek MT8195, MT8188
+        and more. This format have similitude with ``V4L2_PIX_FMT_MM21``.
+        It remains an opaque intermediate format and it is used for AVC.
 .. raw:: latex
 
     \normalsize
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index 869fc09a210b..3a4b15a98e02 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -264,6 +264,8 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
 		{ .format = V4L2_PIX_FMT_YUV48_12, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 6, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
 		{ .format = V4L2_PIX_FMT_MT2110T, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div = { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 2,
 		  .block_w = { 16, 8, 0, 0 }, .block_h = { 32, 16, 0, 0 }},
+		{ .format = V4L2_PIX_FMT_MT2110R, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div = { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 2,
+		  .block_w = { 16, 8, 0, 0 }, .block_h = { 32, 16, 0, 0 }},
 
 		/* YUV planar formats */
 		{ .format = V4L2_PIX_FMT_NV12,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 2 },
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index f465c0e3d6e3..f4d9d6279094 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1509,6 +1509,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
 		case V4L2_PIX_FMT_AV1_FRAME:	descr = "AV1 Frame"; break;
 		case V4L2_PIX_FMT_MT2110T:	descr = "Mediatek 10bit Tile Mode"; break;
+		case V4L2_PIX_FMT_MT2110R:	descr = "Mediatek 10bit Raster Mode"; break;
 		default:
 			if (fmt->description[0])
 				return;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 8c7d71afbdc7..78260e5d9985 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -797,6 +797,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /* Mediatek compressed block mode  */
 #define V4L2_PIX_FMT_MM21     v4l2_fourcc('M', 'M', '2', '1') /* Mediatek 8-bit block mode, two non-contiguous planes */
 #define V4L2_PIX_FMT_MT2110T  v4l2_fourcc('M', 'T', '2', 'T') /* Mediatek 10-bit block tile mode */
+#define V4L2_PIX_FMT_MT2110R  v4l2_fourcc('M', 'T', '2', 'R') /* Mediatek 10-bit block raster mode */
 #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
 #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
 #define V4L2_PIX_FMT_HI240    v4l2_fourcc('H', 'I', '2', '4') /* BTTV 8-bit dithered RGB */
-- 
2.18.0

[PATCH 3/3] media: mediatek: vcodec: Add driver to support 10bit

[Yunfei Dong]

From: Mingjia Zhang <mingjia.zhang@mediatek.com>

Adding to support capture formats V4L2_PIX_FMT_MT2110T and
V4L2_PIX_FMT_MT2110R for 10bit playback. Need to get the size
of each plane again when user space setting syntax to get 10bit
information.

V4L2_PIX_FMT_MT2110T for AV1/VP9/HEVC.
V4L2_PIX_FMT_MT2110R for H264.

Signed-off-by: Mingjia Zhang <mingjia.zhang@mediatek.com>
Co-developed-by: Yunfei Dong <yunfei.dong@mediatek.com>
Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
 .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
 .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140 +++++++++++++++++-
 3 files changed, 163 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
index 5acb7dff18f2..91ed576d6821 100644
--- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
@@ -37,7 +37,9 @@ static bool mtk_vdec_get_cap_fmt(struct mtk_vcodec_dec_ctx *ctx, int format_inde
 {
 	const struct mtk_vcodec_dec_pdata *dec_pdata = ctx->dev->vdec_pdata;
 	const struct mtk_video_fmt *fmt;
+	struct mtk_q_data *q_data;
 	int num_frame_count = 0, i;
+	bool ret = false;
 
 	fmt = &dec_pdata->vdec_formats[format_index];
 	for (i = 0; i < *dec_pdata->num_formats; i++) {
@@ -47,10 +49,26 @@ static bool mtk_vdec_get_cap_fmt(struct mtk_vcodec_dec_ctx *ctx, int format_inde
 		num_frame_count++;
 	}
 
-	if (num_frame_count == 1 || fmt->fourcc == V4L2_PIX_FMT_MM21)
+	if (num_frame_count == 1 || (!ctx->is_10bit_bitstream && fmt->fourcc == V4L2_PIX_FMT_MM21))
 		return true;
 
-	return false;
+	q_data = &ctx->q_data[MTK_Q_DATA_SRC];
+	switch (q_data->fmt->fourcc) {
+	case V4L2_PIX_FMT_H264_SLICE:
+		if (ctx->is_10bit_bitstream && fmt->fourcc == V4L2_PIX_FMT_MT2110R)
+			ret = true;
+		break;
+	case V4L2_PIX_FMT_VP9_FRAME:
+	case V4L2_PIX_FMT_AV1_FRAME:
+	case V4L2_PIX_FMT_HEVC_SLICE:
+		if (ctx->is_10bit_bitstream && fmt->fourcc == V4L2_PIX_FMT_MT2110T)
+			ret = true;
+		break;
+	default:
+		break;
+	}
+
+	return ret;
 }
 
 static struct mtk_q_data *mtk_vdec_get_q_data(struct mtk_vcodec_dec_ctx *ctx,
diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
index c8b4374c5e6c..cd607e90fe9c 100644
--- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
+++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
@@ -31,6 +31,7 @@ enum mtk_vdec_format_types {
 	MTK_VDEC_FORMAT_AV1_FRAME = 0x800,
 	MTK_VDEC_FORMAT_HEVC_FRAME = 0x1000,
 	MTK_VCODEC_INNER_RACING = 0x20000,
+	MTK_VDEC_IS_SUPPORT_10BIT = 0x40000,
 };
 
 /*
@@ -160,6 +161,8 @@ struct mtk_vcodec_dec_pdata {
  * @hw_id: hardware index used to identify different hardware.
  *
  * @msg_queue: msg queue used to store lat buffer information.
+ *
+ * @is_10bit_bitstream: set to true if it's 10bit bitstream
  */
 struct mtk_vcodec_dec_ctx {
 	enum mtk_instance_type type;
@@ -202,6 +205,8 @@ struct mtk_vcodec_dec_ctx {
 	int hw_id;
 
 	struct vdec_msg_queue msg_queue;
+
+	bool is_10bit_bitstream;
 };
 
 /**
diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c
index 99a84c7e1901..cef937fdf462 100644
--- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c
+++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c
@@ -200,7 +200,7 @@ static const struct mtk_stateless_control mtk_stateless_controls[] = {
 
 #define NUM_CTRLS ARRAY_SIZE(mtk_stateless_controls)
 
-static struct mtk_video_fmt mtk_video_formats[7];
+static struct mtk_video_fmt mtk_video_formats[9];
 
 static struct mtk_video_fmt default_out_format;
 static struct mtk_video_fmt default_cap_format;
@@ -387,6 +387,134 @@ static int mtk_vdec_flush_decoder(struct mtk_vcodec_dec_ctx *ctx)
 	return vdec_if_decode(ctx, NULL, NULL, &res_chg);
 }
 
+static int mtk_vcodec_get_pic_info(struct mtk_vcodec_dec_ctx *ctx)
+{
+	struct mtk_q_data *q_data;
+	int ret = 0;
+
+	q_data = &ctx->q_data[MTK_Q_DATA_DST];
+	if (q_data->fmt->num_planes == 1) {
+		mtk_v4l2_vdec_err(ctx, "[%d]Error!! 10bit mode not support one plane", ctx->id);
+		return -EINVAL;
+	}
+
+	ctx->capture_fourcc = q_data->fmt->fourcc;
+	ret = vdec_if_get_param(ctx, GET_PARAM_PIC_INFO, &ctx->picinfo);
+	if (ret) {
+		mtk_v4l2_vdec_err(ctx, "[%d]Error!! Get GET_PARAM_PICTURE_INFO Fail", ctx->id);
+		return ret;
+	}
+
+	ctx->last_decoded_picinfo = ctx->picinfo;
+
+	q_data->sizeimage[0] = ctx->picinfo.fb_sz[0];
+	q_data->bytesperline[0] = ctx->picinfo.buf_w * 5 / 4;
+
+	q_data->sizeimage[1] = ctx->picinfo.fb_sz[1];
+	q_data->bytesperline[1] = ctx->picinfo.buf_w * 5 / 4;
+
+	q_data->coded_width = ctx->picinfo.buf_w;
+	q_data->coded_height = ctx->picinfo.buf_h;
+	mtk_v4l2_vdec_dbg(1, ctx, "[%d] wxh=%dx%d pic wxh=%dx%d sz[0]=0x%x sz[1]=0x%x",
+			  ctx->id, ctx->picinfo.buf_w, ctx->picinfo.buf_h,
+			  ctx->picinfo.pic_w, ctx->picinfo.pic_h,
+			  q_data->sizeimage[0], q_data->sizeimage[1]);
+
+	return ret;
+}
+
+static int mtk_vdec_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct mtk_vcodec_dec_ctx *ctx = ctrl_to_dec_ctx(ctrl);
+	struct v4l2_ctrl_h264_sps *h264;
+	struct v4l2_ctrl_hevc_sps *h265;
+	struct v4l2_ctrl_vp9_frame *frame;
+	struct v4l2_ctrl_av1_sequence *seq;
+	struct v4l2_ctrl *hdr_ctrl;
+	const struct mtk_vcodec_dec_pdata *dec_pdata = ctx->dev->vdec_pdata;
+	const struct mtk_video_fmt *fmt;
+	int i = 0, ret = 0;
+
+	hdr_ctrl = ctrl;
+	if (!hdr_ctrl || !hdr_ctrl->p_cur.p)
+		return -EINVAL;
+
+	switch (hdr_ctrl->id) {
+	case V4L2_CID_STATELESS_H264_SPS:
+		h264 = (struct v4l2_ctrl_h264_sps *)hdr_ctrl->p_new.p;
+		if (h264->bit_depth_chroma_minus8 == 2 && h264->bit_depth_luma_minus8 == 2) {
+			ctx->is_10bit_bitstream = true;
+		} else if (h264->bit_depth_chroma_minus8 != 0 &&
+			   h264->bit_depth_luma_minus8 != 0) {
+			mtk_v4l2_vdec_err(ctx, "H264: chroma_minus8:%d, luma_minus8:%d",
+					  h264->bit_depth_chroma_minus8,
+					  h264->bit_depth_luma_minus8);
+			return -EINVAL;
+		}
+		break;
+	case V4L2_CID_STATELESS_HEVC_SPS:
+		h265 = (struct v4l2_ctrl_hevc_sps *)hdr_ctrl->p_new.p;
+		if (h265->bit_depth_chroma_minus8 == 2 && h265->bit_depth_luma_minus8 == 2) {
+			ctx->is_10bit_bitstream = true;
+		} else if (h265->bit_depth_chroma_minus8 != 0 &&
+			   h265->bit_depth_luma_minus8 != 0) {
+			mtk_v4l2_vdec_err(ctx, "HEVC: chroma_minus8:%d, luma_minus8:%d",
+					  h265->bit_depth_chroma_minus8,
+					  h265->bit_depth_luma_minus8);
+			return -EINVAL;
+		}
+		break;
+	case V4L2_CID_STATELESS_VP9_FRAME:
+		frame = (struct v4l2_ctrl_vp9_frame *)hdr_ctrl->p_new.p;
+		if (frame->bit_depth == 10) {
+			ctx->is_10bit_bitstream = true;
+		} else if (frame->bit_depth != 8) {
+			mtk_v4l2_vdec_err(ctx, "VP9: bit_depth:%d", frame->bit_depth);
+			return -EINVAL;
+		}
+		break;
+	case V4L2_CID_STATELESS_AV1_SEQUENCE:
+		seq = (struct v4l2_ctrl_av1_sequence *)hdr_ctrl->p_new.p;
+		if (seq->bit_depth == 10) {
+			ctx->is_10bit_bitstream = true;
+		} else if (seq->bit_depth != 8) {
+			mtk_v4l2_vdec_err(ctx, "AV1: bit_depth:%d", seq->bit_depth);
+			return -EINVAL;
+		}
+		break;
+	default:
+		mtk_v4l2_vdec_err(ctx, "Not supported ctrl id: 0x%x\n", hdr_ctrl->id);
+		return -EINVAL;
+	}
+
+	if (!ctx->is_10bit_bitstream)
+		return ret;
+
+	for (i = 0; i < *dec_pdata->num_formats; i++) {
+		fmt = &dec_pdata->vdec_formats[i];
+		if (fmt->fourcc == V4L2_PIX_FMT_MT2110R &&
+		    hdr_ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
+			ctx->q_data[MTK_Q_DATA_DST].fmt = fmt;
+			break;
+		}
+
+		if (fmt->fourcc == V4L2_PIX_FMT_MT2110T &&
+		    (hdr_ctrl->id == V4L2_CID_STATELESS_HEVC_SPS ||
+		    hdr_ctrl->id == V4L2_CID_STATELESS_VP9_FRAME ||
+		    hdr_ctrl->id == V4L2_CID_STATELESS_AV1_SEQUENCE)) {
+			ctx->q_data[MTK_Q_DATA_DST].fmt = fmt;
+			break;
+		}
+	}
+	ret = mtk_vcodec_get_pic_info(ctx);
+
+	return ret;
+}
+
+static const struct v4l2_ctrl_ops mtk_vcodec_dec_ctrl_ops = {
+	.s_ctrl = mtk_vdec_s_ctrl,
+};
+
 static int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_dec_ctx *ctx)
 {
 	unsigned int i;
@@ -399,7 +527,7 @@ static int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_dec_ctx *ctx)
 
 	for (i = 0; i < NUM_CTRLS; i++) {
 		struct v4l2_ctrl_config cfg = mtk_stateless_controls[i].cfg;
-
+		cfg.ops = &mtk_vcodec_dec_ctrl_ops;
 		v4l2_ctrl_new_custom(&ctx->ctrl_hdl, &cfg, NULL);
 		if (ctx->ctrl_hdl.error) {
 			mtk_v4l2_vdec_err(ctx, "Adding control %d failed %d", i,
@@ -466,6 +594,8 @@ static void mtk_vcodec_add_formats(unsigned int fourcc,
 		break;
 	case V4L2_PIX_FMT_MM21:
 	case V4L2_PIX_FMT_MT21C:
+	case V4L2_PIX_FMT_MT2110T:
+	case V4L2_PIX_FMT_MT2110R:
 		mtk_video_formats[count_formats].fourcc = fourcc;
 		mtk_video_formats[count_formats].type = MTK_FMT_FRAME;
 		mtk_video_formats[count_formats].num_planes = 2;
@@ -491,6 +621,12 @@ static void mtk_vcodec_get_supported_formats(struct mtk_vcodec_dec_ctx *ctx)
 		mtk_vcodec_add_formats(V4L2_PIX_FMT_MT21C, ctx);
 		cap_format_count++;
 	}
+	if (ctx->dev->dec_capability & MTK_VDEC_IS_SUPPORT_10BIT) {
+		mtk_vcodec_add_formats(V4L2_PIX_FMT_MT2110T, ctx);
+		cap_format_count++;
+		mtk_vcodec_add_formats(V4L2_PIX_FMT_MT2110R, ctx);
+		cap_format_count++;
+	}
 	if (ctx->dev->dec_capability & MTK_VDEC_FORMAT_MM21) {
 		mtk_vcodec_add_formats(V4L2_PIX_FMT_MM21, ctx);
 		cap_format_count++;
-- 
2.18.0

Re: [PATCH 3/3] media: mediatek: vcodec: Add driver to support 10bit

[Nicolas Dufresne]

Hi Yunfei,

this is a partial drive by review, I'll do more testing and more review soon.

Le mardi 11 juillet 2023 à 20:57 +0800, Yunfei Dong a écrit :
> From: Mingjia Zhang <mingjia.zhang@mediatek.com>
> 
> Adding to support capture formats V4L2_PIX_FMT_MT2110T and
> V4L2_PIX_FMT_MT2110R for 10bit playback. Need to get the size
> of each plane again when user space setting syntax to get 10bit
> information.
> 
> V4L2_PIX_FMT_MT2110T for AV1/VP9/HEVC.
> V4L2_PIX_FMT_MT2110R for H264.
> 
> Signed-off-by: Mingjia Zhang <mingjia.zhang@mediatek.com>
> Co-developed-by: Yunfei Dong <yunfei.dong@mediatek.com>
> Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> ---
>  .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
>  .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
>  .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140 +++++++++++++++++-
>  3 files changed, 163 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> index 5acb7dff18f2..91ed576d6821 100644
> --- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> +++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> @@ -37,7 +37,9 @@ static bool mtk_vdec_get_cap_fmt(struct mtk_vcodec_dec_ctx *ctx, int format_inde
>  {
>  	const struct mtk_vcodec_dec_pdata *dec_pdata = ctx->dev->vdec_pdata;
>  	const struct mtk_video_fmt *fmt;
> +	struct mtk_q_data *q_data;
>  	int num_frame_count = 0, i;
> +	bool ret = false;
>  
>  	fmt = &dec_pdata->vdec_formats[format_index];
>  	for (i = 0; i < *dec_pdata->num_formats; i++) {
> @@ -47,10 +49,26 @@ static bool mtk_vdec_get_cap_fmt(struct mtk_vcodec_dec_ctx *ctx, int format_inde
>  		num_frame_count++;
>  	}
>  
> -	if (num_frame_count == 1 || fmt->fourcc == V4L2_PIX_FMT_MM21)
> +	if (num_frame_count == 1 || (!ctx->is_10bit_bitstream && fmt->fourcc == V4L2_PIX_FMT_MM21))
>  		return true;
>  
> -	return false;
> +	q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> +	switch (q_data->fmt->fourcc) {
> +	case V4L2_PIX_FMT_H264_SLICE:
> +		if (ctx->is_10bit_bitstream && fmt->fourcc == V4L2_PIX_FMT_MT2110R)
> +			ret = true;
> +		break;
> +	case V4L2_PIX_FMT_VP9_FRAME:
> +	case V4L2_PIX_FMT_AV1_FRAME:
> +	case V4L2_PIX_FMT_HEVC_SLICE:
> +		if (ctx->is_10bit_bitstream && fmt->fourcc == V4L2_PIX_FMT_MT2110T)
> +			ret = true;
> +		break;
> +	default:
> +		break;
> +	}
> +
> +	return ret;
>  }
>  
>  static struct mtk_q_data *mtk_vdec_get_q_data(struct mtk_vcodec_dec_ctx *ctx,
> diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> index c8b4374c5e6c..cd607e90fe9c 100644
> --- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> +++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> @@ -31,6 +31,7 @@ enum mtk_vdec_format_types {
>  	MTK_VDEC_FORMAT_AV1_FRAME = 0x800,
>  	MTK_VDEC_FORMAT_HEVC_FRAME = 0x1000,
>  	MTK_VCODEC_INNER_RACING = 0x20000,
> +	MTK_VDEC_IS_SUPPORT_10BIT = 0x40000,
>  };
>  
>  /*
> @@ -160,6 +161,8 @@ struct mtk_vcodec_dec_pdata {
>   * @hw_id: hardware index used to identify different hardware.
>   *
>   * @msg_queue: msg queue used to store lat buffer information.
> + *
> + * @is_10bit_bitstream: set to true if it's 10bit bitstream
>   */
>  struct mtk_vcodec_dec_ctx {
>  	enum mtk_instance_type type;
> @@ -202,6 +205,8 @@ struct mtk_vcodec_dec_ctx {
>  	int hw_id;
>  
>  	struct vdec_msg_queue msg_queue;
> +
> +	bool is_10bit_bitstream;
>  };
>  
>  /**
> diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c
> index 99a84c7e1901..cef937fdf462 100644
> --- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c
> +++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c
> @@ -200,7 +200,7 @@ static const struct mtk_stateless_control mtk_stateless_controls[] = {
>  
>  #define NUM_CTRLS ARRAY_SIZE(mtk_stateless_controls)
>  
> -static struct mtk_video_fmt mtk_video_formats[7];
> +static struct mtk_video_fmt mtk_video_formats[9];
>  
>  static struct mtk_video_fmt default_out_format;
>  static struct mtk_video_fmt default_cap_format;
> @@ -387,6 +387,134 @@ static int mtk_vdec_flush_decoder(struct mtk_vcodec_dec_ctx *ctx)
>  	return vdec_if_decode(ctx, NULL, NULL, &res_chg);
>  }
>  
> +static int mtk_vcodec_get_pic_info(struct mtk_vcodec_dec_ctx *ctx)
> +{
> +	struct mtk_q_data *q_data;
> +	int ret = 0;
> +
> +	q_data = &ctx->q_data[MTK_Q_DATA_DST];
> +	if (q_data->fmt->num_planes == 1) {
> +		mtk_v4l2_vdec_err(ctx, "[%d]Error!! 10bit mode not support one plane", ctx->id);
> +		return -EINVAL;
> +	}
> +
> +	ctx->capture_fourcc = q_data->fmt->fourcc;
> +	ret = vdec_if_get_param(ctx, GET_PARAM_PIC_INFO, &ctx->picinfo);
> +	if (ret) {
> +		mtk_v4l2_vdec_err(ctx, "[%d]Error!! Get GET_PARAM_PICTURE_INFO Fail", ctx->id);
> +		return ret;
> +	}
> +
> +	ctx->last_decoded_picinfo = ctx->picinfo;
> +
> +	q_data->sizeimage[0] = ctx->picinfo.fb_sz[0];
> +	q_data->bytesperline[0] = ctx->picinfo.buf_w * 5 / 4;
> +
> +	q_data->sizeimage[1] = ctx->picinfo.fb_sz[1];
> +	q_data->bytesperline[1] = ctx->picinfo.buf_w * 5 / 4;
> +
> +	q_data->coded_width = ctx->picinfo.buf_w;
> +	q_data->coded_height = ctx->picinfo.buf_h;
> +	mtk_v4l2_vdec_dbg(1, ctx, "[%d] wxh=%dx%d pic wxh=%dx%d sz[0]=0x%x sz[1]=0x%x",
> +			  ctx->id, ctx->picinfo.buf_w, ctx->picinfo.buf_h,
> +			  ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> +			  q_data->sizeimage[0], q_data->sizeimage[1]);
> +
> +	return ret;
> +}
> +
> +static int mtk_vdec_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_vcodec_dec_ctx *ctx = ctrl_to_dec_ctx(ctrl);
> +	struct v4l2_ctrl_h264_sps *h264;
> +	struct v4l2_ctrl_hevc_sps *h265;
> +	struct v4l2_ctrl_vp9_frame *frame;
> +	struct v4l2_ctrl_av1_sequence *seq;
> +	struct v4l2_ctrl *hdr_ctrl;
> +	const struct mtk_vcodec_dec_pdata *dec_pdata = ctx->dev->vdec_pdata;
> +	const struct mtk_video_fmt *fmt;
> +	int i = 0, ret = 0;
> +
> +	hdr_ctrl = ctrl;
> +	if (!hdr_ctrl || !hdr_ctrl->p_cur.p)
> +		return -EINVAL;

There is a null check for hdr_ctrl->p_cur.p ...

> +
> +	switch (hdr_ctrl->id) {
> +	case V4L2_CID_STATELESS_H264_SPS:
> +		h264 = (struct v4l2_ctrl_h264_sps *)hdr_ctrl->p_new.p;

But you are using hdr_ctrl->p_new.p. I don't know if the checks are absolutly
required, I'll have a look.

> +		if (h264->bit_depth_chroma_minus8 == 2 && h264->bit_depth_luma_minus8 == 2) {

In the conformance streams, there is a 9bit luma file, which on all decoders
I've tried decodes fine inside a 10bit image.

> +			ctx->is_10bit_bitstream = true;
> +		} else if (h264->bit_depth_chroma_minus8 != 0 &&
> +			   h264->bit_depth_luma_minus8 != 0) {
> +			mtk_v4l2_vdec_err(ctx, "H264: chroma_minus8:%d, luma_minus8:%d",
> +					  h264->bit_depth_chroma_minus8,
> +					  h264->bit_depth_luma_minus8);
> +			return -EINVAL;
> +		}
> +		break;
> +	case V4L2_CID_STATELESS_HEVC_SPS:
> +		h265 = (struct v4l2_ctrl_hevc_sps *)hdr_ctrl->p_new.p;
> +		if (h265->bit_depth_chroma_minus8 == 2 && h265->bit_depth_luma_minus8 == 2) {
> +			ctx->is_10bit_bitstream = true;
> +		} else if (h265->bit_depth_chroma_minus8 != 0 &&
> +			   h265->bit_depth_luma_minus8 != 0) {
> +			mtk_v4l2_vdec_err(ctx, "HEVC: chroma_minus8:%d, luma_minus8:%d",
> +					  h265->bit_depth_chroma_minus8,
> +					  h265->bit_depth_luma_minus8);
> +			return -EINVAL;
> +		}
> +		break;
> +	case V4L2_CID_STATELESS_VP9_FRAME:
> +		frame = (struct v4l2_ctrl_vp9_frame *)hdr_ctrl->p_new.p;
> +		if (frame->bit_depth == 10) {
> +			ctx->is_10bit_bitstream = true;
> +		} else if (frame->bit_depth != 8) {
> +			mtk_v4l2_vdec_err(ctx, "VP9: bit_depth:%d", frame->bit_depth);
> +			return -EINVAL;
> +		}
> +		break;
> +	case V4L2_CID_STATELESS_AV1_SEQUENCE:
> +		seq = (struct v4l2_ctrl_av1_sequence *)hdr_ctrl->p_new.p;
> +		if (seq->bit_depth == 10) {
> +			ctx->is_10bit_bitstream = true;
> +		} else if (seq->bit_depth != 8) {
> +			mtk_v4l2_vdec_err(ctx, "AV1: bit_depth:%d", seq->bit_depth);
> +			return -EINVAL;
> +		}
> +		break;
> +	default:
> +		mtk_v4l2_vdec_err(ctx, "Not supported ctrl id: 0x%x\n", hdr_ctrl->id);
> +		return -EINVAL;

Haven't tested, but it feels like this will prevent setting the PPS among many
other controls. This should likely not be an error, and should return 0.

> +	}
> +
> +	if (!ctx->is_10bit_bitstream)
> +		return ret;
> +
> +	for (i = 0; i < *dec_pdata->num_formats; i++) {
> +		fmt = &dec_pdata->vdec_formats[i];
> +		if (fmt->fourcc == V4L2_PIX_FMT_MT2110R &&
> +		    hdr_ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> +			ctx->q_data[MTK_Q_DATA_DST].fmt = fmt;
> +			break;
> +		}
> +
> +		if (fmt->fourcc == V4L2_PIX_FMT_MT2110T &&
> +		    (hdr_ctrl->id == V4L2_CID_STATELESS_HEVC_SPS ||
> +		    hdr_ctrl->id == V4L2_CID_STATELESS_VP9_FRAME ||
> +		    hdr_ctrl->id == V4L2_CID_STATELESS_AV1_SEQUENCE)) {
> +			ctx->q_data[MTK_Q_DATA_DST].fmt = fmt;
> +			break;
> +		}
> +	}
> +	ret = mtk_vcodec_get_pic_info(ctx);
> +
> +	return ret;
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_vcodec_dec_ctrl_ops = {
> +	.s_ctrl = mtk_vdec_s_ctrl,
> +};
> +
>  static int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_dec_ctx *ctx)
>  {
>  	unsigned int i;
> @@ -399,7 +527,7 @@ static int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_dec_ctx *ctx)
>  
>  	for (i = 0; i < NUM_CTRLS; i++) {
>  		struct v4l2_ctrl_config cfg = mtk_stateless_controls[i].cfg;
> -
> +		cfg.ops = &mtk_vcodec_dec_ctrl_ops;
>  		v4l2_ctrl_new_custom(&ctx->ctrl_hdl, &cfg, NULL);
>  		if (ctx->ctrl_hdl.error) {
>  			mtk_v4l2_vdec_err(ctx, "Adding control %d failed %d", i,
> @@ -466,6 +594,8 @@ static void mtk_vcodec_add_formats(unsigned int fourcc,
>  		break;
>  	case V4L2_PIX_FMT_MM21:
>  	case V4L2_PIX_FMT_MT21C:
> +	case V4L2_PIX_FMT_MT2110T:
> +	case V4L2_PIX_FMT_MT2110R:
>  		mtk_video_formats[count_formats].fourcc = fourcc;
>  		mtk_video_formats[count_formats].type = MTK_FMT_FRAME;
>  		mtk_video_formats[count_formats].num_planes = 2;
> @@ -491,6 +621,12 @@ static void mtk_vcodec_get_supported_formats(struct mtk_vcodec_dec_ctx *ctx)
>  		mtk_vcodec_add_formats(V4L2_PIX_FMT_MT21C, ctx);
>  		cap_format_count++;
>  	}
> +	if (ctx->dev->dec_capability & MTK_VDEC_IS_SUPPORT_10BIT) {
> +		mtk_vcodec_add_formats(V4L2_PIX_FMT_MT2110T, ctx);
> +		cap_format_count++;
> +		mtk_vcodec_add_formats(V4L2_PIX_FMT_MT2110R, ctx);
> +		cap_format_count++;
> +	}
>  	if (ctx->dev->dec_capability & MTK_VDEC_FORMAT_MM21) {
>  		mtk_vcodec_add_formats(V4L2_PIX_FMT_MM21, ctx);
>  		cap_format_count++;

Re: [PATCH 0/3] media: mediatek: vcodec: Add driver to support 10bit

[Nicolas Dufresne]

Hi,

phew things missing in this cover letter,

Le mardi 11 juillet 2023 à 20:57 +0800, Yunfei Dong a écrit :
> Define two capture formats V4L2_PIX_FMT_MT2110R and
> V4L2_PIX_FMT_MT2110T to support 10bit in mt8195, mt8199 and more.
> Getting the size of each plane again when userspace set 10bit
> syntax to driver.
> 
> V4L2_PIX_FMT_MT2110R is used for H264, and V4L2_PIX_FMT_MT2110T
> is used for AV1/VP9/HEVC.
> 
> patch 1 Add driver to support 10bit
> patch 2 Add capture format V4L2_PIX_FMT_MT2110T to support 10bit tile mode
> patch 3 Add capture format V4L2_PIX_FMT_MT2110R to support 10bit raster mode
> ---
> - send the first version v1:
> - Run 10bit VP9/AV1 fluster test pass.

Please at least provide the score before and after these changes. Full report is
also nice. You didn't explain why H.264 is not tested.

> - Will return error when the 10bit parameter no correctly in function mtk_vdec_s_ctrl.
> ---

This cover letter is missing v4l2-compliance report. This is needed whenever
format mechanism is modified in a driver (not just for new drivers). Please add
and make sure there is not regression too.

> 
> Reference series:
> [1]: this series depends on v6 which is send by Yunfei Dong.
>      message-id: 20230704131349.8354-1-yunfei.dong@mediatek.com

Its seems like 6.5.0-rc1 with the depedency and this patchset does not boot on
MT8195 Chromebooks. Which paltform has this been validated on ?


For the record:


[   13.286252] platform 1c015000.dp-intf: deferred probe pending
[   13.292007] platform 1c113000.dp-intf: deferred probe pending
[   28.523484] rcu: INFO: rcu_preempt self-detected stall on CPU
[   28.529231] rcu: 	5-....: (5250 ticks this GP)
idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=2471
[   28.538706] rcu: 	(t=5254 jiffies g=-119 q=13320 ncpus=8)
[   28.544095] Task dump for CPU 0:
[   28.547313] task:cpuhp/0         state:R  running task     stack:0     pid:17
ppid:2      flags:0x0000000a
[   28.557221] Call trace:
[   28.559658]  __switch_to+0xe4/0x15c
[   28.563147]  0xffff776000196740
[   28.566282] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
[   28.572453] Hardware name: Acer Tomato (rev3 - 4) board (DT)
[   28.578101] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   28.585053] pc : smp_call_function_single+0x1a4/0x1bc
[   28.590098] lr : smp_call_function_single+0x178/0x1bc
[   28.595140] sp : ffff80008008b9c0
[   28.598444] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
[   28.605572] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
[   28.612699] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
[   28.619825] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
[   28.626952] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
[   28.634078] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
[   28.641204] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
[   28.648331] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
[   28.655458] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
[   28.662585] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
[   28.669711] Call trace:
[   28.672148]  smp_call_function_single+0x1a4/0x1bc
[   28.676843]  perf_cgroup_attach+0x74/0xd8
[   28.680847]  cgroup_migrate_execute+0x374/0x444
[   28.685368]  cgroup_migrate+0x74/0x8c
[   28.689021]  cgroup_attach_task+0x114/0x120
[   28.693195]  __cgroup_procs_write+0x108/0x230
[   28.697543]  cgroup_procs_write+0x1c/0x34
[   28.701543]  cgroup_file_write+0xa0/0x1a4
[   28.705545]  kernfs_fop_write_iter+0x118/0x1a8
[   28.709983]  vfs_write+0x2d0/0x39c
[   28.713376]  ksys_write+0x68/0xf4
[   28.716682]  __arm64_sys_write+0x1c/0x28
[   28.720594]  invoke_syscall+0x48/0x114
[   28.724337]  el0_svc_common.constprop.0+0x44/0xe4
[   28.729034]  do_el0_svc+0x38/0xa4
[   28.732341]  el0_svc+0x2c/0x84
[   28.735386]  el0t_64_sync_handler+0xc0/0xc4
[   28.739561]  el0t_64_sync+0x190/0x194
[   33.759553] vproc2: disabling
[   33.762551] vproc1: disabling
[   33.765548] vaud18: disabling
[   33.768760] va09: disabling
[   33.771599] vsram_md: disabling
[   91.755483] rcu: INFO: rcu_preempt self-detected stall on CPU
[   91.761220] rcu: 	5-....: (21005 ticks this GP)
idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=8569
[   91.770778] rcu: 	(t=21062 jiffies g=-119 q=13628 ncpus=8)
[   91.776253] Task dump for CPU 0:
[   91.779471] task:cpuhp/0         state:R  running task     stack:0     pid:17
ppid:2      flags:0x0000000a
[   91.789376] Call trace:
[   91.791812]  __switch_to+0xe4/0x15c
[   91.795294]  0xffff776000196740
[   91.798426] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
[   91.804597] Hardware name: Acer Tomato (rev3 - 4) board (DT)
[   91.810244] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   91.817195] pc : smp_call_function_single+0x1a4/0x1bc
[   91.822237] lr : smp_call_function_single+0x178/0x1bc
[   91.827278] sp : ffff80008008b9c0
[   91.830582] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
[   91.837709] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
[   91.844835] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
[   91.851962] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
[   91.859089] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
[   91.866215] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
[   91.873342] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
[   91.880468] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
[   91.887595] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
[   91.894721] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
[   91.901848] Call trace:
[   91.904284]  smp_call_function_single+0x1a4/0x1bc
[   91.908979]  perf_cgroup_attach+0x74/0xd8
[   91.912981]  cgroup_migrate_execute+0x374/0x444
[   91.917502]  cgroup_migrate+0x74/0x8c
[   91.921155]  cgroup_attach_task+0x114/0x120
[   91.925329]  __cgroup_procs_write+0x108/0x230
[   91.929677]  cgroup_procs_write+0x1c/0x34
[   91.933677]  cgroup_file_write+0xa0/0x1a4
[   91.937679]  kernfs_fop_write_iter+0x118/0x1a8
[   91.942117]  vfs_write+0x2d0/0x39c
[   91.945509]  ksys_write+0x68/0xf4
[   91.948814]  __arm64_sys_write+0x1c/0x28
[   91.952726]  invoke_syscall+0x48/0x114
[   91.956467]  el0_svc_common.constprop.0+0x44/0xe4
[   91.961164]  do_el0_svc+0x38/0xa4
[   91.964472]  el0_svc+0x2c/0x84
[   91.967517]  el0t_64_sync_handler+0xc0/0xc4
[   91.971691]  el0t_64_sync+0x190/0x194
[  154.987483] rcu: INFO: rcu_preempt self-detected stall on CPU
[  154.993218] rcu: 	5-....: (36760 ticks this GP)
idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=14547
[  155.002862] rcu: 	(t=36870 jiffies g=-119 q=13628 ncpus=8)
[  155.008337] Task dump for CPU 0:
[  155.011554] task:cpuhp/0         state:R  running task     stack:0     pid:17
ppid:2      flags:0x0000000a
[  155.021458] Call trace:
[  155.023894]  __switch_to+0xe4/0x15c
[  155.027376]  0xffff776000196740
[  155.030507] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
[  155.036676] Hardware name: Acer Tomato (rev3 - 4) board (DT)
[  155.042323] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  155.049274] pc : smp_call_function_single+0x1a4/0x1bc
[  155.054316] lr : smp_call_function_single+0x178/0x1bc
[  155.059358] sp : ffff80008008b9c0
[  155.062662] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
[  155.069788] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
[  155.076914] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
[  155.084041] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
[  155.091168] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
[  155.098294] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
[  155.105421] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
[  155.112548] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
[  155.119675] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
[  155.126801] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
[  155.133928] Call trace:
[  155.136363]  smp_call_function_single+0x1a4/0x1bc
[  155.141059]  perf_cgroup_attach+0x74/0xd8
[  155.145060]  cgroup_migrate_execute+0x374/0x444
[  155.149581]  cgroup_migrate+0x74/0x8c
[  155.153234]  cgroup_attach_task+0x114/0x120
[  155.157408]  __cgroup_procs_write+0x108/0x230
[  155.161755]  cgroup_procs_write+0x1c/0x34
[  155.165756]  cgroup_file_write+0xa0/0x1a4
[  155.169757]  kernfs_fop_write_iter+0x118/0x1a8
[  155.174195]  vfs_write+0x2d0/0x39c
[  155.177587]  ksys_write+0x68/0xf4
[  155.180893]  __arm64_sys_write+0x1c/0x28
[  155.184805]  invoke_syscall+0x48/0x114
[  155.188547]  el0_svc_common.constprop.0+0x44/0xe4
[  155.193243]  do_el0_svc+0x38/0xa4
[  155.196551]  el0_svc+0x2c/0x84
[  155.199595]  el0t_64_sync_handler+0xc0/0xc4
[  155.203769]  el0t_64_sync+0x190/0x194


> 
> Mingjia Zhang (3):
>   media: mediatek: vcodec: Add capture format to support 10bit tile mode
>   media: mediatek: vcodec: Add capture format to support 10bit raster
>     mode
>   media: mediatek: vcodec: Add driver to support 10bit
> 
>  .../media/v4l/pixfmt-reserved.rst             |  15 ++
>  .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
>  .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
>  .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140 +++++++++++++++++-
>  drivers/media/v4l2-core/v4l2-common.c         |   4 +
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
>  include/uapi/linux/videodev2.h                |   2 +
>  7 files changed, 186 insertions(+), 4 deletions(-)
> 

Re: [PATCH 0/3] media: mediatek: vcodec: Add driver to support 10bit

[Nicolas Dufresne]

Le mardi 11 juillet 2023 à 15:15 -0400, Nicolas Dufresne a écrit :
> Hi,
> 
> phew things missing in this cover letter,
> 
> Le mardi 11 juillet 2023 à 20:57 +0800, Yunfei Dong a écrit :
> > Define two capture formats V4L2_PIX_FMT_MT2110R and
> > V4L2_PIX_FMT_MT2110T to support 10bit in mt8195, mt8199 and more.
> > Getting the size of each plane again when userspace set 10bit
> > syntax to driver.
> > 
> > V4L2_PIX_FMT_MT2110R is used for H264, and V4L2_PIX_FMT_MT2110T
> > is used for AV1/VP9/HEVC.
> > 
> > patch 1 Add driver to support 10bit
> > patch 2 Add capture format V4L2_PIX_FMT_MT2110T to support 10bit tile mode
> > patch 3 Add capture format V4L2_PIX_FMT_MT2110R to support 10bit raster mode
> > ---
> > - send the first version v1:
> > - Run 10bit VP9/AV1 fluster test pass.
> 
> Please at least provide the score before and after these changes. Full report is
> also nice. You didn't explain why H.264 is not tested.
> 
> > - Will return error when the 10bit parameter no correctly in function mtk_vdec_s_ctrl.
> > ---
> 
> This cover letter is missing v4l2-compliance report. This is needed whenever
> format mechanism is modified in a driver (not just for new drivers). Please add
> and make sure there is not regression too.
> 
> > 
> > Reference series:
> > [1]: this series depends on v6 which is send by Yunfei Dong.
> >      message-id: 20230704131349.8354-1-yunfei.dong@mediatek.com
> 
> Its seems like 6.5.0-rc1 with the depedency and this patchset does not boot on
> MT8195 Chromebooks. Which paltform has this been validated on ?

I've manged to boot it using Collabora for-kernel-ci next kernel, which is based
on media-state/master 6.5.0-rc1. I suspect there is more depedencies then
described which might be pending in other trees. As all the patches are from
other trees submission, I suppose this is fine for now. Notice that the boot log
does not look very promising though, I will do some more testing, but from clean
boot, after udev have filled the HW database:

[   10.399826] mt7921e 0000:01:00.0: WM Firmware Version: ____010000, Build Time: 20230117170942
[   11.882202] mtk-vcodec-dec 18000000.video-codec: [MTK_V4L2][ERROR] Not supported ctrl id: 0xa40903
[   11.882202] 
[   11.910945] mtk-vcodec-dec 18000000.video-codec: [MTK_V4L2][ERROR] Not supported ctrl id: 0xa40903

This is produced when udev calls /usr/lib/udev/v4l_id, I don't think its doing
anything that would explain error logs.

regards,
Nicolas

> 
> 
> For the record:
> 
> 
> [   13.286252] platform 1c015000.dp-intf: deferred probe pending
> [   13.292007] platform 1c113000.dp-intf: deferred probe pending
> [   28.523484] rcu: INFO: rcu_preempt self-detected stall on CPU
> [   28.529231] rcu: 	5-....: (5250 ticks this GP)
> idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=2471
> [   28.538706] rcu: 	(t=5254 jiffies g=-119 q=13320 ncpus=8)
> [   28.544095] Task dump for CPU 0:
> [   28.547313] task:cpuhp/0         state:R  running task     stack:0     pid:17
> ppid:2      flags:0x0000000a
> [   28.557221] Call trace:
> [   28.559658]  __switch_to+0xe4/0x15c
> [   28.563147]  0xffff776000196740
> [   28.566282] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> [   28.572453] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> [   28.578101] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [   28.585053] pc : smp_call_function_single+0x1a4/0x1bc
> [   28.590098] lr : smp_call_function_single+0x178/0x1bc
> [   28.595140] sp : ffff80008008b9c0
> [   28.598444] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
> [   28.605572] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
> [   28.612699] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
> [   28.619825] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
> [   28.626952] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
> [   28.634078] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
> [   28.641204] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
> [   28.648331] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
> [   28.655458] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
> [   28.662585] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
> [   28.669711] Call trace:
> [   28.672148]  smp_call_function_single+0x1a4/0x1bc
> [   28.676843]  perf_cgroup_attach+0x74/0xd8
> [   28.680847]  cgroup_migrate_execute+0x374/0x444
> [   28.685368]  cgroup_migrate+0x74/0x8c
> [   28.689021]  cgroup_attach_task+0x114/0x120
> [   28.693195]  __cgroup_procs_write+0x108/0x230
> [   28.697543]  cgroup_procs_write+0x1c/0x34
> [   28.701543]  cgroup_file_write+0xa0/0x1a4
> [   28.705545]  kernfs_fop_write_iter+0x118/0x1a8
> [   28.709983]  vfs_write+0x2d0/0x39c
> [   28.713376]  ksys_write+0x68/0xf4
> [   28.716682]  __arm64_sys_write+0x1c/0x28
> [   28.720594]  invoke_syscall+0x48/0x114
> [   28.724337]  el0_svc_common.constprop.0+0x44/0xe4
> [   28.729034]  do_el0_svc+0x38/0xa4
> [   28.732341]  el0_svc+0x2c/0x84
> [   28.735386]  el0t_64_sync_handler+0xc0/0xc4
> [   28.739561]  el0t_64_sync+0x190/0x194
> [   33.759553] vproc2: disabling
> [   33.762551] vproc1: disabling
> [   33.765548] vaud18: disabling
> [   33.768760] va09: disabling
> [   33.771599] vsram_md: disabling
> [   91.755483] rcu: INFO: rcu_preempt self-detected stall on CPU
> [   91.761220] rcu: 	5-....: (21005 ticks this GP)
> idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=8569
> [   91.770778] rcu: 	(t=21062 jiffies g=-119 q=13628 ncpus=8)
> [   91.776253] Task dump for CPU 0:
> [   91.779471] task:cpuhp/0         state:R  running task     stack:0     pid:17
> ppid:2      flags:0x0000000a
> [   91.789376] Call trace:
> [   91.791812]  __switch_to+0xe4/0x15c
> [   91.795294]  0xffff776000196740
> [   91.798426] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> [   91.804597] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> [   91.810244] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [   91.817195] pc : smp_call_function_single+0x1a4/0x1bc
> [   91.822237] lr : smp_call_function_single+0x178/0x1bc
> [   91.827278] sp : ffff80008008b9c0
> [   91.830582] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
> [   91.837709] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
> [   91.844835] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
> [   91.851962] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
> [   91.859089] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
> [   91.866215] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
> [   91.873342] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
> [   91.880468] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
> [   91.887595] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
> [   91.894721] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
> [   91.901848] Call trace:
> [   91.904284]  smp_call_function_single+0x1a4/0x1bc
> [   91.908979]  perf_cgroup_attach+0x74/0xd8
> [   91.912981]  cgroup_migrate_execute+0x374/0x444
> [   91.917502]  cgroup_migrate+0x74/0x8c
> [   91.921155]  cgroup_attach_task+0x114/0x120
> [   91.925329]  __cgroup_procs_write+0x108/0x230
> [   91.929677]  cgroup_procs_write+0x1c/0x34
> [   91.933677]  cgroup_file_write+0xa0/0x1a4
> [   91.937679]  kernfs_fop_write_iter+0x118/0x1a8
> [   91.942117]  vfs_write+0x2d0/0x39c
> [   91.945509]  ksys_write+0x68/0xf4
> [   91.948814]  __arm64_sys_write+0x1c/0x28
> [   91.952726]  invoke_syscall+0x48/0x114
> [   91.956467]  el0_svc_common.constprop.0+0x44/0xe4
> [   91.961164]  do_el0_svc+0x38/0xa4
> [   91.964472]  el0_svc+0x2c/0x84
> [   91.967517]  el0t_64_sync_handler+0xc0/0xc4
> [   91.971691]  el0t_64_sync+0x190/0x194
> [  154.987483] rcu: INFO: rcu_preempt self-detected stall on CPU
> [  154.993218] rcu: 	5-....: (36760 ticks this GP)
> idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=14547
> [  155.002862] rcu: 	(t=36870 jiffies g=-119 q=13628 ncpus=8)
> [  155.008337] Task dump for CPU 0:
> [  155.011554] task:cpuhp/0         state:R  running task     stack:0     pid:17
> ppid:2      flags:0x0000000a
> [  155.021458] Call trace:
> [  155.023894]  __switch_to+0xe4/0x15c
> [  155.027376]  0xffff776000196740
> [  155.030507] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> [  155.036676] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> [  155.042323] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [  155.049274] pc : smp_call_function_single+0x1a4/0x1bc
> [  155.054316] lr : smp_call_function_single+0x178/0x1bc
> [  155.059358] sp : ffff80008008b9c0
> [  155.062662] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
> [  155.069788] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
> [  155.076914] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
> [  155.084041] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
> [  155.091168] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
> [  155.098294] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
> [  155.105421] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
> [  155.112548] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
> [  155.119675] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
> [  155.126801] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
> [  155.133928] Call trace:
> [  155.136363]  smp_call_function_single+0x1a4/0x1bc
> [  155.141059]  perf_cgroup_attach+0x74/0xd8
> [  155.145060]  cgroup_migrate_execute+0x374/0x444
> [  155.149581]  cgroup_migrate+0x74/0x8c
> [  155.153234]  cgroup_attach_task+0x114/0x120
> [  155.157408]  __cgroup_procs_write+0x108/0x230
> [  155.161755]  cgroup_procs_write+0x1c/0x34
> [  155.165756]  cgroup_file_write+0xa0/0x1a4
> [  155.169757]  kernfs_fop_write_iter+0x118/0x1a8
> [  155.174195]  vfs_write+0x2d0/0x39c
> [  155.177587]  ksys_write+0x68/0xf4
> [  155.180893]  __arm64_sys_write+0x1c/0x28
> [  155.184805]  invoke_syscall+0x48/0x114
> [  155.188547]  el0_svc_common.constprop.0+0x44/0xe4
> [  155.193243]  do_el0_svc+0x38/0xa4
> [  155.196551]  el0_svc+0x2c/0x84
> [  155.199595]  el0t_64_sync_handler+0xc0/0xc4
> [  155.203769]  el0t_64_sync+0x190/0x194
> 
> 
> > 
> > Mingjia Zhang (3):
> >   media: mediatek: vcodec: Add capture format to support 10bit tile mode
> >   media: mediatek: vcodec: Add capture format to support 10bit raster
> >     mode
> >   media: mediatek: vcodec: Add driver to support 10bit
> > 
> >  .../media/v4l/pixfmt-reserved.rst             |  15 ++
> >  .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
> >  .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
> >  .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140 +++++++++++++++++-
> >  drivers/media/v4l2-core/v4l2-common.c         |   4 +
> >  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
> >  include/uapi/linux/videodev2.h                |   2 +
> >  7 files changed, 186 insertions(+), 4 deletions(-)
> > 
> 

Re: [PATCH 0/3] media: mediatek: vcodec: Add driver to support 10bit

[Nícolas F. R. A. Prado]

On Tue, Jul 11, 2023 at 03:15:33PM -0400, Nicolas Dufresne wrote:
[..]
> > Reference series:
> > [1]: this series depends on v6 which is send by Yunfei Dong.
> >      message-id: 20230704131349.8354-1-yunfei.dong@mediatek.com
> 
> Its seems like 6.5.0-rc1 with the depedency and this patchset does not boot on
> MT8195 Chromebooks. Which paltform has this been validated on ?

Are you sure this was caused by these patches?

I've recently noticed two issues that cause MT8195 Tomato to softlock the CPU
similar to your trace below.

One of them is caused by having CONFIG_ARM_DSU_PMU=m. Note that it is present in
the arm64 defconfig. To workaround, the config needs to be disabled.

The other is caused by commit 46600ab142f8 ("regulator: Set
PROBE_PREFER_ASYNCHRONOUS for drivers between 5.10 and 5.15"). The whole machine
gets really slow, including the serial. This issue only happens sometimes. To
workaround that commit can be reverted.

I intend to look into those issues and provide proper fixes in the following
days.

Thanks,
Nícolas

> 
> 
> For the record:
> 
> 
> [   13.286252] platform 1c015000.dp-intf: deferred probe pending
> [   13.292007] platform 1c113000.dp-intf: deferred probe pending
> [   28.523484] rcu: INFO: rcu_preempt self-detected stall on CPU
> [   28.529231] rcu: 	5-....: (5250 ticks this GP)
> idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=2471
> [   28.538706] rcu: 	(t=5254 jiffies g=-119 q=13320 ncpus=8)
> [   28.544095] Task dump for CPU 0:
> [   28.547313] task:cpuhp/0         state:R  running task     stack:0     pid:17
> ppid:2      flags:0x0000000a
> [   28.557221] Call trace:
> [   28.559658]  __switch_to+0xe4/0x15c
> [   28.563147]  0xffff776000196740
> [   28.566282] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> [   28.572453] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> [   28.578101] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [   28.585053] pc : smp_call_function_single+0x1a4/0x1bc
> [   28.590098] lr : smp_call_function_single+0x178/0x1bc
> [   28.595140] sp : ffff80008008b9c0
> [   28.598444] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
> [   28.605572] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
> [   28.612699] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
> [   28.619825] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
> [   28.626952] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
> [   28.634078] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
> [   28.641204] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
> [   28.648331] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
> [   28.655458] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
> [   28.662585] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
> [   28.669711] Call trace:
> [   28.672148]  smp_call_function_single+0x1a4/0x1bc
> [   28.676843]  perf_cgroup_attach+0x74/0xd8
> [   28.680847]  cgroup_migrate_execute+0x374/0x444
> [   28.685368]  cgroup_migrate+0x74/0x8c
> [   28.689021]  cgroup_attach_task+0x114/0x120
> [   28.693195]  __cgroup_procs_write+0x108/0x230
> [   28.697543]  cgroup_procs_write+0x1c/0x34
> [   28.701543]  cgroup_file_write+0xa0/0x1a4
> [   28.705545]  kernfs_fop_write_iter+0x118/0x1a8
> [   28.709983]  vfs_write+0x2d0/0x39c
> [   28.713376]  ksys_write+0x68/0xf4
> [   28.716682]  __arm64_sys_write+0x1c/0x28
> [   28.720594]  invoke_syscall+0x48/0x114
> [   28.724337]  el0_svc_common.constprop.0+0x44/0xe4
> [   28.729034]  do_el0_svc+0x38/0xa4
> [   28.732341]  el0_svc+0x2c/0x84
> [   28.735386]  el0t_64_sync_handler+0xc0/0xc4
> [   28.739561]  el0t_64_sync+0x190/0x194
> [   33.759553] vproc2: disabling
> [   33.762551] vproc1: disabling
> [   33.765548] vaud18: disabling
> [   33.768760] va09: disabling
> [   33.771599] vsram_md: disabling
> [   91.755483] rcu: INFO: rcu_preempt self-detected stall on CPU
> [   91.761220] rcu: 	5-....: (21005 ticks this GP)
> idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=8569
> [   91.770778] rcu: 	(t=21062 jiffies g=-119 q=13628 ncpus=8)
> [   91.776253] Task dump for CPU 0:
> [   91.779471] task:cpuhp/0         state:R  running task     stack:0     pid:17
> ppid:2      flags:0x0000000a
> [   91.789376] Call trace:
> [   91.791812]  __switch_to+0xe4/0x15c
> [   91.795294]  0xffff776000196740
> [   91.798426] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> [   91.804597] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> [   91.810244] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [   91.817195] pc : smp_call_function_single+0x1a4/0x1bc
> [   91.822237] lr : smp_call_function_single+0x178/0x1bc
> [   91.827278] sp : ffff80008008b9c0
> [   91.830582] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
> [   91.837709] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
> [   91.844835] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
> [   91.851962] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
> [   91.859089] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
> [   91.866215] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
> [   91.873342] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
> [   91.880468] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
> [   91.887595] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
> [   91.894721] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
> [   91.901848] Call trace:
> [   91.904284]  smp_call_function_single+0x1a4/0x1bc
> [   91.908979]  perf_cgroup_attach+0x74/0xd8
> [   91.912981]  cgroup_migrate_execute+0x374/0x444
> [   91.917502]  cgroup_migrate+0x74/0x8c
> [   91.921155]  cgroup_attach_task+0x114/0x120
> [   91.925329]  __cgroup_procs_write+0x108/0x230
> [   91.929677]  cgroup_procs_write+0x1c/0x34
> [   91.933677]  cgroup_file_write+0xa0/0x1a4
> [   91.937679]  kernfs_fop_write_iter+0x118/0x1a8
> [   91.942117]  vfs_write+0x2d0/0x39c
> [   91.945509]  ksys_write+0x68/0xf4
> [   91.948814]  __arm64_sys_write+0x1c/0x28
> [   91.952726]  invoke_syscall+0x48/0x114
> [   91.956467]  el0_svc_common.constprop.0+0x44/0xe4
> [   91.961164]  do_el0_svc+0x38/0xa4
> [   91.964472]  el0_svc+0x2c/0x84
> [   91.967517]  el0t_64_sync_handler+0xc0/0xc4
> [   91.971691]  el0t_64_sync+0x190/0x194
> [  154.987483] rcu: INFO: rcu_preempt self-detected stall on CPU
> [  154.993218] rcu: 	5-....: (36760 ticks this GP)
> idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=14547
> [  155.002862] rcu: 	(t=36870 jiffies g=-119 q=13628 ncpus=8)
> [  155.008337] Task dump for CPU 0:
> [  155.011554] task:cpuhp/0         state:R  running task     stack:0     pid:17
> ppid:2      flags:0x0000000a
> [  155.021458] Call trace:
> [  155.023894]  __switch_to+0xe4/0x15c
> [  155.027376]  0xffff776000196740
> [  155.030507] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> [  155.036676] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> [  155.042323] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [  155.049274] pc : smp_call_function_single+0x1a4/0x1bc
> [  155.054316] lr : smp_call_function_single+0x178/0x1bc
> [  155.059358] sp : ffff80008008b9c0
> [  155.062662] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
> [  155.069788] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
> [  155.076914] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
> [  155.084041] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
> [  155.091168] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
> [  155.098294] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
> [  155.105421] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
> [  155.112548] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
> [  155.119675] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
> [  155.126801] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
> [  155.133928] Call trace:
> [  155.136363]  smp_call_function_single+0x1a4/0x1bc
> [  155.141059]  perf_cgroup_attach+0x74/0xd8
> [  155.145060]  cgroup_migrate_execute+0x374/0x444
> [  155.149581]  cgroup_migrate+0x74/0x8c
> [  155.153234]  cgroup_attach_task+0x114/0x120
> [  155.157408]  __cgroup_procs_write+0x108/0x230
> [  155.161755]  cgroup_procs_write+0x1c/0x34
> [  155.165756]  cgroup_file_write+0xa0/0x1a4
> [  155.169757]  kernfs_fop_write_iter+0x118/0x1a8
> [  155.174195]  vfs_write+0x2d0/0x39c
> [  155.177587]  ksys_write+0x68/0xf4
> [  155.180893]  __arm64_sys_write+0x1c/0x28
> [  155.184805]  invoke_syscall+0x48/0x114
> [  155.188547]  el0_svc_common.constprop.0+0x44/0xe4
> [  155.193243]  do_el0_svc+0x38/0xa4
> [  155.196551]  el0_svc+0x2c/0x84
> [  155.199595]  el0t_64_sync_handler+0xc0/0xc4
> [  155.203769]  el0t_64_sync+0x190/0x194
> 
> 
> > 
> > Mingjia Zhang (3):
> >   media: mediatek: vcodec: Add capture format to support 10bit tile mode
> >   media: mediatek: vcodec: Add capture format to support 10bit raster
> >     mode
> >   media: mediatek: vcodec: Add driver to support 10bit
> > 
> >  .../media/v4l/pixfmt-reserved.rst             |  15 ++
> >  .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
> >  .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
> >  .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140 +++++++++++++++++-
> >  drivers/media/v4l2-core/v4l2-common.c         |   4 +
> >  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
> >  include/uapi/linux/videodev2.h                |   2 +
> >  7 files changed, 186 insertions(+), 4 deletions(-)
> > 
> 

Re: [PATCH 0/3] media: mediatek: vcodec: Add driver to support 10bit

[Nicolas Dufresne]

Le mardi 11 juillet 2023 à 20:57 +0800, Yunfei Dong a écrit :
> Define two capture formats V4L2_PIX_FMT_MT2110R and
> V4L2_PIX_FMT_MT2110T to support 10bit in mt8195, mt8199 and more.
> Getting the size of each plane again when userspace set 10bit
> syntax to driver.
> 
> V4L2_PIX_FMT_MT2110R is used for H264, and V4L2_PIX_FMT_MT2110T
> is used for AV1/VP9/HEVC.
> 
> patch 1 Add driver to support 10bit
> patch 2 Add capture format V4L2_PIX_FMT_MT2110T to support 10bit tile mode
> patch 3 Add capture format V4L2_PIX_FMT_MT2110R to support 10bit raster mode
> ---
> - send the first version v1:
> - Run 10bit VP9/AV1 fluster test pass.
> - Will return error when the 10bit parameter no correctly in function mtk_vdec_s_ctrl.

Just ran AV1 and VP8 tests locally. VP8 completely fails (0/61) as the update
rejects the VP8 controls in mtk_vdec_s_ctrl. As for AV1, it completely regressed
(0 success, there is "invalid tge_size" kind of errors). I've been testing with
the upstream firmware since you didn't mention any firmware update (md5 is
52d11ed015d4e2f6d4e9ce9a8fcf20e3).

You can retrace my step using this kernel and normal Debian OS:

https://gitlab.collabora.com/nicolas/linux/-/commits/mt8195-10bit-2

regards,
Nicolas

> ---
> 
> Reference series:
> [1]: this series depends on v6 which is send by Yunfei Dong.
>      message-id: 20230704131349.8354-1-yunfei.dong@mediatek.com
> 
> Mingjia Zhang (3):
>   media: mediatek: vcodec: Add capture format to support 10bit tile mode
>   media: mediatek: vcodec: Add capture format to support 10bit raster
>     mode
>   media: mediatek: vcodec: Add driver to support 10bit
> 
>  .../media/v4l/pixfmt-reserved.rst             |  15 ++
>  .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
>  .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
>  .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140 +++++++++++++++++-
>  drivers/media/v4l2-core/v4l2-common.c         |   4 +
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
>  include/uapi/linux/videodev2.h                |   2 +
>  7 files changed, 186 insertions(+), 4 deletions(-)
> 

Re: [PATCH 0/3] media: mediatek: vcodec: Add driver to support 10bit

[Nicolas Dufresne]

Le mardi 11 juillet 2023 à 15:40 -0400, Nícolas F. R. A. Prado a écrit :
> On Tue, Jul 11, 2023 at 03:15:33PM -0400, Nicolas Dufresne wrote:
> [..]
> > > Reference series:
> > > [1]: this series depends on v6 which is send by Yunfei Dong.
> > >      message-id: 20230704131349.8354-1-yunfei.dong@mediatek.com
> > 
> > Its seems like 6.5.0-rc1 with the depedency and this patchset does not boot on
> > MT8195 Chromebooks. Which paltform has this been validated on ?
> 
> Are you sure this was caused by these patches?
> 
> I've recently noticed two issues that cause MT8195 Tomato to softlock the CPU
> similar to your trace below.
> 
> One of them is caused by having CONFIG_ARM_DSU_PMU=m. Note that it is present in
> the arm64 defconfig. To workaround, the config needs to be disabled.

This isn't set in my config.

  # CONFIG_ARM_DSU_PMU is not set

> 
> The other is caused by commit 46600ab142f8 ("regulator: Set
> PROBE_PREFER_ASYNCHRONOUS for drivers between 5.10 and 5.15"). The whole machine
> gets really slow, including the serial. This issue only happens sometimes. To
> workaround that commit can be reverted.
> 
> I intend to look into those issues and provide proper fixes in the following
> days.

If by slow you mean the self stall detection triggers, maybe, I haven't spent
time studying the current delta from 6.5.0-rc1 and Collabora forci branch, but
its non-null. Also, applying the same patches on top did not break the boot, so
quite unlikely. Here's a snapshot of my branch I made, I simply revert some
conflicting  changes related to VP9 racing, then it applied cleanly.

https://gitlab.collabora.com/nicolas/linux/-/commits/mt8195-10bit-2

Nicolas

> 
> Thanks,
> Nícolas
> 
> > 
> > 
> > For the record:
> > 
> > 
> > [   13.286252] platform 1c015000.dp-intf: deferred probe pending
> > [   13.292007] platform 1c113000.dp-intf: deferred probe pending
> > [   28.523484] rcu: INFO: rcu_preempt self-detected stall on CPU
> > [   28.529231] rcu: 	5-....: (5250 ticks this GP)
> > idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=2471
> > [   28.538706] rcu: 	(t=5254 jiffies g=-119 q=13320 ncpus=8)
> > [   28.544095] Task dump for CPU 0:
> > [   28.547313] task:cpuhp/0         state:R  running task     stack:0     pid:17
> > ppid:2      flags:0x0000000a
> > [   28.557221] Call trace:
> > [   28.559658]  __switch_to+0xe4/0x15c
> > [   28.563147]  0xffff776000196740
> > [   28.566282] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> > [   28.572453] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> > [   28.578101] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > [   28.585053] pc : smp_call_function_single+0x1a4/0x1bc
> > [   28.590098] lr : smp_call_function_single+0x178/0x1bc
> > [   28.595140] sp : ffff80008008b9c0
> > [   28.598444] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
> > [   28.605572] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
> > [   28.612699] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
> > [   28.619825] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
> > [   28.626952] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
> > [   28.634078] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
> > [   28.641204] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
> > [   28.648331] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
> > [   28.655458] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
> > [   28.662585] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
> > [   28.669711] Call trace:
> > [   28.672148]  smp_call_function_single+0x1a4/0x1bc
> > [   28.676843]  perf_cgroup_attach+0x74/0xd8
> > [   28.680847]  cgroup_migrate_execute+0x374/0x444
> > [   28.685368]  cgroup_migrate+0x74/0x8c
> > [   28.689021]  cgroup_attach_task+0x114/0x120
> > [   28.693195]  __cgroup_procs_write+0x108/0x230
> > [   28.697543]  cgroup_procs_write+0x1c/0x34
> > [   28.701543]  cgroup_file_write+0xa0/0x1a4
> > [   28.705545]  kernfs_fop_write_iter+0x118/0x1a8
> > [   28.709983]  vfs_write+0x2d0/0x39c
> > [   28.713376]  ksys_write+0x68/0xf4
> > [   28.716682]  __arm64_sys_write+0x1c/0x28
> > [   28.720594]  invoke_syscall+0x48/0x114
> > [   28.724337]  el0_svc_common.constprop.0+0x44/0xe4
> > [   28.729034]  do_el0_svc+0x38/0xa4
> > [   28.732341]  el0_svc+0x2c/0x84
> > [   28.735386]  el0t_64_sync_handler+0xc0/0xc4
> > [   28.739561]  el0t_64_sync+0x190/0x194
> > [   33.759553] vproc2: disabling
> > [   33.762551] vproc1: disabling
> > [   33.765548] vaud18: disabling
> > [   33.768760] va09: disabling
> > [   33.771599] vsram_md: disabling
> > [   91.755483] rcu: INFO: rcu_preempt self-detected stall on CPU
> > [   91.761220] rcu: 	5-....: (21005 ticks this GP)
> > idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=8569
> > [   91.770778] rcu: 	(t=21062 jiffies g=-119 q=13628 ncpus=8)
> > [   91.776253] Task dump for CPU 0:
> > [   91.779471] task:cpuhp/0         state:R  running task     stack:0     pid:17
> > ppid:2      flags:0x0000000a
> > [   91.789376] Call trace:
> > [   91.791812]  __switch_to+0xe4/0x15c
> > [   91.795294]  0xffff776000196740
> > [   91.798426] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> > [   91.804597] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> > [   91.810244] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > [   91.817195] pc : smp_call_function_single+0x1a4/0x1bc
> > [   91.822237] lr : smp_call_function_single+0x178/0x1bc
> > [   91.827278] sp : ffff80008008b9c0
> > [   91.830582] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
> > [   91.837709] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
> > [   91.844835] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
> > [   91.851962] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
> > [   91.859089] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
> > [   91.866215] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
> > [   91.873342] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
> > [   91.880468] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
> > [   91.887595] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
> > [   91.894721] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
> > [   91.901848] Call trace:
> > [   91.904284]  smp_call_function_single+0x1a4/0x1bc
> > [   91.908979]  perf_cgroup_attach+0x74/0xd8
> > [   91.912981]  cgroup_migrate_execute+0x374/0x444
> > [   91.917502]  cgroup_migrate+0x74/0x8c
> > [   91.921155]  cgroup_attach_task+0x114/0x120
> > [   91.925329]  __cgroup_procs_write+0x108/0x230
> > [   91.929677]  cgroup_procs_write+0x1c/0x34
> > [   91.933677]  cgroup_file_write+0xa0/0x1a4
> > [   91.937679]  kernfs_fop_write_iter+0x118/0x1a8
> > [   91.942117]  vfs_write+0x2d0/0x39c
> > [   91.945509]  ksys_write+0x68/0xf4
> > [   91.948814]  __arm64_sys_write+0x1c/0x28
> > [   91.952726]  invoke_syscall+0x48/0x114
> > [   91.956467]  el0_svc_common.constprop.0+0x44/0xe4
> > [   91.961164]  do_el0_svc+0x38/0xa4
> > [   91.964472]  el0_svc+0x2c/0x84
> > [   91.967517]  el0t_64_sync_handler+0xc0/0xc4
> > [   91.971691]  el0t_64_sync+0x190/0x194
> > [  154.987483] rcu: INFO: rcu_preempt self-detected stall on CPU
> > [  154.993218] rcu: 	5-....: (36760 ticks this GP)
> > idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=14547
> > [  155.002862] rcu: 	(t=36870 jiffies g=-119 q=13628 ncpus=8)
> > [  155.008337] Task dump for CPU 0:
> > [  155.011554] task:cpuhp/0         state:R  running task     stack:0     pid:17
> > ppid:2      flags:0x0000000a
> > [  155.021458] Call trace:
> > [  155.023894]  __switch_to+0xe4/0x15c
> > [  155.027376]  0xffff776000196740
> > [  155.030507] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> > [  155.036676] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> > [  155.042323] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > [  155.049274] pc : smp_call_function_single+0x1a4/0x1bc
> > [  155.054316] lr : smp_call_function_single+0x178/0x1bc
> > [  155.059358] sp : ffff80008008b9c0
> > [  155.062662] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27: ffff77600515c858
> > [  155.069788] x26: ffff80008008bbb0 x25: 00000000ffffffff x24: 0000000000000000
> > [  155.076914] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21: ffff80008008bbb0
> > [  155.084041] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18: 0000000000000000
> > [  155.091168] x17: ffffa66484655000 x16: ffff800080028000 x15: 000000079c8b8c8b
> > [  155.098294] x14: 00000000000001c1 x13: 00000000000001c1 x12: 0000000000000000
> > [  155.105421] x11: 0000000000000031 x10: ffff77613ef540c0 x9 : 0000000000000000
> > [  155.112548] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 : ffffd0fcb8de2af4
> > [  155.119675] x5 : 0000000000000001 x4 : 0000000000000040 x3 : ffff80008008ba08
> > [  155.126801] x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
> > [  155.133928] Call trace:
> > [  155.136363]  smp_call_function_single+0x1a4/0x1bc
> > [  155.141059]  perf_cgroup_attach+0x74/0xd8
> > [  155.145060]  cgroup_migrate_execute+0x374/0x444
> > [  155.149581]  cgroup_migrate+0x74/0x8c
> > [  155.153234]  cgroup_attach_task+0x114/0x120
> > [  155.157408]  __cgroup_procs_write+0x108/0x230
> > [  155.161755]  cgroup_procs_write+0x1c/0x34
> > [  155.165756]  cgroup_file_write+0xa0/0x1a4
> > [  155.169757]  kernfs_fop_write_iter+0x118/0x1a8
> > [  155.174195]  vfs_write+0x2d0/0x39c
> > [  155.177587]  ksys_write+0x68/0xf4
> > [  155.180893]  __arm64_sys_write+0x1c/0x28
> > [  155.184805]  invoke_syscall+0x48/0x114
> > [  155.188547]  el0_svc_common.constprop.0+0x44/0xe4
> > [  155.193243]  do_el0_svc+0x38/0xa4
> > [  155.196551]  el0_svc+0x2c/0x84
> > [  155.199595]  el0t_64_sync_handler+0xc0/0xc4
> > [  155.203769]  el0t_64_sync+0x190/0x194
> > 
> > 
> > > 
> > > Mingjia Zhang (3):
> > >   media: mediatek: vcodec: Add capture format to support 10bit tile mode
> > >   media: mediatek: vcodec: Add capture format to support 10bit raster
> > >     mode
> > >   media: mediatek: vcodec: Add driver to support 10bit
> > > 
> > >  .../media/v4l/pixfmt-reserved.rst             |  15 ++
> > >  .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
> > >  .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
> > >  .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140 +++++++++++++++++-
> > >  drivers/media/v4l2-core/v4l2-common.c         |   4 +
> > >  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
> > >  include/uapi/linux/videodev2.h                |   2 +
> > >  7 files changed, 186 insertions(+), 4 deletions(-)
> > > 
> > 

Re: [PATCH 3/3] media: mediatek: vcodec: Add driver to support 10bit

[Nicolas Dufresne]

Le mardi 11 juillet 2023 à 20:57 +0800, Yunfei Dong a écrit :
> From: Mingjia Zhang <mingjia.zhang@mediatek.com>
> 
> Adding to support capture formats V4L2_PIX_FMT_MT2110T and
> V4L2_PIX_FMT_MT2110R for 10bit playback. Need to get the size
> of each plane again when user space setting syntax to get 10bit
> information.
> 
> V4L2_PIX_FMT_MT2110T for AV1/VP9/HEVC.
> V4L2_PIX_FMT_MT2110R for H264.
> 
> Signed-off-by: Mingjia Zhang <mingjia.zhang@mediatek.com>
> Co-developed-by: Yunfei Dong <yunfei.dong@mediatek.com>
> Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> ---
>  .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
>  .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
>  .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140 +++++++++++++++++-
>  3 files changed, 163 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> index 5acb7dff18f2..91ed576d6821 100644
> --- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> +++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> @@ -37,7 +37,9 @@ static bool mtk_vdec_get_cap_fmt(struct mtk_vcodec_dec_ctx *ctx, int format_inde
>  {
>  	const struct mtk_vcodec_dec_pdata *dec_pdata = ctx->dev->vdec_pdata;
>  	const struct mtk_video_fmt *fmt;
> +	struct mtk_q_data *q_data;
>  	int num_frame_count = 0, i;
> +	bool ret = false;
>  
>  	fmt = &dec_pdata->vdec_formats[format_index];
>  	for (i = 0; i < *dec_pdata->num_formats; i++) {
> @@ -47,10 +49,26 @@ static bool mtk_vdec_get_cap_fmt(struct mtk_vcodec_dec_ctx *ctx, int format_inde
>  		num_frame_count++;
>  	}
>  
> -	if (num_frame_count == 1 || fmt->fourcc == V4L2_PIX_FMT_MM21)
> +	if (num_frame_count == 1 || (!ctx->is_10bit_bitstream && fmt->fourcc == V4L2_PIX_FMT_MM21))
>  		return true;
>  
> -	return false;
> +	q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> +	switch (q_data->fmt->fourcc) {
> +	case V4L2_PIX_FMT_H264_SLICE:
> +		if (ctx->is_10bit_bitstream && fmt->fourcc == V4L2_PIX_FMT_MT2110R)
> +			ret = true;
> +		break;
> +	case V4L2_PIX_FMT_VP9_FRAME:
> +	case V4L2_PIX_FMT_AV1_FRAME:
> +	case V4L2_PIX_FMT_HEVC_SLICE:
> +		if (ctx->is_10bit_bitstream && fmt->fourcc == V4L2_PIX_FMT_MT2110T)
> +			ret = true;
> +		break;
> +	default:
> +		break;
> +	}
> +
> +	return ret;
>  }
>  
>  static struct mtk_q_data *mtk_vdec_get_q_data(struct mtk_vcodec_dec_ctx *ctx,
> diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> index c8b4374c5e6c..cd607e90fe9c 100644
> --- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> +++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> @@ -31,6 +31,7 @@ enum mtk_vdec_format_types {
>  	MTK_VDEC_FORMAT_AV1_FRAME = 0x800,
>  	MTK_VDEC_FORMAT_HEVC_FRAME = 0x1000,
>  	MTK_VCODEC_INNER_RACING = 0x20000,
> +	MTK_VDEC_IS_SUPPORT_10BIT = 0x40000,
>  };
>  
>  /*
> @@ -160,6 +161,8 @@ struct mtk_vcodec_dec_pdata {
>   * @hw_id: hardware index used to identify different hardware.
>   *
>   * @msg_queue: msg queue used to store lat buffer information.
> + *
> + * @is_10bit_bitstream: set to true if it's 10bit bitstream
>   */
>  struct mtk_vcodec_dec_ctx {
>  	enum mtk_instance_type type;
> @@ -202,6 +205,8 @@ struct mtk_vcodec_dec_ctx {
>  	int hw_id;
>  
>  	struct vdec_msg_queue msg_queue;
> +
> +	bool is_10bit_bitstream;
>  };
>  
>  /**
> diff --git a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c
> index 99a84c7e1901..cef937fdf462 100644
> --- a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c
> +++ b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_stateless.c
> @@ -200,7 +200,7 @@ static const struct mtk_stateless_control mtk_stateless_controls[] = {
>  
>  #define NUM_CTRLS ARRAY_SIZE(mtk_stateless_controls)
>  
> -static struct mtk_video_fmt mtk_video_formats[7];
> +static struct mtk_video_fmt mtk_video_formats[9];
>  
>  static struct mtk_video_fmt default_out_format;
>  static struct mtk_video_fmt default_cap_format;
> @@ -387,6 +387,134 @@ static int mtk_vdec_flush_decoder(struct mtk_vcodec_dec_ctx *ctx)
>  	return vdec_if_decode(ctx, NULL, NULL, &res_chg);
>  }
>  
> +static int mtk_vcodec_get_pic_info(struct mtk_vcodec_dec_ctx *ctx)
> +{
> +	struct mtk_q_data *q_data;
> +	int ret = 0;
> +
> +	q_data = &ctx->q_data[MTK_Q_DATA_DST];
> +	if (q_data->fmt->num_planes == 1) {
> +		mtk_v4l2_vdec_err(ctx, "[%d]Error!! 10bit mode not support one plane", ctx->id);
> +		return -EINVAL;
> +	}
> +
> +	ctx->capture_fourcc = q_data->fmt->fourcc;
> +	ret = vdec_if_get_param(ctx, GET_PARAM_PIC_INFO, &ctx->picinfo);
> +	if (ret) {
> +		mtk_v4l2_vdec_err(ctx, "[%d]Error!! Get GET_PARAM_PICTURE_INFO Fail", ctx->id);
> +		return ret;
> +	}
> +
> +	ctx->last_decoded_picinfo = ctx->picinfo;
> +
> +	q_data->sizeimage[0] = ctx->picinfo.fb_sz[0];
> +	q_data->bytesperline[0] = ctx->picinfo.buf_w * 5 / 4;
> +
> +	q_data->sizeimage[1] = ctx->picinfo.fb_sz[1];
> +	q_data->bytesperline[1] = ctx->picinfo.buf_w * 5 / 4;
> +
> +	q_data->coded_width = ctx->picinfo.buf_w;
> +	q_data->coded_height = ctx->picinfo.buf_h;
> +	mtk_v4l2_vdec_dbg(1, ctx, "[%d] wxh=%dx%d pic wxh=%dx%d sz[0]=0x%x sz[1]=0x%x",
> +			  ctx->id, ctx->picinfo.buf_w, ctx->picinfo.buf_h,
> +			  ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> +			  q_data->sizeimage[0], q_data->sizeimage[1]);
> +
> +	return ret;
> +}
> +
> +static int mtk_vdec_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	struct mtk_vcodec_dec_ctx *ctx = ctrl_to_dec_ctx(ctrl);
> +	struct v4l2_ctrl_h264_sps *h264;
> +	struct v4l2_ctrl_hevc_sps *h265;
> +	struct v4l2_ctrl_vp9_frame *frame;
> +	struct v4l2_ctrl_av1_sequence *seq;
> +	struct v4l2_ctrl *hdr_ctrl;
> +	const struct mtk_vcodec_dec_pdata *dec_pdata = ctx->dev->vdec_pdata;
> +	const struct mtk_video_fmt *fmt;
> +	int i = 0, ret = 0;
> +
> +	hdr_ctrl = ctrl;
> +	if (!hdr_ctrl || !hdr_ctrl->p_cur.p)
> +		return -EINVAL;
> +
> +	switch (hdr_ctrl->id) {
> +	case V4L2_CID_STATELESS_H264_SPS:
> +		h264 = (struct v4l2_ctrl_h264_sps *)hdr_ctrl->p_new.p;
> +		if (h264->bit_depth_chroma_minus8 == 2 && h264->bit_depth_luma_minus8 == 2) {
> +			ctx->is_10bit_bitstream = true;
> +		} else if (h264->bit_depth_chroma_minus8 != 0 &&
> +			   h264->bit_depth_luma_minus8 != 0) {
> +			mtk_v4l2_vdec_err(ctx, "H264: chroma_minus8:%d, luma_minus8:%d",
> +					  h264->bit_depth_chroma_minus8,
> +					  h264->bit_depth_luma_minus8);
> +			return -EINVAL;
> +		}
> +		break;
> +	case V4L2_CID_STATELESS_HEVC_SPS:
> +		h265 = (struct v4l2_ctrl_hevc_sps *)hdr_ctrl->p_new.p;
> +		if (h265->bit_depth_chroma_minus8 == 2 && h265->bit_depth_luma_minus8 == 2) {
> +			ctx->is_10bit_bitstream = true;
> +		} else if (h265->bit_depth_chroma_minus8 != 0 &&
> +			   h265->bit_depth_luma_minus8 != 0) {
> +			mtk_v4l2_vdec_err(ctx, "HEVC: chroma_minus8:%d, luma_minus8:%d",
> +					  h265->bit_depth_chroma_minus8,
> +					  h265->bit_depth_luma_minus8);
> +			return -EINVAL;
> +		}
> +		break;
> +	case V4L2_CID_STATELESS_VP9_FRAME:
> +		frame = (struct v4l2_ctrl_vp9_frame *)hdr_ctrl->p_new.p;
> +		if (frame->bit_depth == 10) {
> +			ctx->is_10bit_bitstream = true;
> +		} else if (frame->bit_depth != 8) {
> +			mtk_v4l2_vdec_err(ctx, "VP9: bit_depth:%d", frame->bit_depth);
> +			return -EINVAL;
> +		}
> +		break;
> +	case V4L2_CID_STATELESS_AV1_SEQUENCE:
> +		seq = (struct v4l2_ctrl_av1_sequence *)hdr_ctrl->p_new.p;
> +		if (seq->bit_depth == 10) {
> +			ctx->is_10bit_bitstream = true;
> +		} else if (seq->bit_depth != 8) {
> +			mtk_v4l2_vdec_err(ctx, "AV1: bit_depth:%d", seq->bit_depth);
> +			return -EINVAL;
> +		}
> +		break;
> +	default:
> +		mtk_v4l2_vdec_err(ctx, "Not supported ctrl id: 0x%x\n", hdr_ctrl->id);
> +		return -EINVAL;

I can confirm we hit this for every single codec and decoding fails. Written
this way, this code should never have worked, even for 10bit decoding.

> +	}
> +
> +	if (!ctx->is_10bit_bitstream)
> +		return ret;
> +
> +	for (i = 0; i < *dec_pdata->num_formats; i++) {
> +		fmt = &dec_pdata->vdec_formats[i];
> +		if (fmt->fourcc == V4L2_PIX_FMT_MT2110R &&
> +		    hdr_ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> +			ctx->q_data[MTK_Q_DATA_DST].fmt = fmt;
> +			break;
> +		}
> +
> +		if (fmt->fourcc == V4L2_PIX_FMT_MT2110T &&
> +		    (hdr_ctrl->id == V4L2_CID_STATELESS_HEVC_SPS ||
> +		    hdr_ctrl->id == V4L2_CID_STATELESS_VP9_FRAME ||
> +		    hdr_ctrl->id == V4L2_CID_STATELESS_AV1_SEQUENCE)) {
> +			ctx->q_data[MTK_Q_DATA_DST].fmt = fmt;
> +			break;
> +		}
> +	}
> +	ret = mtk_vcodec_get_pic_info(ctx);
> +
> +	return ret;
> +}
> +
> +static const struct v4l2_ctrl_ops mtk_vcodec_dec_ctrl_ops = {
> +	.s_ctrl = mtk_vdec_s_ctrl,
> +};
> +
>  static int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_dec_ctx *ctx)
>  {
>  	unsigned int i;
> @@ -399,7 +527,7 @@ static int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_dec_ctx *ctx)
>  
>  	for (i = 0; i < NUM_CTRLS; i++) {
>  		struct v4l2_ctrl_config cfg = mtk_stateless_controls[i].cfg;
> -
> +		cfg.ops = &mtk_vcodec_dec_ctrl_ops;
>  		v4l2_ctrl_new_custom(&ctx->ctrl_hdl, &cfg, NULL);
>  		if (ctx->ctrl_hdl.error) {
>  			mtk_v4l2_vdec_err(ctx, "Adding control %d failed %d", i,
> @@ -466,6 +594,8 @@ static void mtk_vcodec_add_formats(unsigned int fourcc,
>  		break;
>  	case V4L2_PIX_FMT_MM21:
>  	case V4L2_PIX_FMT_MT21C:
> +	case V4L2_PIX_FMT_MT2110T:
> +	case V4L2_PIX_FMT_MT2110R:
>  		mtk_video_formats[count_formats].fourcc = fourcc;
>  		mtk_video_formats[count_formats].type = MTK_FMT_FRAME;
>  		mtk_video_formats[count_formats].num_planes = 2;
> @@ -491,6 +621,12 @@ static void mtk_vcodec_get_supported_formats(struct mtk_vcodec_dec_ctx *ctx)
>  		mtk_vcodec_add_formats(V4L2_PIX_FMT_MT21C, ctx);
>  		cap_format_count++;
>  	}
> +	if (ctx->dev->dec_capability & MTK_VDEC_IS_SUPPORT_10BIT) {
> +		mtk_vcodec_add_formats(V4L2_PIX_FMT_MT2110T, ctx);
> +		cap_format_count++;
> +		mtk_vcodec_add_formats(V4L2_PIX_FMT_MT2110R, ctx);
> +		cap_format_count++;
> +	}
>  	if (ctx->dev->dec_capability & MTK_VDEC_FORMAT_MM21) {
>  		mtk_vcodec_add_formats(V4L2_PIX_FMT_MM21, ctx);
>  		cap_format_count++;

Re: [PATCH 1/3] media: mediatek: vcodec: Add capture format to support 10bit tile mode

[Nicolas Dufresne]

Le mardi 11 juillet 2023 à 20:57 +0800, Yunfei Dong a écrit :
> From: Mingjia Zhang <mingjia.zhang@mediatek.com>
> 
> Define one uncompressed capture format V4L2_PIX_FMT_MT2110T in order to
> support 10bit for AV1/VP9/HEVC in mt8195.
> 
> Signed-off-by: Mingjia Zhang <mingjia.zhang@mediatek.com>
> Co-developed-by: Yunfei Dong <yunfei.dong@mediatek.com>
> Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> ---
>  Documentation/userspace-api/media/v4l/pixfmt-reserved.rst | 8 ++++++++
>  drivers/media/v4l2-core/v4l2-common.c                     | 2 ++
>  drivers/media/v4l2-core/v4l2-ioctl.c                      | 1 +
>  include/uapi/linux/videodev2.h                            | 1 +
>  4 files changed, 12 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
> index 58f6ae25b2e7..b16a7257580c 100644
> --- a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
> +++ b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
> @@ -275,6 +275,14 @@ please make a proposal on the linux-media mailing list.
>  
>          Decoder's implementation can be found here,
>          `aspeed_codec <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
> +    * .. _V4L2-PIX-FMT-MT2110T:
> +
> +      - ``V4L2_PIX_FMT_MT2110T``
> +      - 'MT2110T'
> +      - Two-planar 10-Bit tile mode YVU420 format used by Mediatek MT8195, MT8188
> +        and more. This format have similitude with ``V4L2_PIX_FMT_MM21``.
> +        It remains an opaque intermediate format and it is used for VP9, AV1
> +        and HEVC.

Documenting uncompressed video formats as "opaque" is always last resort in
V4L2. There is no justification here since this format is already implemented in
software and pending in GStreamer (so its well understood format). I will try
and provide some better doc for you to include.

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3444

>  .. raw:: latex
>  
>      \normalsize
> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> index bee1535b04d3..869fc09a210b 100644
> --- a/drivers/media/v4l2-core/v4l2-common.c
> +++ b/drivers/media/v4l2-core/v4l2-common.c
> @@ -262,6 +262,8 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
>  		{ .format = V4L2_PIX_FMT_VYUY,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
>  		{ .format = V4L2_PIX_FMT_Y212,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 4, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
>  		{ .format = V4L2_PIX_FMT_YUV48_12, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 6, 0, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> +		{ .format = V4L2_PIX_FMT_MT2110T, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 2, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div = { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 2,
> +		  .block_w = { 16, 8, 0, 0 }, .block_h = { 32, 16, 0, 0 }},
>  
>  		/* YUV planar formats */
>  		{ .format = V4L2_PIX_FMT_NV12,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 2 },
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 01ba27f2ef87..f465c0e3d6e3 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1508,6 +1508,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_PIX_FMT_QC10C:	descr = "QCOM Compressed 10-bit Format"; break;
>  		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
>  		case V4L2_PIX_FMT_AV1_FRAME:	descr = "AV1 Frame"; break;
> +		case V4L2_PIX_FMT_MT2110T:	descr = "Mediatek 10bit Tile Mode"; break;
>  		default:
>  			if (fmt->description[0])
>  				return;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 3af6a82d0cad..8c7d71afbdc7 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -796,6 +796,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /* Depth data 16-bit */
>  #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /* Mediatek compressed block mode  */
>  #define V4L2_PIX_FMT_MM21     v4l2_fourcc('M', 'M', '2', '1') /* Mediatek 8-bit block mode, two non-contiguous planes */
> +#define V4L2_PIX_FMT_MT2110T  v4l2_fourcc('M', 'T', '2', 'T') /* Mediatek 10-bit block tile mode */
>  #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>  #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>  #define V4L2_PIX_FMT_HI240    v4l2_fourcc('H', 'I', '2', '4') /* BTTV 8-bit dithered RGB */

Re: [PATCH 3/3] media: mediatek: vcodec: Add driver to support 10bit

[Yunfei Dong (董云飞)]

Hi Nicolas,

Thanks for your suggestion.

On Tue, 2023-07-11 at 16:12 -0400, Nicolas Dufresne wrote:
>  	 
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
>  Le mardi 11 juillet 2023 à 20:57 +0800, Yunfei Dong a écrit :
> > From: Mingjia Zhang <mingjia.zhang@mediatek.com>
> > 
> > Adding to support capture formats V4L2_PIX_FMT_MT2110T and
> > V4L2_PIX_FMT_MT2110R for 10bit playback. Need to get the size
> > of each plane again when user space setting syntax to get 10bit
> > information.
> > 
> > V4L2_PIX_FMT_MT2110T for AV1/VP9/HEVC.
> > V4L2_PIX_FMT_MT2110R for H264.
> > 
> > Signed-off-by: Mingjia Zhang <mingjia.zhang@mediatek.com>
> > Co-developed-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > ---
> >  .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
> >  .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
> >  .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140
> +++++++++++++++++-
> >  3 files changed, 163 insertions(+), 4 deletions(-)
> > 
> > diff --git
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> > index 5acb7dff18f2..91ed576d6821 100644
> > ---
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> > +++
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> > @@ -37,7 +37,9 @@ static bool mtk_vdec_get_cap_fmt(struct
> mtk_vcodec_dec_ctx *ctx, int format_inde
> >  {
> >  const struct mtk_vcodec_dec_pdata *dec_pdata = ctx->dev-
> >vdec_pdata;
> >  const struct mtk_video_fmt *fmt;
> > +struct mtk_q_data *q_data;
> >  int num_frame_count = 0, i;
> > +bool ret = false;
> >  
> >  fmt = &dec_pdata->vdec_formats[format_index];
> >  for (i = 0; i < *dec_pdata->num_formats; i++) {
> > @@ -47,10 +49,26 @@ static bool mtk_vdec_get_cap_fmt(struct
> mtk_vcodec_dec_ctx *ctx, int format_inde
> >  num_frame_count++;
> >  }
> >  
> > -if (num_frame_count == 1 || fmt->fourcc == V4L2_PIX_FMT_MM21)
> > +if (num_frame_count == 1 || (!ctx->is_10bit_bitstream && fmt-
> >fourcc == V4L2_PIX_FMT_MM21))
> >  return true;
> >  
> > -return false;
> > +q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> > +switch (q_data->fmt->fourcc) {
> > +case V4L2_PIX_FMT_H264_SLICE:
> > +if (ctx->is_10bit_bitstream && fmt->fourcc ==
> V4L2_PIX_FMT_MT2110R)
> > +ret = true;
> > +break;
> > +case V4L2_PIX_FMT_VP9_FRAME:
> > +case V4L2_PIX_FMT_AV1_FRAME:
> > +case V4L2_PIX_FMT_HEVC_SLICE:
> > +if (ctx->is_10bit_bitstream && fmt->fourcc ==
> V4L2_PIX_FMT_MT2110T)
> > +ret = true;
> > +break;
> > +default:
> > +break;
> > +}
> > +
> > +return ret;
> >  }
> >  
> >  static struct mtk_q_data *mtk_vdec_get_q_data(struct
> mtk_vcodec_dec_ctx *ctx,
> > diff --git
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> > index c8b4374c5e6c..cd607e90fe9c 100644
> > ---
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> > +++
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> > @@ -31,6 +31,7 @@ enum mtk_vdec_format_types {
> >  MTK_VDEC_FORMAT_AV1_FRAME = 0x800,
> >  MTK_VDEC_FORMAT_HEVC_FRAME = 0x1000,
> >  MTK_VCODEC_INNER_RACING = 0x20000,
> > +MTK_VDEC_IS_SUPPORT_10BIT = 0x40000,
> >  };
> >  
> >  /*
> > @@ -160,6 +161,8 @@ struct mtk_vcodec_dec_pdata {
> >   * @hw_id: hardware index used to identify different hardware.
> >   *
> >   * @msg_queue: msg queue used to store lat buffer information.
> > + *
> > + * @is_10bit_bitstream: set to true if it's 10bit bitstream
> >   */
> >  struct mtk_vcodec_dec_ctx {
> >  enum mtk_instance_type type;
> > @@ -202,6 +205,8 @@ struct mtk_vcodec_dec_ctx {
> >  int hw_id;
> >  
> >  struct vdec_msg_queue msg_queue;
> > +
> > +bool is_10bit_bitstream;
> >  };
> >  
> >  /**
> > diff --git
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_state
> less.c
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_state
> less.c
> > index 99a84c7e1901..cef937fdf462 100644
> > ---
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_state
> less.c
> > +++
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_state
> less.c
> > @@ -200,7 +200,7 @@ static const struct mtk_stateless_control
> mtk_stateless_controls[] = {
> >  
> >  #define NUM_CTRLS ARRAY_SIZE(mtk_stateless_controls)
> >  
> > -static struct mtk_video_fmt mtk_video_formats[7];
> > +static struct mtk_video_fmt mtk_video_formats[9];
> >  
> >  static struct mtk_video_fmt default_out_format;
> >  static struct mtk_video_fmt default_cap_format;
> > @@ -387,6 +387,134 @@ static int mtk_vdec_flush_decoder(struct
> mtk_vcodec_dec_ctx *ctx)
> >  return vdec_if_decode(ctx, NULL, NULL, &res_chg);
> >  }
> >  
> > +static int mtk_vcodec_get_pic_info(struct mtk_vcodec_dec_ctx *ctx)
> > +{
> > +struct mtk_q_data *q_data;
> > +int ret = 0;
> > +
> > +q_data = &ctx->q_data[MTK_Q_DATA_DST];
> > +if (q_data->fmt->num_planes == 1) {
> > +mtk_v4l2_vdec_err(ctx, "[%d]Error!! 10bit mode not support one
> plane", ctx->id);
> > +return -EINVAL;
> > +}
> > +
> > +ctx->capture_fourcc = q_data->fmt->fourcc;
> > +ret = vdec_if_get_param(ctx, GET_PARAM_PIC_INFO, &ctx->picinfo);
> > +if (ret) {
> > +mtk_v4l2_vdec_err(ctx, "[%d]Error!! Get GET_PARAM_PICTURE_INFO
> Fail", ctx->id);
> > +return ret;
> > +}
> > +
> > +ctx->last_decoded_picinfo = ctx->picinfo;
> > +
> > +q_data->sizeimage[0] = ctx->picinfo.fb_sz[0];
> > +q_data->bytesperline[0] = ctx->picinfo.buf_w * 5 / 4;
> > +
> > +q_data->sizeimage[1] = ctx->picinfo.fb_sz[1];
> > +q_data->bytesperline[1] = ctx->picinfo.buf_w * 5 / 4;
> > +
> > +q_data->coded_width = ctx->picinfo.buf_w;
> > +q_data->coded_height = ctx->picinfo.buf_h;
> > +mtk_v4l2_vdec_dbg(1, ctx, "[%d] wxh=%dx%d pic wxh=%dx%d sz[0]=0x%x
> sz[1]=0x%x",
> > +  ctx->id, ctx->picinfo.buf_w, ctx->picinfo.buf_h,
> > +  ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> > +  q_data->sizeimage[0], q_data->sizeimage[1]);
> > +
> > +return ret;
> > +}
> > +
> > +static int mtk_vdec_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +struct mtk_vcodec_dec_ctx *ctx = ctrl_to_dec_ctx(ctrl);
> > +struct v4l2_ctrl_h264_sps *h264;
> > +struct v4l2_ctrl_hevc_sps *h265;
> > +struct v4l2_ctrl_vp9_frame *frame;
> > +struct v4l2_ctrl_av1_sequence *seq;
> > +struct v4l2_ctrl *hdr_ctrl;
> > +const struct mtk_vcodec_dec_pdata *dec_pdata = ctx->dev-
> >vdec_pdata;
> > +const struct mtk_video_fmt *fmt;
> > +int i = 0, ret = 0;
> > +
> > +hdr_ctrl = ctrl;
> > +if (!hdr_ctrl || !hdr_ctrl->p_cur.p)
> > +return -EINVAL;
> > +
> > +switch (hdr_ctrl->id) {
> > +case V4L2_CID_STATELESS_H264_SPS:
> > +h264 = (struct v4l2_ctrl_h264_sps *)hdr_ctrl->p_new.p;
> > +if (h264->bit_depth_chroma_minus8 == 2 && h264-
> >bit_depth_luma_minus8 == 2) {
> > +ctx->is_10bit_bitstream = true;
> > +} else if (h264->bit_depth_chroma_minus8 != 0 &&
> > +   h264->bit_depth_luma_minus8 != 0) {
> > +mtk_v4l2_vdec_err(ctx, "H264: chroma_minus8:%d, luma_minus8:%d",
> > +  h264->bit_depth_chroma_minus8,
> > +  h264->bit_depth_luma_minus8);
> > +return -EINVAL;
> > +}
> > +break;
> > +case V4L2_CID_STATELESS_HEVC_SPS:
> > +h265 = (struct v4l2_ctrl_hevc_sps *)hdr_ctrl->p_new.p;
> > +if (h265->bit_depth_chroma_minus8 == 2 && h265-
> >bit_depth_luma_minus8 == 2) {
> > +ctx->is_10bit_bitstream = true;
> > +} else if (h265->bit_depth_chroma_minus8 != 0 &&
> > +   h265->bit_depth_luma_minus8 != 0) {
> > +mtk_v4l2_vdec_err(ctx, "HEVC: chroma_minus8:%d, luma_minus8:%d",
> > +  h265->bit_depth_chroma_minus8,
> > +  h265->bit_depth_luma_minus8);
> > +return -EINVAL;
> > +}
> > +break;
> > +case V4L2_CID_STATELESS_VP9_FRAME:
> > +frame = (struct v4l2_ctrl_vp9_frame *)hdr_ctrl->p_new.p;
> > +if (frame->bit_depth == 10) {
> > +ctx->is_10bit_bitstream = true;
> > +} else if (frame->bit_depth != 8) {
> > +mtk_v4l2_vdec_err(ctx, "VP9: bit_depth:%d", frame->bit_depth);
> > +return -EINVAL;
> > +}
> > +break;
> > +case V4L2_CID_STATELESS_AV1_SEQUENCE:
> > +seq = (struct v4l2_ctrl_av1_sequence *)hdr_ctrl->p_new.p;
> > +if (seq->bit_depth == 10) {
> > +ctx->is_10bit_bitstream = true;
> > +} else if (seq->bit_depth != 8) {
> > +mtk_v4l2_vdec_err(ctx, "AV1: bit_depth:%d", seq->bit_depth);
> > +return -EINVAL;
> > +}
> > +break;
> > +default:
> > +mtk_v4l2_vdec_err(ctx, "Not supported ctrl id: 0x%x\n", hdr_ctrl-
> >id);
> > +return -EINVAL;
> 
> I can confirm we hit this for every single codec and decoding fails.
> Written
> this way, this code should never have worked, even for 10bit
> decoding.
> 
Removed this line in local test, forgot PPS and other syntax. Will fix
in next patch.

Will return 0 when in default case, and replace mtk_v4l2_vdec_err with
mtk_v4l2_vdec_dbg.

Best Regards,
Yunfei Dong
> > +}
> > +
> > +if (!ctx->is_10bit_bitstream)
> > +return ret;
> > +
> > +for (i = 0; i < *dec_pdata->num_formats; i++) {
> > +fmt = &dec_pdata->vdec_formats[i];
> > +if (fmt->fourcc == V4L2_PIX_FMT_MT2110R &&
> > +    hdr_ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> > +ctx->q_data[MTK_Q_DATA_DST].fmt = fmt;
> > +break;
> > +}
> > +
> > +if (fmt->fourcc == V4L2_PIX_FMT_MT2110T &&
> > +    (hdr_ctrl->id == V4L2_CID_STATELESS_HEVC_SPS ||
> > +    hdr_ctrl->id == V4L2_CID_STATELESS_VP9_FRAME ||
> > +    hdr_ctrl->id == V4L2_CID_STATELESS_AV1_SEQUENCE)) {
> > +ctx->q_data[MTK_Q_DATA_DST].fmt = fmt;
> > +break;
> > +}
> > +}
> > +ret = mtk_vcodec_get_pic_info(ctx);
> > +
> > +return ret;
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_vcodec_dec_ctrl_ops = {
> > +.s_ctrl = mtk_vdec_s_ctrl,
> > +};
> > +
> >  static int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_dec_ctx
> *ctx)
> >  {
> >  unsigned int i;
> > @@ -399,7 +527,7 @@ static int mtk_vcodec_dec_ctrls_setup(struct
> mtk_vcodec_dec_ctx *ctx)
> >  
> >  for (i = 0; i < NUM_CTRLS; i++) {
> >  struct v4l2_ctrl_config cfg = mtk_stateless_controls[i].cfg;
> > -
> > +cfg.ops = &mtk_vcodec_dec_ctrl_ops;
> >  v4l2_ctrl_new_custom(&ctx->ctrl_hdl, &cfg, NULL);
> >  if (ctx->ctrl_hdl.error) {
> >  mtk_v4l2_vdec_err(ctx, "Adding control %d failed %d", i,
> > @@ -466,6 +594,8 @@ static void mtk_vcodec_add_formats(unsigned int
> fourcc,
> >  break;
> >  case V4L2_PIX_FMT_MM21:
> >  case V4L2_PIX_FMT_MT21C:
> > +case V4L2_PIX_FMT_MT2110T:
> > +case V4L2_PIX_FMT_MT2110R:
> >  mtk_video_formats[count_formats].fourcc = fourcc;
> >  mtk_video_formats[count_formats].type = MTK_FMT_FRAME;
> >  mtk_video_formats[count_formats].num_planes = 2;
> > @@ -491,6 +621,12 @@ static void
> mtk_vcodec_get_supported_formats(struct mtk_vcodec_dec_ctx *ctx)
> >  mtk_vcodec_add_formats(V4L2_PIX_FMT_MT21C, ctx);
> >  cap_format_count++;
> >  }
> > +if (ctx->dev->dec_capability & MTK_VDEC_IS_SUPPORT_10BIT) {
> > +mtk_vcodec_add_formats(V4L2_PIX_FMT_MT2110T, ctx);
> > +cap_format_count++;
> > +mtk_vcodec_add_formats(V4L2_PIX_FMT_MT2110R, ctx);
> > +cap_format_count++;
> > +}
> >  if (ctx->dev->dec_capability & MTK_VDEC_FORMAT_MM21) {
> >  mtk_vcodec_add_formats(V4L2_PIX_FMT_MM21, ctx);
> >  cap_format_count++;
> 

Re: [PATCH 0/3] media: mediatek: vcodec: Add driver to support 10bit

[Yunfei Dong (董云飞)]

Hi Nicolas,

Thanks for your suggestion.

On Tue, 2023-07-11 at 15:15 -0400, Nicolas Dufresne wrote:
>  	 
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
>  Hi,
> 
> phew things missing in this cover letter,
> 
> Le mardi 11 juillet 2023 à 20:57 +0800, Yunfei Dong a écrit :
> > Define two capture formats V4L2_PIX_FMT_MT2110R and
> > V4L2_PIX_FMT_MT2110T to support 10bit in mt8195, mt8199 and more.
> > Getting the size of each plane again when userspace set 10bit
> > syntax to driver.
> > 
> > V4L2_PIX_FMT_MT2110R is used for H264, and V4L2_PIX_FMT_MT2110T
> > is used for AV1/VP9/HEVC.
> > 
> > patch 1 Add driver to support 10bit
> > patch 2 Add capture format V4L2_PIX_FMT_MT2110T to support 10bit
> tile mode
> > patch 3 Add capture format V4L2_PIX_FMT_MT2110R to support 10bit
> raster mode
> > ---
> > - send the first version v1:
> > - Run 10bit VP9/AV1 fluster test pass.
> 
> Please at least provide the score before and after these changes.
> Full report is
> also nice. You didn't explain why H.264 is not tested.
> 
> > - Will return error when the 10bit parameter no correctly in
> function mtk_vdec_s_ctrl.
> > ---
> 
> This cover letter is missing v4l2-compliance report. This is needed
> whenever
> format mechanism is modified in a driver (not just for new drivers).
> Please add
> and make sure there is not regression too.
> 
Forgot to test v4l2-compliance, will add in next patch.

The flush test not include H264 for test fail, according to mingjia's
information, maybe gstreamer set wrong format. H264 fluster can test
pass when kernel driver force h264 in raster mode.
> > 
> > Reference series:
> > [1]: this series depends on v6 which is send by Yunfei Dong.
> >      message-id: 20230704131349.8354-1-yunfei.dong@mediatek.com
> 
> Its seems like 6.5.0-rc1 with the depedency and this patchset does
> not boot on
> MT8195 Chromebooks. Which paltform has this been validated on ?
> 
> 
Test the patch series "separate encoder and decoder" and "Add driver to
support 10bit" in mt8188 and mt8195, the driver can work well with tast
test and Youtube. Not found the crash log. Maybe the crash log isn't
related with decoder?

Best Regards,
Yunfei Dong
> For the record:
> 
> 
> [   13.286252] platform 1c015000.dp-intf: deferred probe pending
> [   13.292007] platform 1c113000.dp-intf: deferred probe pending
> [   28.523484] rcu: INFO: rcu_preempt self-detected stall on CPU
> [   28.529231] rcu: 5-....: (5250 ticks this GP)
> idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=2471
> [   28.538706] rcu: (t=5254 jiffies g=-119 q=13320 ncpus=8)
> [   28.544095] Task dump for CPU 0:
> [   28.547313] task:cpuhp/0         state:R  running
> task     stack:0     pid:17
> ppid:2      flags:0x0000000a
> [   28.557221] Call trace:
> [   28.559658]  __switch_to+0xe4/0x15c
> [   28.563147]  0xffff776000196740
> [   28.566282] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> [   28.572453] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> [   28.578101] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS
> BTYPE=--)
> [   28.585053] pc : smp_call_function_single+0x1a4/0x1bc
> [   28.590098] lr : smp_call_function_single+0x178/0x1bc
> [   28.595140] sp : ffff80008008b9c0
> [   28.598444] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27:
> ffff77600515c858
> [   28.605572] x26: ffff80008008bbb0 x25: 00000000ffffffff x24:
> 0000000000000000
> [   28.612699] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21:
> ffff80008008bbb0
> [   28.619825] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18:
> 0000000000000000
> [   28.626952] x17: ffffa66484655000 x16: ffff800080028000 x15:
> 000000079c8b8c8b
> [   28.634078] x14: 00000000000001c1 x13: 00000000000001c1 x12:
> 0000000000000000
> [   28.641204] x11: 0000000000000031 x10: ffff77613ef540c0 x9 :
> 0000000000000000
> [   28.648331] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 :
> ffffd0fcb8de2af4
> [   28.655458] x5 : 0000000000000001 x4 : 0000000000000040 x3 :
> ffff80008008ba08
> [   28.662585] x2 : 0000000000000000 x1 : 0000000000000011 x0 :
> 0000000000000000
> [   28.669711] Call trace:
> [   28.672148]  smp_call_function_single+0x1a4/0x1bc
> [   28.676843]  perf_cgroup_attach+0x74/0xd8
> [   28.680847]  cgroup_migrate_execute+0x374/0x444
> [   28.685368]  cgroup_migrate+0x74/0x8c
> [   28.689021]  cgroup_attach_task+0x114/0x120
> [   28.693195]  __cgroup_procs_write+0x108/0x230
> [   28.697543]  cgroup_procs_write+0x1c/0x34
> [   28.701543]  cgroup_file_write+0xa0/0x1a4
> [   28.705545]  kernfs_fop_write_iter+0x118/0x1a8
> [   28.709983]  vfs_write+0x2d0/0x39c
> [   28.713376]  ksys_write+0x68/0xf4
> [   28.716682]  __arm64_sys_write+0x1c/0x28
> [   28.720594]  invoke_syscall+0x48/0x114
> [   28.724337]  el0_svc_common.constprop.0+0x44/0xe4
> [   28.729034]  do_el0_svc+0x38/0xa4
> [   28.732341]  el0_svc+0x2c/0x84
> [   28.735386]  el0t_64_sync_handler+0xc0/0xc4
> [   28.739561]  el0t_64_sync+0x190/0x194
> [   33.759553] vproc2: disabling
> [   33.762551] vproc1: disabling
> [   33.765548] vaud18: disabling
> [   33.768760] va09: disabling
> [   33.771599] vsram_md: disabling
> [   91.755483] rcu: INFO: rcu_preempt self-detected stall on CPU
> [   91.761220] rcu: 5-....: (21005 ticks this GP)
> idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=8569
> [   91.770778] rcu: (t=21062 jiffies g=-119 q=13628 ncpus=8)
> [   91.776253] Task dump for CPU 0:
> [   91.779471] task:cpuhp/0         state:R  running
> task     stack:0     pid:17
> ppid:2      flags:0x0000000a
> [   91.789376] Call trace:
> [   91.791812]  __switch_to+0xe4/0x15c
> [   91.795294]  0xffff776000196740
> [   91.798426] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> [   91.804597] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> [   91.810244] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS
> BTYPE=--)
> [   91.817195] pc : smp_call_function_single+0x1a4/0x1bc
> [   91.822237] lr : smp_call_function_single+0x178/0x1bc
> [   91.827278] sp : ffff80008008b9c0
> [   91.830582] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27:
> ffff77600515c858
> [   91.837709] x26: ffff80008008bbb0 x25: 00000000ffffffff x24:
> 0000000000000000
> [   91.844835] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21:
> ffff80008008bbb0
> [   91.851962] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18:
> 0000000000000000
> [   91.859089] x17: ffffa66484655000 x16: ffff800080028000 x15:
> 000000079c8b8c8b
> [   91.866215] x14: 00000000000001c1 x13: 00000000000001c1 x12:
> 0000000000000000
> [   91.873342] x11: 0000000000000031 x10: ffff77613ef540c0 x9 :
> 0000000000000000
> [   91.880468] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 :
> ffffd0fcb8de2af4
> [   91.887595] x5 : 0000000000000001 x4 : 0000000000000040 x3 :
> ffff80008008ba08
> [   91.894721] x2 : 0000000000000000 x1 : 0000000000000011 x0 :
> 0000000000000000
> [   91.901848] Call trace:
> [   91.904284]  smp_call_function_single+0x1a4/0x1bc
> [   91.908979]  perf_cgroup_attach+0x74/0xd8
> [   91.912981]  cgroup_migrate_execute+0x374/0x444
> [   91.917502]  cgroup_migrate+0x74/0x8c
> [   91.921155]  cgroup_attach_task+0x114/0x120
> [   91.925329]  __cgroup_procs_write+0x108/0x230
> [   91.929677]  cgroup_procs_write+0x1c/0x34
> [   91.933677]  cgroup_file_write+0xa0/0x1a4
> [   91.937679]  kernfs_fop_write_iter+0x118/0x1a8
> [   91.942117]  vfs_write+0x2d0/0x39c
> [   91.945509]  ksys_write+0x68/0xf4
> [   91.948814]  __arm64_sys_write+0x1c/0x28
> [   91.952726]  invoke_syscall+0x48/0x114
> [   91.956467]  el0_svc_common.constprop.0+0x44/0xe4
> [   91.961164]  do_el0_svc+0x38/0xa4
> [   91.964472]  el0_svc+0x2c/0x84
> [   91.967517]  el0t_64_sync_handler+0xc0/0xc4
> [   91.971691]  el0t_64_sync+0x190/0x194
> [  154.987483] rcu: INFO: rcu_preempt self-detected stall on CPU
> [  154.993218] rcu: 5-....: (36760 ticks this GP)
> idle=51c4/1/0x4000000000000000 softirq=1434/1447 fqs=14547
> [  155.002862] rcu: (t=36870 jiffies g=-119 q=13628 ncpus=8)
> [  155.008337] Task dump for CPU 0:
> [  155.011554] task:cpuhp/0         state:R  running
> task     stack:0     pid:17
> ppid:2      flags:0x0000000a
> [  155.021458] Call trace:
> [  155.023894]  __switch_to+0xe4/0x15c
> [  155.027376]  0xffff776000196740
> [  155.030507] CPU: 5 PID: 1 Comm: systemd Not tainted 6.5.0-rc1+ #36
> [  155.036676] Hardware name: Acer Tomato (rev3 - 4) board (DT)
> [  155.042323] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS
> BTYPE=--)
> [  155.049274] pc : smp_call_function_single+0x1a4/0x1bc
> [  155.054316] lr : smp_call_function_single+0x178/0x1bc
> [  155.059358] sp : ffff80008008b9c0
> [  155.062662] x29: ffff80008008b9c0 x28: ffff80008008bb90 x27:
> ffff77600515c858
> [  155.069788] x26: ffff80008008bbb0 x25: 00000000ffffffff x24:
> 0000000000000000
> [  155.076914] x23: 00000000fffffff5 x22: ffffd0fcb8de9b50 x21:
> ffff80008008bbb0
> [  155.084041] x20: ffffd0fcb8de2af4 x19: ffff80008008ba00 x18:
> 0000000000000000
> [  155.091168] x17: ffffa66484655000 x16: ffff800080028000 x15:
> 000000079c8b8c8b
> [  155.098294] x14: 00000000000001c1 x13: 00000000000001c1 x12:
> 0000000000000000
> [  155.105421] x11: 0000000000000031 x10: ffff77613ef540c0 x9 :
> 0000000000000000
> [  155.112548] x8 : ffff77613ef54140 x7 : 0000000000000005 x6 :
> ffffd0fcb8de2af4
> [  155.119675] x5 : 0000000000000001 x4 : 0000000000000040 x3 :
> ffff80008008ba08
> [  155.126801] x2 : 0000000000000000 x1 : 0000000000000011 x0 :
> 0000000000000000
> [  155.133928] Call trace:
> [  155.136363]  smp_call_function_single+0x1a4/0x1bc
> [  155.141059]  perf_cgroup_attach+0x74/0xd8
> [  155.145060]  cgroup_migrate_execute+0x374/0x444
> [  155.149581]  cgroup_migrate+0x74/0x8c
> [  155.153234]  cgroup_attach_task+0x114/0x120
> [  155.157408]  __cgroup_procs_write+0x108/0x230
> [  155.161755]  cgroup_procs_write+0x1c/0x34
> [  155.165756]  cgroup_file_write+0xa0/0x1a4
> [  155.169757]  kernfs_fop_write_iter+0x118/0x1a8
> [  155.174195]  vfs_write+0x2d0/0x39c
> [  155.177587]  ksys_write+0x68/0xf4
> [  155.180893]  __arm64_sys_write+0x1c/0x28
> [  155.184805]  invoke_syscall+0x48/0x114
> [  155.188547]  el0_svc_common.constprop.0+0x44/0xe4
> [  155.193243]  do_el0_svc+0x38/0xa4
> [  155.196551]  el0_svc+0x2c/0x84
> [  155.199595]  el0t_64_sync_handler+0xc0/0xc4
> [  155.203769]  el0t_64_sync+0x190/0x194
> 
> 
> > 
> > Mingjia Zhang (3):
> >   media: mediatek: vcodec: Add capture format to support 10bit tile
> mode
> >   media: mediatek: vcodec: Add capture format to support 10bit
> raster
> >     mode
> >   media: mediatek: vcodec: Add driver to support 10bit
> > 
> >  .../media/v4l/pixfmt-reserved.rst             |  15 ++
> >  .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
> >  .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
> >  .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140
> +++++++++++++++++-
> >  drivers/media/v4l2-core/v4l2-common.c         |   4 +
> >  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
> >  include/uapi/linux/videodev2.h                |   2 +
> >  7 files changed, 186 insertions(+), 4 deletions(-)
> > 
> 
> 

Re: [PATCH 1/3] media: mediatek: vcodec: Add capture format to support 10bit tile mode

[Yunfei Dong (董云飞)]

Hi Nicolas,

Thanks for your suggestion.

On Tue, 2023-07-11 at 16:16 -0400, Nicolas Dufresne wrote:
>  	 
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
>  Le mardi 11 juillet 2023 à 20:57 +0800, Yunfei Dong a écrit :
> > From: Mingjia Zhang <mingjia.zhang@mediatek.com>
> > 
> > Define one uncompressed capture format V4L2_PIX_FMT_MT2110T in
> order to
> > support 10bit for AV1/VP9/HEVC in mt8195.
> > 
> > Signed-off-by: Mingjia Zhang <mingjia.zhang@mediatek.com>
> > Co-developed-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > ---
> >  Documentation/userspace-api/media/v4l/pixfmt-reserved.rst | 8
> ++++++++
> >  drivers/media/v4l2-core/v4l2-common.c                     | 2 ++
> >  drivers/media/v4l2-core/v4l2-ioctl.c                      | 1 +
> >  include/uapi/linux/videodev2.h                            | 1 +
> >  4 files changed, 12 insertions(+)
> > 
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-
> reserved.rst b/Documentation/userspace-api/media/v4l/pixfmt-
> reserved.rst
> > index 58f6ae25b2e7..b16a7257580c 100644
> > --- a/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
> > @@ -275,6 +275,14 @@ please make a proposal on the linux-media
> mailing list.
> >  
> >          Decoder's implementation can be found here,
> >          `aspeed_codec <
> https://github.com/AspeedTech-BMC/aspeed_codec/>`__
> > +    * .. _V4L2-PIX-FMT-MT2110T:
> > +
> > +      - ``V4L2_PIX_FMT_MT2110T``
> > +      - 'MT2110T'
> > +      - Two-planar 10-Bit tile mode YVU420 format used by Mediatek
> MT8195, MT8188
> > +        and more. This format have similitude with
> ``V4L2_PIX_FMT_MM21``.
> > +        It remains an opaque intermediate format and it is used
> for VP9, AV1
> > +        and HEVC.
> 
> Documenting uncompressed video formats as "opaque" is always last
> resort in
> V4L2. There is no justification here since this format is already
> implemented in
> software and pending in GStreamer (so its well understood format). I
> will try
> and provide some better doc for you to include.
> 
> 
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3444
> 
It's much better if you can provider the better doc to me.
Thanks you again.

Best Regards,
Yunfei Dong

> >  .. raw:: latex
> >  
> >      \normalsize
> > diff --git a/drivers/media/v4l2-core/v4l2-common.c
> b/drivers/media/v4l2-core/v4l2-common.c
> > index bee1535b04d3..869fc09a210b 100644
> > --- a/drivers/media/v4l2-core/v4l2-common.c
> > +++ b/drivers/media/v4l2-core/v4l2-common.c
> > @@ -262,6 +262,8 @@ const struct v4l2_format_info
> *v4l2_format_info(u32 format)
> >  { .format = V4L2_PIX_FMT_VYUY,    .pixel_enc = V4L2_PIXEL_ENC_YUV,
> .mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .bpp_div =
> { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
> >  { .format = V4L2_PIX_FMT_Y212,    .pixel_enc = V4L2_PIXEL_ENC_YUV,
> .mem_planes = 1, .comp_planes = 1, .bpp = { 4, 0, 0, 0 }, .bpp_div =
> { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
> >  { .format = V4L2_PIX_FMT_YUV48_12, .pixel_enc =
> V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 1, .bpp = { 6, 0,
> 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
> > +{ .format = V4L2_PIX_FMT_MT2110T, .pixel_enc = V4L2_PIXEL_ENC_YUV,
> .mem_planes = 2, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div =
> { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 2,
> > +  .block_w = { 16, 8, 0, 0 }, .block_h = { 32, 16, 0, 0 }},
> >  
> >  /* YUV planar formats */
> >  { .format = V4L2_PIX_FMT_NV12,    .pixel_enc = V4L2_PIXEL_ENC_YUV,
> .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div =
> { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 2 },
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 01ba27f2ef87..f465c0e3d6e3 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1508,6 +1508,7 @@ static void v4l_fill_fmtdesc(struct
> v4l2_fmtdesc *fmt)
> >  case V4L2_PIX_FMT_QC10C:descr = "QCOM Compressed 10-bit Format";
> break;
> >  case V4L2_PIX_FMT_AJPG:descr = "Aspeed JPEG"; break;
> >  case V4L2_PIX_FMT_AV1_FRAME:descr = "AV1 Frame"; break;
> > +case V4L2_PIX_FMT_MT2110T:descr = "Mediatek 10bit Tile Mode";
> break;
> >  default:
> >  if (fmt->description[0])
> >  return;
> > diff --git a/include/uapi/linux/videodev2.h
> b/include/uapi/linux/videodev2.h
> > index 3af6a82d0cad..8c7d71afbdc7 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -796,6 +796,7 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /*
> Depth data 16-bit */
> >  #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /*
> Mediatek compressed block mode  */
> >  #define V4L2_PIX_FMT_MM21     v4l2_fourcc('M', 'M', '2', '1') /*
> Mediatek 8-bit block mode, two non-contiguous planes */
> > +#define V4L2_PIX_FMT_MT2110T  v4l2_fourcc('M', 'T', '2', 'T') /*
> Mediatek 10-bit block tile mode */
> >  #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /*
> Intel Planar Greyscale 10-bit and Depth 16-bit */
> >  #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /*
> Intel 4-bit packed depth confidence information */
> >  #define V4L2_PIX_FMT_HI240    v4l2_fourcc('H', 'I', '2', '4') /*
> BTTV 8-bit dithered RGB */
> 

Re: [PATCH 3/3] media: mediatek: vcodec: Add driver to support 10bit

[Yunfei Dong (董云飞)]

Hi Nicolas,

Thanks for your suggestion.

On Tue, 2023-07-11 at 12:53 -0400, Nicolas Dufresne wrote:
>  	 
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
>  Hi Yunfei,
> 
> this is a partial drive by review, I'll do more testing and more
> review soon.
> 
> Le mardi 11 juillet 2023 à 20:57 +0800, Yunfei Dong a écrit :
> > From: Mingjia Zhang <mingjia.zhang@mediatek.com>
> > 
> > Adding to support capture formats V4L2_PIX_FMT_MT2110T and
> > V4L2_PIX_FMT_MT2110R for 10bit playback. Need to get the size
> > of each plane again when user space setting syntax to get 10bit
> > information.
> > 
> > V4L2_PIX_FMT_MT2110T for AV1/VP9/HEVC.
> > V4L2_PIX_FMT_MT2110R for H264.
> > 
> > Signed-off-by: Mingjia Zhang <mingjia.zhang@mediatek.com>
> > Co-developed-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > ---
> >  .../mediatek/vcodec/decoder/mtk_vcodec_dec.c  |  22 ++-
> >  .../vcodec/decoder/mtk_vcodec_dec_drv.h       |   5 +
> >  .../vcodec/decoder/mtk_vcodec_dec_stateless.c | 140
> +++++++++++++++++-
> >  3 files changed, 163 insertions(+), 4 deletions(-)
> > 
> > diff --git
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> > index 5acb7dff18f2..91ed576d6821 100644
> > ---
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> > +++
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec.c
> > @@ -37,7 +37,9 @@ static bool mtk_vdec_get_cap_fmt(struct
> mtk_vcodec_dec_ctx *ctx, int format_inde
> >  {
> >  const struct mtk_vcodec_dec_pdata *dec_pdata = ctx->dev-
> >vdec_pdata;
> >  const struct mtk_video_fmt *fmt;
> > +struct mtk_q_data *q_data;
> >  int num_frame_count = 0, i;
> > +bool ret = false;
> >  
> >  fmt = &dec_pdata->vdec_formats[format_index];
> >  for (i = 0; i < *dec_pdata->num_formats; i++) {
> > @@ -47,10 +49,26 @@ static bool mtk_vdec_get_cap_fmt(struct
> mtk_vcodec_dec_ctx *ctx, int format_inde
> >  num_frame_count++;
> >  }
> >  
> > -if (num_frame_count == 1 || fmt->fourcc == V4L2_PIX_FMT_MM21)
> > +if (num_frame_count == 1 || (!ctx->is_10bit_bitstream && fmt-
> >fourcc == V4L2_PIX_FMT_MM21))
> >  return true;
> >  
> > -return false;
> > +q_data = &ctx->q_data[MTK_Q_DATA_SRC];
> > +switch (q_data->fmt->fourcc) {
> > +case V4L2_PIX_FMT_H264_SLICE:
> > +if (ctx->is_10bit_bitstream && fmt->fourcc ==
> V4L2_PIX_FMT_MT2110R)
> > +ret = true;
> > +break;
> > +case V4L2_PIX_FMT_VP9_FRAME:
> > +case V4L2_PIX_FMT_AV1_FRAME:
> > +case V4L2_PIX_FMT_HEVC_SLICE:
> > +if (ctx->is_10bit_bitstream && fmt->fourcc ==
> V4L2_PIX_FMT_MT2110T)
> > +ret = true;
> > +break;
> > +default:
> > +break;
> > +}
> > +
> > +return ret;
> >  }
> >  
> >  static struct mtk_q_data *mtk_vdec_get_q_data(struct
> mtk_vcodec_dec_ctx *ctx,
> > diff --git
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> > index c8b4374c5e6c..cd607e90fe9c 100644
> > ---
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> > +++
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_drv.h
> > @@ -31,6 +31,7 @@ enum mtk_vdec_format_types {
> >  MTK_VDEC_FORMAT_AV1_FRAME = 0x800,
> >  MTK_VDEC_FORMAT_HEVC_FRAME = 0x1000,
> >  MTK_VCODEC_INNER_RACING = 0x20000,
> > +MTK_VDEC_IS_SUPPORT_10BIT = 0x40000,
> >  };
> >  
> >  /*
> > @@ -160,6 +161,8 @@ struct mtk_vcodec_dec_pdata {
> >   * @hw_id: hardware index used to identify different hardware.
> >   *
> >   * @msg_queue: msg queue used to store lat buffer information.
> > + *
> > + * @is_10bit_bitstream: set to true if it's 10bit bitstream
> >   */
> >  struct mtk_vcodec_dec_ctx {
> >  enum mtk_instance_type type;
> > @@ -202,6 +205,8 @@ struct mtk_vcodec_dec_ctx {
> >  int hw_id;
> >  
> >  struct vdec_msg_queue msg_queue;
> > +
> > +bool is_10bit_bitstream;
> >  };
> >  
> >  /**
> > diff --git
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_state
> less.c
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_state
> less.c
> > index 99a84c7e1901..cef937fdf462 100644
> > ---
> a/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_state
> less.c
> > +++
> b/drivers/media/platform/mediatek/vcodec/decoder/mtk_vcodec_dec_state
> less.c
> > @@ -200,7 +200,7 @@ static const struct mtk_stateless_control
> mtk_stateless_controls[] = {
> >  
> >  #define NUM_CTRLS ARRAY_SIZE(mtk_stateless_controls)
> >  
> > -static struct mtk_video_fmt mtk_video_formats[7];
> > +static struct mtk_video_fmt mtk_video_formats[9];
> >  
> >  static struct mtk_video_fmt default_out_format;
> >  static struct mtk_video_fmt default_cap_format;
> > @@ -387,6 +387,134 @@ static int mtk_vdec_flush_decoder(struct
> mtk_vcodec_dec_ctx *ctx)
> >  return vdec_if_decode(ctx, NULL, NULL, &res_chg);
> >  }
> >  
> > +static int mtk_vcodec_get_pic_info(struct mtk_vcodec_dec_ctx *ctx)
> > +{
> > +struct mtk_q_data *q_data;
> > +int ret = 0;
> > +
> > +q_data = &ctx->q_data[MTK_Q_DATA_DST];
> > +if (q_data->fmt->num_planes == 1) {
> > +mtk_v4l2_vdec_err(ctx, "[%d]Error!! 10bit mode not support one
> plane", ctx->id);
> > +return -EINVAL;
> > +}
> > +
> > +ctx->capture_fourcc = q_data->fmt->fourcc;
> > +ret = vdec_if_get_param(ctx, GET_PARAM_PIC_INFO, &ctx->picinfo);
> > +if (ret) {
> > +mtk_v4l2_vdec_err(ctx, "[%d]Error!! Get GET_PARAM_PICTURE_INFO
> Fail", ctx->id);
> > +return ret;
> > +}
> > +
> > +ctx->last_decoded_picinfo = ctx->picinfo;
> > +
> > +q_data->sizeimage[0] = ctx->picinfo.fb_sz[0];
> > +q_data->bytesperline[0] = ctx->picinfo.buf_w * 5 / 4;
> > +
> > +q_data->sizeimage[1] = ctx->picinfo.fb_sz[1];
> > +q_data->bytesperline[1] = ctx->picinfo.buf_w * 5 / 4;
> > +
> > +q_data->coded_width = ctx->picinfo.buf_w;
> > +q_data->coded_height = ctx->picinfo.buf_h;
> > +mtk_v4l2_vdec_dbg(1, ctx, "[%d] wxh=%dx%d pic wxh=%dx%d sz[0]=0x%x
> sz[1]=0x%x",
> > +  ctx->id, ctx->picinfo.buf_w, ctx->picinfo.buf_h,
> > +  ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> > +  q_data->sizeimage[0], q_data->sizeimage[1]);
> > +
> > +return ret;
> > +}
> > +
> > +static int mtk_vdec_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > +struct mtk_vcodec_dec_ctx *ctx = ctrl_to_dec_ctx(ctrl);
> > +struct v4l2_ctrl_h264_sps *h264;
> > +struct v4l2_ctrl_hevc_sps *h265;
> > +struct v4l2_ctrl_vp9_frame *frame;
> > +struct v4l2_ctrl_av1_sequence *seq;
> > +struct v4l2_ctrl *hdr_ctrl;
> > +const struct mtk_vcodec_dec_pdata *dec_pdata = ctx->dev-
> >vdec_pdata;
> > +const struct mtk_video_fmt *fmt;
> > +int i = 0, ret = 0;
> > +
> > +hdr_ctrl = ctrl;
> > +if (!hdr_ctrl || !hdr_ctrl->p_cur.p)
> > +return -EINVAL;
> 
> There is a null check for hdr_ctrl->p_cur.p ...
> 
Should be hdr_ctrl->p_new.p.
> > +
> > +switch (hdr_ctrl->id) {
> > +case V4L2_CID_STATELESS_H264_SPS:
> > +h264 = (struct v4l2_ctrl_h264_sps *)hdr_ctrl->p_new.p;
> 
> But you are using hdr_ctrl->p_new.p. I don't know if the checks are
> absolutly
> required, I'll have a look.
> 
> > +if (h264->bit_depth_chroma_minus8 == 2 && h264-
> >bit_depth_luma_minus8 == 2) {
> 
> In the conformance streams, there is a 9bit luma file, which on all
> decoders
> I've tried decodes fine inside a 10bit image.
> 
Confirmed with our designer, the hardware not support 9bit bitstream. 
M
aybe test pass randomly.

> > +ctx->is_10bit_bitstream = true;
> > +} else if (h264->bit_depth_chroma_minus8 != 0 &&
> > +   h264->bit_depth_luma_minus8 != 0) {
> > +mtk_v4l2_vdec_err(ctx, "H264: chroma_minus8:%d, luma_minus8:%d",
> > +  h264->bit_depth_chroma_minus8,
> > +  h264->bit_depth_luma_minus8);
> > +return -EINVAL;
> > +}
> > +break;
> > +case V4L2_CID_STATELESS_HEVC_SPS:
> > +h265 = (struct v4l2_ctrl_hevc_sps *)hdr_ctrl->p_new.p;
> > +if (h265->bit_depth_chroma_minus8 == 2 && h265-
> >bit_depth_luma_minus8 == 2) {
> > +ctx->is_10bit_bitstream = true;
> > +} else if (h265->bit_depth_chroma_minus8 != 0 &&
> > +   h265->bit_depth_luma_minus8 != 0) {
> > +mtk_v4l2_vdec_err(ctx, "HEVC: chroma_minus8:%d, luma_minus8:%d",
> > +  h265->bit_depth_chroma_minus8,
> > +  h265->bit_depth_luma_minus8);
> > +return -EINVAL;
> > +}
> > +break;
> > +case V4L2_CID_STATELESS_VP9_FRAME:
> > +frame = (struct v4l2_ctrl_vp9_frame *)hdr_ctrl->p_new.p;
> > +if (frame->bit_depth == 10) {
> > +ctx->is_10bit_bitstream = true;
> > +} else if (frame->bit_depth != 8) {
> > +mtk_v4l2_vdec_err(ctx, "VP9: bit_depth:%d", frame->bit_depth);
> > +return -EINVAL;
> > +}
> > +break;
> > +case V4L2_CID_STATELESS_AV1_SEQUENCE:
> > +seq = (struct v4l2_ctrl_av1_sequence *)hdr_ctrl->p_new.p;
> > +if (seq->bit_depth == 10) {
> > +ctx->is_10bit_bitstream = true;
> > +} else if (seq->bit_depth != 8) {
> > +mtk_v4l2_vdec_err(ctx, "AV1: bit_depth:%d", seq->bit_depth);
> > +return -EINVAL;
> > +}
> > +break;
> > +default:
> > +mtk_v4l2_vdec_err(ctx, "Not supported ctrl id: 0x%x\n", hdr_ctrl-
> >id);
> > +return -EINVAL;
> 
> Haven't tested, but it feels like this will prevent setting the PPS
> among many
> other controls. This should likely not be an error, and should return
> 0.
> 
You are right, need to remove 'return -EINVAL' with 'return ret';

Best Regards,
Yunfei Dong
> > +}
> > +
> > +if (!ctx->is_10bit_bitstream)
> > +return ret;
> > +
> > +for (i = 0; i < *dec_pdata->num_formats; i++) {
> > +fmt = &dec_pdata->vdec_formats[i];
> > +if (fmt->fourcc == V4L2_PIX_FMT_MT2110R &&
> > +    hdr_ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> > +ctx->q_data[MTK_Q_DATA_DST].fmt = fmt;
> > +break;
> > +}
> > +
> > +if (fmt->fourcc == V4L2_PIX_FMT_MT2110T &&
> > +    (hdr_ctrl->id == V4L2_CID_STATELESS_HEVC_SPS ||
> > +    hdr_ctrl->id == V4L2_CID_STATELESS_VP9_FRAME ||
> > +    hdr_ctrl->id == V4L2_CID_STATELESS_AV1_SEQUENCE)) {
> > +ctx->q_data[MTK_Q_DATA_DST].fmt = fmt;
> > +break;
> > +}
> > +}
> > +ret = mtk_vcodec_get_pic_info(ctx);
> > +
> > +return ret;
> > +}
> > +
> > +static const struct v4l2_ctrl_ops mtk_vcodec_dec_ctrl_ops = {
> > +.s_ctrl = mtk_vdec_s_ctrl,
> > +};
> > +
> >  static int mtk_vcodec_dec_ctrls_setup(struct mtk_vcodec_dec_ctx
> *ctx)
> >  {
> >  unsigned int i;
> > @@ -399,7 +527,7 @@ static int mtk_vcodec_dec_ctrls_setup(struct
> mtk_vcodec_dec_ctx *ctx)
> >  
> >  for (i = 0; i < NUM_CTRLS; i++) {
> >  struct v4l2_ctrl_config cfg = mtk_stateless_controls[i].cfg;
> > -
> > +cfg.ops = &mtk_vcodec_dec_ctrl_ops;
> >  v4l2_ctrl_new_custom(&ctx->ctrl_hdl, &cfg, NULL);
> >  if (ctx->ctrl_hdl.error) {
> >  mtk_v4l2_vdec_err(ctx, "Adding control %d failed %d", i,
> > @@ -466,6 +594,8 @@ static void mtk_vcodec_add_formats(unsigned int
> fourcc,
> >  break;
> >  case V4L2_PIX_FMT_MM21:
> >  case V4L2_PIX_FMT_MT21C:
> > +case V4L2_PIX_FMT_MT2110T:
> > +case V4L2_PIX_FMT_MT2110R:
> >  mtk_video_formats[count_formats].fourcc = fourcc;
> >  mtk_video_formats[count_formats].type = MTK_FMT_FRAME;
> >  mtk_video_formats[count_formats].num_planes = 2;
> > @@ -491,6 +621,12 @@ static void
> mtk_vcodec_get_supported_formats(struct mtk_vcodec_dec_ctx *ctx)
> >  mtk_vcodec_add_formats(V4L2_PIX_FMT_MT21C, ctx);
> >  cap_format_count++;
> >  }
> > +if (ctx->dev->dec_capability & MTK_VDEC_IS_SUPPORT_10BIT) {
> > +mtk_vcodec_add_formats(V4L2_PIX_FMT_MT2110T, ctx);
> > +cap_format_count++;
> > +mtk_vcodec_add_formats(V4L2_PIX_FMT_MT2110R, ctx);
> > +cap_format_count++;
> > +}
> >  if (ctx->dev->dec_capability & MTK_VDEC_FORMAT_MM21) {
> >  mtk_vcodec_add_formats(V4L2_PIX_FMT_MM21, ctx);
> >  cap_format_count++;
> 

Re: [PATCH 0/3] media: mediatek: vcodec: Add driver to support 10bit

[Nicolas Dufresne]

Le mercredi 12 juillet 2023 à 03:11 +0000, Yunfei Dong (董云飞) a écrit :
> > This cover letter is missing v4l2-compliance report. This is needed
> > whenever
> > format mechanism is modified in a driver (not just for new drivers).
> > Please add
> > and make sure there is not regression too.
> > 
> Forgot to test v4l2-compliance, will add in next patch.
> 
> The flush test not include H264 for test fail, according to mingjia's
> information, maybe gstreamer set wrong format. H264 fluster can test
> pass when kernel driver force h264 in raster mode.

For everyone interested, I've rebased MT2110R and MT2110T support MR in
GStreamer. With the offending "return -EINVAL" removed, the test passes for me.
Consider updating your GStreamer build, let me know if you see further issue
with GStreamer in this regard. I'm only missing a firmware with HEVC and 10bit
combined to confirm HEVC works too.

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3444

cheer,
Nicolas