[PATCH v1, 00/12] media: mtk-vcodec: support for MT8192 h264 decoder

[PATCH v1, 00/12] media: mtk-vcodec: support for MT8192 h264 decoder

[Yunfei Dong]

This series adds support for mt8192 h264 decoder. Firstly, need to refactor
power/clk/irq interface for mt8192 is lat and core architecture. Secondly,
add new functions to get frame buffer size and resolution according to decoder
capability from scp side. Then add callback function to get/put capture buffer
in order to enable lat and core decoder in parallel. Then add to support MT21C
compressed mode and fix v4l2-compliance fail. Next, extract H264 request
api driver to let mt8183 and mt8192 use the common code, at last, add mt8192 h264
slice api driver.

Patches 1 refactor power/clk/irq interface.
Patches 2~4 get frame buffer size and resolution according to decoder capability.
Patches 5~6 enable lat and core decode in parallel.
Patch 7~10 Add to support MT21C compressed mode and fix v4l2-compliance fail.
Patch 11~12 extract h264 driver and add mt8192 h264 slice api driver
----
Dependents on "Support multi hardware decode using of_platform_populate"[1].

This patches are the second part used to add mt8192 h264 decoder. And the base part is [1].

[1]https://patchwork.linuxtv.org/project/linux-media/cover/20211215061552.8523-1-yunfei.dong@mediatek.com/
----
Yunfei Dong (12):
  media: mtk-vcodec: Add vdec enable/disable hardware helpers
  media: mtk-vcodec: Using firmware type to separate different firmware
    architecture
  media: mtk-vcodec: get frame buffer size from scp
  media: mtk-vcodec: Read max resolution from dec_capability
  media: mtk-vcodec: Call v4l2_m2m_set_dst_buffered() set capture buffer
    buffered
  media: mtk-vcodec: Refactor get and put capture buffer flow
  media: mtk-vcodec: Refactor supported vdec formats and framesizes
  media: mtk-vcodec: Add format to support MT21C
  media: mtk-vcodec: disable vp8 4K capability
  media: mtk-vcodec: Fix v4l2-compliance fail
  media: mtk-vcodec: Extract H264 common code
  media: mtk-vcodec: Add h264 slice api driver for mt8192

 drivers/media/platform/mtk-vcodec/Makefile    |   2 +
 .../platform/mtk-vcodec/mtk_vcodec_dec.c      |  49 +-
 .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |   5 -
 .../platform/mtk-vcodec/mtk_vcodec_dec_pm.c   | 162 +++--
 .../platform/mtk-vcodec/mtk_vcodec_dec_pm.h   |   6 +-
 .../mtk-vcodec/mtk_vcodec_dec_stateful.c      |  14 +-
 .../mtk-vcodec/mtk_vcodec_dec_stateless.c     | 245 +++++--
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  27 +-
 .../media/platform/mtk-vcodec/mtk_vcodec_fw.c |   6 +
 .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |   1 +
 .../mtk-vcodec/vdec/vdec_h264_req_common.c    | 303 +++++++++
 .../mtk-vcodec/vdec/vdec_h264_req_common.h    | 247 +++++++
 .../mtk-vcodec/vdec/vdec_h264_req_if.c        | 402 +-----------
 .../mtk-vcodec/vdec/vdec_h264_req_lat_if.c    | 620 ++++++++++++++++++
 .../media/platform/mtk-vcodec/vdec_drv_if.c   |  28 +-
 .../media/platform/mtk-vcodec/vdec_drv_if.h   |   1 +
 .../media/platform/mtk-vcodec/vdec_ipi_msg.h  |  36 +
 .../platform/mtk-vcodec/vdec_msg_queue.c      |   2 +
 .../media/platform/mtk-vcodec/vdec_vpu_if.c   |  55 +-
 .../media/platform/mtk-vcodec/vdec_vpu_if.h   |  15 +
 include/linux/remoteproc/mtk_scp.h            |   2 +
 21 files changed, 1672 insertions(+), 556 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_common.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_common.h
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_lat_if.c

-- 
2.25.1

[PATCH v1, 01/12] media: mtk-vcodec: Add vdec enable/disable hardware helpers

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

Lock, power and clock are highly coupled operations. Adds vdec
enable/disable hardware helpers and uses them.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |   5 -
 .../platform/mtk-vcodec/mtk_vcodec_dec_pm.c   | 162 +++++++++++-------
 .../platform/mtk-vcodec/mtk_vcodec_dec_pm.h   |   6 +-
 .../media/platform/mtk-vcodec/vdec_drv_if.c   |  20 +--
 .../platform/mtk-vcodec/vdec_msg_queue.c      |   2 +
 5 files changed, 110 insertions(+), 85 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index 3f8f3309cf9d..3cbdbf1ad3e5 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -223,9 +223,6 @@ static int fops_vcodec_open(struct file *file)
 	mtk_vcodec_dec_set_default_params(ctx);
 
 	if (v4l2_fh_is_singular(&ctx->fh)) {
-		ret = mtk_vcodec_dec_pw_on(dev, MTK_VDEC_LAT0);
-		if (ret < 0)
-			goto err_load_fw;
 		/*
 		 * Does nothing if firmware was already loaded.
 		 */
@@ -282,8 +279,6 @@ static int fops_vcodec_release(struct file *file)
 	v4l2_m2m_ctx_release(ctx->m2m_ctx);
 	mtk_vcodec_dec_release(ctx);
 
-	if (v4l2_fh_is_singular(&ctx->fh))
-		mtk_vcodec_dec_pw_off(dev, MTK_VDEC_LAT0);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
index b9f5ef979c69..aa24e073e3b2 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
@@ -80,74 +80,31 @@ int mtk_vcodec_init_dec_clk(struct platform_device *pdev,
 }
 EXPORT_SYMBOL_GPL(mtk_vcodec_init_dec_clk);
 
-int mtk_vcodec_dec_pw_on(struct mtk_vcodec_dev *vdec_dev, int hw_idx)
+static int mtk_vcodec_dec_pw_on(struct mtk_vcodec_pm *pm)
 {
-	struct mtk_vdec_hw_dev *subdev_dev;
-	struct mtk_vcodec_pm *pm;
 	int ret;
 
-	if (vdec_dev->vdec_pdata->is_subdev_supported) {
-		subdev_dev = mtk_vcodec_get_hw_dev(vdec_dev, hw_idx);
-		if (!subdev_dev) {
-			mtk_v4l2_err("Failed to get hw dev\n");
-			return -EINVAL;
-		}
-		pm = &subdev_dev->pm;
-	} else {
-		pm = &vdec_dev->pm;
-	}
-
 	ret = pm_runtime_resume_and_get(pm->dev);
 	if (ret)
 		mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
 
 	return ret;
 }
-EXPORT_SYMBOL_GPL(mtk_vcodec_dec_pw_on);
 
-void mtk_vcodec_dec_pw_off(struct mtk_vcodec_dev *vdec_dev, int hw_idx)
+static void mtk_vcodec_dec_pw_off(struct mtk_vcodec_pm *pm)
 {
-	struct mtk_vdec_hw_dev *subdev_dev;
-	struct mtk_vcodec_pm *pm;
 	int ret;
 
-	if (vdec_dev->vdec_pdata->is_subdev_supported) {
-		subdev_dev = mtk_vcodec_get_hw_dev(vdec_dev, hw_idx);
-		if (!subdev_dev) {
-			mtk_v4l2_err("Failed to get hw dev\n");
-			return;
-		}
-		pm = &subdev_dev->pm;
-	} else {
-		pm = &vdec_dev->pm;
-	}
-
 	ret = pm_runtime_put_sync(pm->dev);
 	if (ret)
 		mtk_v4l2_err("pm_runtime_put_sync fail %d", ret);
 }
-EXPORT_SYMBOL_GPL(mtk_vcodec_dec_pw_off);
 
-void mtk_vcodec_dec_clock_on(struct mtk_vcodec_dev *vdec_dev, int hw_idx)
+static void mtk_vcodec_dec_clock_on(struct mtk_vcodec_pm *pm)
 {
-	struct mtk_vdec_hw_dev *subdev_dev;
-	struct mtk_vcodec_pm *pm;
 	struct mtk_vcodec_clk *dec_clk;
 	int ret, i;
 
-	if (vdec_dev->vdec_pdata->is_subdev_supported) {
-		subdev_dev = mtk_vcodec_get_hw_dev(vdec_dev, hw_idx);
-		if (!subdev_dev) {
-			mtk_v4l2_err("Failed to get hw dev\n");
-			return;
-		}
-		pm = &subdev_dev->pm;
-		enable_irq(subdev_dev->dec_irq);
-	} else {
-		pm = &vdec_dev->pm;
-		enable_irq(vdec_dev->dec_irq);
-	}
-
 	dec_clk = &pm->vdec_clk;
 	for (i = 0; i < dec_clk->clk_num; i++) {
 		ret = clk_prepare_enable(dec_clk->clk_info[i].vcodec_clk);
@@ -169,31 +126,112 @@ void mtk_vcodec_dec_clock_on(struct mtk_vcodec_dev *vdec_dev, int hw_idx)
 	for (i -= 1; i >= 0; i--)
 		clk_disable_unprepare(dec_clk->clk_info[i].vcodec_clk);
 }
-EXPORT_SYMBOL_GPL(mtk_vcodec_dec_clock_on);
 
-void mtk_vcodec_dec_clock_off(struct mtk_vcodec_dev *vdec_dev, int hw_idx)
+static void mtk_vcodec_dec_clock_off(struct mtk_vcodec_pm *pm)
 {
-	struct mtk_vdec_hw_dev *subdev_dev;
-	struct mtk_vcodec_pm *pm;
 	struct mtk_vcodec_clk *dec_clk;
 	int i;
 
+	dec_clk = &pm->vdec_clk;
+	mtk_smi_larb_put(pm->larbvdec);
+	for (i = dec_clk->clk_num - 1; i >= 0; i--)
+		clk_disable_unprepare(dec_clk->clk_info[i].vcodec_clk);
+}
+
+static void mtk_vcodec_dec_enable_irq(struct mtk_vcodec_dev *vdec_dev, int comp_idx)
+ {
+	struct mtk_vdec_hw_dev *subdev_dev;
+
+	if (!test_bit(comp_idx, vdec_dev->subdev_bitmap))
+		return;
+
 	if (vdec_dev->vdec_pdata->is_subdev_supported) {
-		subdev_dev = mtk_vcodec_get_hw_dev(vdec_dev, hw_idx);
-		if (!subdev_dev) {
+		subdev_dev = mtk_vcodec_get_hw_dev(vdec_dev, comp_idx);
+		if (subdev_dev)
+			enable_irq(subdev_dev->dec_irq);
+		else
+			mtk_v4l2_err("Failed to get hw dev\n");
+	} else {
+		enable_irq(vdec_dev->dec_irq);
+	}
+}
+
+static void mtk_vcodec_dec_disable_irq(struct mtk_vcodec_dev *vdec_dev, int comp_idx)
+{
+	struct mtk_vdec_hw_dev *subdev_dev;
+
+	if (!test_bit(comp_idx, vdec_dev->subdev_bitmap))
+		return;
+
+	if (vdec_dev->vdec_pdata->is_subdev_supported) {
+		subdev_dev = mtk_vcodec_get_hw_dev(vdec_dev, comp_idx);
+		if (subdev_dev)
+			disable_irq(subdev_dev->dec_irq);
+		else
 			mtk_v4l2_err("Failed to get hw dev\n");
-			return;
-		}
-		pm = &subdev_dev->pm;
-		disable_irq(subdev_dev->dec_irq);
 	} else {
-		pm = &vdec_dev->pm;
 		disable_irq(vdec_dev->dec_irq);
 	}
+}
 
-	dec_clk = &pm->vdec_clk;
-	mtk_smi_larb_put(pm->larbvdec);
-	for (i = dec_clk->clk_num - 1; i >= 0; i--)
-		clk_disable_unprepare(dec_clk->clk_info[i].vcodec_clk);
+static struct mtk_vcodec_pm *mtk_vcodec_dec_get_pm(struct mtk_vcodec_dev *vdec_dev,
+	int comp_idx)
+{
+	struct mtk_vdec_hw_dev *subdev_dev;
+
+	if (!test_bit(comp_idx, vdec_dev->subdev_bitmap))
+		return NULL;
+
+	if (vdec_dev->vdec_pdata->is_subdev_supported) {
+		subdev_dev = mtk_vcodec_get_hw_dev(vdec_dev, comp_idx);
+		if (subdev_dev)
+			return &subdev_dev->pm;
+
+		mtk_v4l2_err("Failed to get hw dev\n");
+		return NULL;
+	}
+
+	return &vdec_dev->pm;
+}
+
+void mtk_vcodec_dec_enable_hardware(struct mtk_vcodec_ctx *ctx, int comp_idx)
+{
+	struct mtk_vcodec_pm *pm;
+
+	mutex_lock(&ctx->dev->dec_mutex[comp_idx]);
+	if (IS_VDEC_LAT_ARCH(ctx->dev->vdec_pdata->hw_arch) &&
+		comp_idx == MTK_VDEC_CORE) {
+		pm = mtk_vcodec_dec_get_pm(ctx->dev, MTK_VDEC_LAT0);
+		mtk_vcodec_dec_pw_on(pm);
+		mtk_vcodec_dec_clock_on(pm);
+	}
+
+	pm = mtk_vcodec_dec_get_pm(ctx->dev, comp_idx);
+	if (pm) {
+		mtk_vcodec_dec_pw_on(pm);
+		mtk_vcodec_dec_clock_on(pm);
+	}
+
+	mtk_vcodec_dec_enable_irq(ctx->dev, comp_idx);
+}
+EXPORT_SYMBOL_GPL(mtk_vcodec_dec_enable_hardware);
+
+void mtk_vcodec_dec_disable_hardware(struct mtk_vcodec_ctx *ctx, int comp_idx)
+{
+	struct mtk_vcodec_pm *pm;
+
+	mtk_vcodec_dec_disable_irq(ctx->dev, comp_idx);
+
+	pm = mtk_vcodec_dec_get_pm(ctx->dev, comp_idx);
+	mtk_vcodec_dec_clock_off(pm);
+	mtk_vcodec_dec_pw_off(pm);
+
+	if (IS_VDEC_LAT_ARCH(ctx->dev->vdec_pdata->hw_arch) &&
+		comp_idx == MTK_VDEC_CORE) {
+		pm = mtk_vcodec_dec_get_pm(ctx->dev, MTK_VDEC_LAT0);
+		mtk_vcodec_dec_clock_off(pm);
+		mtk_vcodec_dec_pw_off(pm);
+	}
+	mutex_unlock(&ctx->dev->dec_mutex[comp_idx]);
 }
-EXPORT_SYMBOL_GPL(mtk_vcodec_dec_clock_off);
+EXPORT_SYMBOL_GPL(mtk_vcodec_dec_disable_hardware);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
index c4121df9764f..f68ae3bd43a1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.h
@@ -12,9 +12,7 @@
 int mtk_vcodec_init_dec_clk(struct platform_device *pdev,
 	struct mtk_vcodec_pm *pm);
 
-int mtk_vcodec_dec_pw_on(struct mtk_vcodec_dev *vdec_dev, int hw_idx);
-void mtk_vcodec_dec_pw_off(struct mtk_vcodec_dev *vdec_dev, int hw_idx);
-void mtk_vcodec_dec_clock_on(struct mtk_vcodec_dev *vdec_dev, int hw_idx);
-void mtk_vcodec_dec_clock_off(struct mtk_vcodec_dev *vdec_dev, int hw_idx);
+void mtk_vcodec_dec_enable_hardware(struct mtk_vcodec_ctx *ctx, int comp_idx);
+void mtk_vcodec_dec_disable_hardware(struct mtk_vcodec_ctx *ctx, int comp_idx);
 
 #endif /* _MTK_VCODEC_DEC_PM_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
index 05a5b240e906..c93dd0ea3537 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
@@ -38,11 +38,9 @@ int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 		return -EINVAL;
 	}
 
-	mtk_vdec_lock(ctx);
-	mtk_vcodec_dec_clock_on(ctx->dev, ctx->hw_id);
+	mtk_vcodec_dec_enable_hardware(ctx, ctx->hw_id);
 	ret = ctx->dec_if->init(ctx);
-	mtk_vcodec_dec_clock_off(ctx->dev, ctx->hw_id);
-	mtk_vdec_unlock(ctx);
+	mtk_vcodec_dec_disable_hardware(ctx, ctx->hw_id);
 
 	return ret;
 }
@@ -70,15 +68,11 @@ int vdec_if_decode(struct mtk_vcodec_ctx *ctx, struct mtk_vcodec_mem *bs,
 	if (!ctx->drv_handle)
 		return -EIO;
 
-	mtk_vdec_lock(ctx);
-
+	mtk_vcodec_dec_enable_hardware(ctx, ctx->hw_id);
 	mtk_vcodec_set_curr_ctx(ctx->dev, ctx, ctx->hw_id);
-	mtk_vcodec_dec_clock_on(ctx->dev, ctx->hw_id);
 	ret = ctx->dec_if->decode(ctx->drv_handle, bs, fb, res_chg);
-	mtk_vcodec_dec_clock_off(ctx->dev, ctx->hw_id);
 	mtk_vcodec_set_curr_ctx(ctx->dev, NULL, ctx->hw_id);
-
-	mtk_vdec_unlock(ctx);
+	mtk_vcodec_dec_disable_hardware(ctx, ctx->hw_id);
 
 	return ret;
 }
@@ -103,11 +97,9 @@ void vdec_if_deinit(struct mtk_vcodec_ctx *ctx)
 	if (!ctx->drv_handle)
 		return;
 
-	mtk_vdec_lock(ctx);
-	mtk_vcodec_dec_clock_on(ctx->dev, ctx->hw_id);
+	mtk_vcodec_dec_enable_hardware(ctx, ctx->hw_id);
 	ctx->dec_if->deinit(ctx->drv_handle);
-	mtk_vcodec_dec_clock_off(ctx->dev, ctx->hw_id);
-	mtk_vdec_unlock(ctx);
+	mtk_vcodec_dec_disable_hardware(ctx, ctx->hw_id);
 
 	ctx->drv_handle = NULL;
 }
diff --git a/drivers/media/platform/mtk-vcodec/vdec_msg_queue.c b/drivers/media/platform/mtk-vcodec/vdec_msg_queue.c
index 24f1d03df9f1..2f54d04b8bdf 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_msg_queue.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_msg_queue.c
@@ -212,11 +212,13 @@ static void vdec_msg_queue_core_work(struct work_struct *work)
 		return;
 
 	ctx = lat_buf->ctx;
+	mtk_vcodec_dec_enable_hardware(ctx, MTK_VDEC_CORE);
 	mtk_vcodec_set_curr_ctx(dev, ctx, MTK_VDEC_CORE);
 
 	lat_buf->core_decode(lat_buf);
 
 	mtk_vcodec_set_curr_ctx(dev, NULL, MTK_VDEC_CORE);
+	mtk_vcodec_dec_disable_hardware(ctx, MTK_VDEC_CORE);
 	vdec_msg_queue_qbuf(&ctx->msg_queue.lat_ctx, lat_buf);
 
 	if (!list_empty(&ctx->msg_queue.lat_ctx.ready_queue)) {
-- 
2.25.1

[PATCH v1, 02/12] media: mtk-vcodec: Using firmware type to separate different firmware architecture

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

MT8173 platform use vpu firmware, mt8183/mt8192 will use scp
firmware instead, using chip name is not reasonable to separate
different firmware architecture. Using firmware type is much better.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c | 1 -
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c    | 2 --
 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h          | 2 --
 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c           | 6 ++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h           | 1 +
 drivers/media/platform/mtk-vcodec/vdec_vpu_if.c             | 4 ++--
 6 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
index 04ca43c77e5f..7966c132be8f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
@@ -613,7 +613,6 @@ static struct vb2_ops mtk_vdec_frame_vb2_ops = {
 };
 
 const struct mtk_vcodec_dec_pdata mtk_vdec_8173_pdata = {
-	.chip = MTK_MT8173,
 	.init_vdec_params = mtk_init_vdec_params,
 	.ctrls_setup = mtk_vcodec_dec_ctrls_setup,
 	.vdec_vb2_ops = &mtk_vdec_frame_vb2_ops,
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
index 6c67b7f65f9c..1a4013f4fb83 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
@@ -344,7 +344,6 @@ static struct vb2_ops mtk_vdec_request_vb2_ops = {
 };
 
 const struct mtk_vcodec_dec_pdata mtk_vdec_8183_pdata = {
-	.chip = MTK_MT8183,
 	.init_vdec_params = mtk_init_vdec_params,
 	.ctrls_setup = mtk_vcodec_dec_ctrls_setup,
 	.vdec_vb2_ops = &mtk_vdec_request_vb2_ops,
@@ -363,7 +362,6 @@ const struct mtk_vcodec_dec_pdata mtk_vdec_8183_pdata = {
 
 /* This platform data is used for one lat and one core architecture. */
 const struct mtk_vcodec_dec_pdata mtk_lat_sig_core_pdata = {
-	.chip = MTK_MT8192,
 	.init_vdec_params = mtk_init_vdec_params,
 	.ctrls_setup = mtk_vcodec_dec_ctrls_setup,
 	.vdec_vb2_ops = &mtk_vdec_request_vb2_ops,
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index a558cc16026d..e9a31306b956 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -367,7 +367,6 @@ enum mtk_vdec_hw_arch {
  * @vdec_framesizes: supported video decoder frame sizes
  * @num_framesizes: count of video decoder frame sizes
  *
- * @chip: chip this decoder is compatible with
  * @hw_arch: hardware arch is used to separate pure_sin_core and lat_sin_core
  *
  * @is_subdev_supported: whether support parent-node architecture(subdev)
@@ -390,7 +389,6 @@ struct mtk_vcodec_dec_pdata {
 	const struct mtk_codec_framesizes *vdec_framesizes;
 	const int num_framesizes;
 
-	enum mtk_chip chip;
 	enum mtk_vdec_hw_arch hw_arch;
 
 	bool is_subdev_supported;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
index 94b39ae5c2e1..556e54aadac9 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
@@ -65,3 +65,9 @@ int mtk_vcodec_fw_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
 	return fw->ops->ipi_send(fw, id, buf, len, wait);
 }
 EXPORT_SYMBOL_GPL(mtk_vcodec_fw_ipi_send);
+
+int mtk_vcodec_fw_get_type(struct mtk_vcodec_fw *fw)
+{
+	return fw->type;
+}
+EXPORT_SYMBOL_GPL(mtk_vcodec_fw_get_type);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
index 539bb626772c..acd355961e3a 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
@@ -39,5 +39,6 @@ int mtk_vcodec_fw_ipi_register(struct mtk_vcodec_fw *fw, int id,
 			       const char *name, void *priv);
 int mtk_vcodec_fw_ipi_send(struct mtk_vcodec_fw *fw, int id,
 			   void *buf, unsigned int len, unsigned int wait);
+int mtk_vcodec_fw_get_type(struct mtk_vcodec_fw *fw);
 
 #endif /* _MTK_VCODEC_FW_H_ */
diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
index c84fac52fe26..21f6d9c5a371 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
@@ -33,8 +33,8 @@ static void handle_init_ack_msg(const struct vdec_vpu_ipi_init_ack *msg)
 	 */
 	vpu->inst_id = 0xdeadbeef;
 
-	/* Firmware version field does not exist on MT8173. */
-	if (vpu->ctx->dev->vdec_pdata->chip == MTK_MT8173)
+	/* VPU firmware does not contain a version field. */
+	if (mtk_vcodec_fw_get_type(vpu->ctx->dev->fw_handler) == VPU)
 		return;
 
 	/* Check firmware version. */
-- 
2.25.1

[PATCH v1, 03/12] media: mtk-vcodec: get frame buffer size from scp

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

Different capture buffer format has different buffer size, need to get
real buffer size according to buffer type from scp.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_dec.c      |  2 +
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  2 +
 .../media/platform/mtk-vcodec/vdec_ipi_msg.h  | 36 +++++++++++++
 .../media/platform/mtk-vcodec/vdec_vpu_if.c   | 51 +++++++++++++++++++
 .../media/platform/mtk-vcodec/vdec_vpu_if.h   | 15 ++++++
 5 files changed, 106 insertions(+)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 130ecef2e766..87891ebd7246 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -466,6 +466,8 @@ static int vidioc_vdec_s_fmt(struct file *file, void *priv,
 			}
 			ctx->state = MTK_STATE_INIT;
 		}
+	} else {
+		ctx->capture_fourcc = fmt->fourcc;
 	}
 
 	/*
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index e9a31306b956..fc2c23deb825 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -277,6 +277,7 @@ struct vdec_pic_info {
  *		     to be used with encoder and stateful decoder.
  * @is_flushing: set to true if flushing is in progress.
  * @current_codec: current set input codec, in V4L2 pixel format
+ * @capture_fourcc: capture queue type, in V4L2 pixel format
  *
  * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
  * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
@@ -322,6 +323,7 @@ struct mtk_vcodec_ctx {
 	bool is_flushing;
 
 	u32 current_codec;
+	u32 capture_fourcc;
 
 	enum v4l2_colorspace colorspace;
 	enum v4l2_ycbcr_encoding ycbcr_enc;
diff --git a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
index 5daca8d52ebb..d00e555cf27a 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
+++ b/drivers/media/platform/mtk-vcodec/vdec_ipi_msg.h
@@ -20,6 +20,7 @@ enum vdec_ipi_msgid {
 	AP_IPIMSG_DEC_RESET = 0xA004,
 	AP_IPIMSG_DEC_CORE = 0xA005,
 	AP_IPIMSG_DEC_CORE_END = 0xA006,
+	AP_IPIMSG_DEC_GET_PARAM = 0xA007,
 
 	VPU_IPIMSG_DEC_INIT_ACK = 0xB000,
 	VPU_IPIMSG_DEC_START_ACK = 0xB001,
@@ -28,6 +29,7 @@ enum vdec_ipi_msgid {
 	VPU_IPIMSG_DEC_RESET_ACK = 0xB004,
 	VPU_IPIMSG_DEC_CORE_ACK = 0xB005,
 	VPU_IPIMSG_DEC_CORE_END_ACK = 0xB006,
+	VPU_IPIMSG_DEC_GET_PARAM_ACK = 0xB007,
 };
 
 /**
@@ -114,4 +116,38 @@ struct vdec_vpu_ipi_init_ack {
 	uint32_t inst_id;
 };
 
+/**
+ * struct vdec_ap_ipi_get_param - for AP_IPIMSG_SET_PARAM
+ * @msg_id	: AP_IPIMSG_DEC_START
+ * @inst_id     : instance ID. Used if the ABI version >= 2.
+ * @data	: picture information
+ * @param_type	: get param type
+ * @codec_type	: Codec fourcc
+ */
+struct vdec_ap_ipi_get_param {
+	uint32_t msg_id;
+	uint32_t inst_id;
+	uint32_t data[4];
+	uint32_t param_type;
+	uint32_t codec_type;
+};
+
+
+/**
+ * struct vdec_vpu_ipi_init_ack - for VPU_IPIMSG_DEC_INIT_ACK
+ * @msg_id	: VPU_IPIMSG_DEC_INIT_ACK
+ * @status	: VPU exeuction result
+ * @ap_inst_addr	: AP vcodec_vpu_inst instance address
+ * @data     : picture information from SCP.
+ * @param_type	: get param type
+ */
+struct vdec_vpu_ipi_get_param_ack {
+	uint32_t msg_id;
+	int32_t status;
+	uint64_t ap_inst_addr;
+	uint32_t data[4];
+	uint32_t param_type;
+	uint32_t reserved;
+};
+
 #endif
diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
index 21f6d9c5a371..6f9bcc2b0bb9 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
@@ -54,6 +54,27 @@ static void handle_init_ack_msg(const struct vdec_vpu_ipi_init_ack *msg)
 	}
 }
 
+static void handle_get_param_msg_ack(
+	const struct vdec_vpu_ipi_get_param_ack *msg)
+{
+	struct vdec_vpu_inst *vpu = (struct vdec_vpu_inst *)
+					(unsigned long)msg->ap_inst_addr;
+
+	mtk_vcodec_debug(vpu, "+ ap_inst_addr = 0x%llx", msg->ap_inst_addr);
+
+	/* param_type is enum vdec_get_param_type */
+	switch(msg->param_type) {
+	case 2:
+		vpu->fb_sz[0] = msg->data[0];
+		vpu->fb_sz[1] = msg->data[1];
+		break;
+	default:
+		mtk_vcodec_err(vpu, "invalid get param type=%d", msg->param_type);
+		vpu->failure = 1;
+		break;
+	}
+}
+
 /*
  * vpu_dec_ipi_handler - Handler for VPU ipi message.
  *
@@ -89,6 +110,9 @@ static void vpu_dec_ipi_handler(void *data, unsigned int len, void *priv)
 		case VPU_IPIMSG_DEC_CORE_END_ACK:
 			break;
 
+		case VPU_IPIMSG_DEC_GET_PARAM_ACK:
+			handle_get_param_msg_ack(data);
+			break;
 		default:
 			mtk_vcodec_err(vpu, "invalid msg=%X", msg->msg_id);
 			break;
@@ -217,6 +241,33 @@ int vpu_dec_start(struct vdec_vpu_inst *vpu, uint32_t *data, unsigned int len)
 	return err;
 }
 
+int vpu_dec_get_param(struct vdec_vpu_inst *vpu, uint32_t *data,
+	unsigned int len, unsigned int param_type)
+{
+	struct vdec_ap_ipi_get_param msg;
+	int i;
+	int err;
+
+	mtk_vcodec_debug_enter(vpu);
+
+	if (len > ARRAY_SIZE(msg.data)) {
+		mtk_vcodec_err(vpu, "invalid len = %d\n", len);
+		return -EINVAL;
+	}
+
+	memset(&msg, 0, sizeof(msg));
+	msg.msg_id = AP_IPIMSG_DEC_GET_PARAM;
+	msg.inst_id = vpu->inst_id;
+	for (i = 0; i < len; i++)
+		msg.data[i] = data[i];
+	msg.param_type = param_type;
+	msg.codec_type = vpu->codec_type;
+
+	err = vcodec_vpu_send_msg(vpu, (void *)&msg, sizeof(msg));
+	mtk_vcodec_debug(vpu, "- ret=%d", err);
+	return err;
+}
+
 int vpu_dec_core(struct vdec_vpu_inst *vpu)
 {
 	return vcodec_send_ap_ipi(vpu, AP_IPIMSG_DEC_CORE);
diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
index 4cb3c7f5a3ad..963f8d4877b7 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
+++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
@@ -28,6 +28,8 @@ struct mtk_vcodec_ctx;
  * @wq          : wait queue to wait VPU message ack
  * @handler     : ipi handler for each decoder
  * @codec_type     : use codec type to separate different codecs
+ * @capture_type    : used capture type to separate different capture format
+ * @fb_sz  : frame buffer size of each plane
  */
 struct vdec_vpu_inst {
 	int id;
@@ -42,6 +44,8 @@ struct vdec_vpu_inst {
 	wait_queue_head_t wq;
 	mtk_vcodec_ipi_handler handler;
 	unsigned int codec_type;
+	unsigned int capture_type;
+	unsigned int fb_sz[2];
 };
 
 /**
@@ -104,4 +108,15 @@ int vpu_dec_core(struct vdec_vpu_inst *vpu);
  */
 int vpu_dec_core_end(struct vdec_vpu_inst *vpu);
 
+/**
+ * vpu_dec_get_param - get param from scp
+ *
+ * @vpu : instance for vdec_vpu_inst
+ * @data: meta data to pass bitstream info to VPU decoder
+ * @len : meta data length
+ * @param_type : get param type
+ */
+int vpu_dec_get_param(struct vdec_vpu_inst *vpu, uint32_t *data,
+	unsigned int len, unsigned int param_type);
+
 #endif
-- 
2.25.1

[PATCH v1, 04/12] media: mtk-vcodec: Read max resolution from dec_capability

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

Reads supported max resolution from dec_capability.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_dec.c      | 31 +++++++++++--------
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  4 +++
 2 files changed, 22 insertions(+), 13 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 87891ebd7246..ad7cb9d4f280 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -152,13 +152,15 @@ void mtk_vcodec_dec_set_default_params(struct mtk_vcodec_ctx *ctx)
 	q_data->coded_height = DFT_CFG_HEIGHT;
 	q_data->fmt = ctx->dev->vdec_pdata->default_cap_fmt;
 	q_data->field = V4L2_FIELD_NONE;
+	ctx->max_width = MTK_VDEC_MAX_W;
+	ctx->max_height = MTK_VDEC_MAX_H;
 
 	v4l_bound_align_image(&q_data->coded_width,
 				MTK_VDEC_MIN_W,
-				MTK_VDEC_MAX_W, 4,
+				ctx->max_width, 4,
 				&q_data->coded_height,
 				MTK_VDEC_MIN_H,
-				MTK_VDEC_MAX_H, 5, 6);
+				ctx->max_height, 5, 6);
 
 	q_data->sizeimage[0] = q_data->coded_width * q_data->coded_height;
 	q_data->bytesperline[0] = q_data->coded_width;
@@ -217,17 +219,17 @@ static int vidioc_vdec_subscribe_evt(struct v4l2_fh *fh,
 	}
 }
 
-static int vidioc_try_fmt(struct v4l2_format *f,
-			  const struct mtk_video_fmt *fmt)
+static int vidioc_try_fmt(struct mtk_vcodec_ctx *ctx,
+	struct v4l2_format *f, const struct mtk_video_fmt *fmt)
 {
 	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
 
 	pix_fmt_mp->field = V4L2_FIELD_NONE;
 
 	pix_fmt_mp->width =
-		clamp(pix_fmt_mp->width, MTK_VDEC_MIN_W, MTK_VDEC_MAX_W);
+		clamp(pix_fmt_mp->width, MTK_VDEC_MIN_W, ctx->max_width);
 	pix_fmt_mp->height =
-		clamp(pix_fmt_mp->height, MTK_VDEC_MIN_H, MTK_VDEC_MAX_H);
+		clamp(pix_fmt_mp->height, MTK_VDEC_MIN_H, ctx->max_height);
 
 	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
 		pix_fmt_mp->num_planes = 1;
@@ -245,16 +247,16 @@ static int vidioc_try_fmt(struct v4l2_format *f,
 		tmp_h = pix_fmt_mp->height;
 		v4l_bound_align_image(&pix_fmt_mp->width,
 					MTK_VDEC_MIN_W,
-					MTK_VDEC_MAX_W, 6,
+					ctx->max_width, 6,
 					&pix_fmt_mp->height,
 					MTK_VDEC_MIN_H,
-					MTK_VDEC_MAX_H, 6, 9);
+					ctx->max_height, 6, 9);
 
 		if (pix_fmt_mp->width < tmp_w &&
-			(pix_fmt_mp->width + 64) <= MTK_VDEC_MAX_W)
+			(pix_fmt_mp->width + 64) <= ctx->max_width)
 			pix_fmt_mp->width += 64;
 		if (pix_fmt_mp->height < tmp_h &&
-			(pix_fmt_mp->height + 64) <= MTK_VDEC_MAX_H)
+			(pix_fmt_mp->height + 64) <= ctx->max_height)
 			pix_fmt_mp->height += 64;
 
 		mtk_v4l2_debug(0,
@@ -294,7 +296,7 @@ static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 		fmt = mtk_vdec_find_format(f, dec_pdata);
 	}
 
-	return vidioc_try_fmt(f, fmt);
+	return vidioc_try_fmt(ctx, f, fmt);
 }
 
 static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
@@ -317,7 +319,7 @@ static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
 		return -EINVAL;
 	}
 
-	return vidioc_try_fmt(f, fmt);
+	return vidioc_try_fmt(ctx, f, fmt);
 }
 
 static int vidioc_vdec_g_selection(struct file *file, void *priv,
@@ -445,7 +447,7 @@ static int vidioc_vdec_s_fmt(struct file *file, void *priv,
 		return -EINVAL;
 
 	q_data->fmt = fmt;
-	vidioc_try_fmt(f, q_data->fmt);
+	vidioc_try_fmt(ctx, f, q_data->fmt);
 	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
 		q_data->sizeimage[0] = pix_mp->plane_fmt[0].sizeimage;
 		q_data->coded_width = pix_mp->width;
@@ -547,6 +549,9 @@ static int vidioc_enum_framesizes(struct file *file, void *priv,
 				fsize->stepwise.min_height,
 				fsize->stepwise.max_height,
 				fsize->stepwise.step_height);
+
+		ctx->max_width = fsize->stepwise.max_width;
+		ctx->max_height = fsize->stepwise.max_height;
 		return 0;
 	}
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index fc2c23deb825..1938b94eb147 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -288,6 +288,8 @@ struct vdec_pic_info {
  *	  mtk_video_dec_buf.
  * @hw_id: hardware index used to identify different hardware.
  *
+ * @max_width: hardware supported max width
+ * @max_height: hardware supported max height
  * @msg_queue: msg queue used to store lat buffer information.
  */
 struct mtk_vcodec_ctx {
@@ -334,6 +336,8 @@ struct mtk_vcodec_ctx {
 	struct mutex lock;
 	int hw_id;
 
+	unsigned int max_width;
+	unsigned int max_height;
 	struct vdec_msg_queue msg_queue;
 };
 
-- 
2.25.1

[PATCH v1, 05/12] media: mtk-vcodec: Call v4l2_m2m_set_dst_buffered() set capture buffer buffered

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

For lat and core architecture, just need to get output buffer for lat
decode. For core hardware need to get lat's message and capture buffer.
In order to let lat and core decode in parallel, need to set capture
buffer buffered.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
index 1a4013f4fb83..0d19b0f67fd6 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
@@ -315,6 +315,9 @@ static void mtk_init_vdec_params(struct mtk_vcodec_ctx *ctx)
 	src_vq = v4l2_m2m_get_vq(ctx->m2m_ctx,
 				 V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
 
+	if (ctx->dev->vdec_pdata->hw_arch != MTK_VDEC_PURE_SINGLE_CORE)
+		v4l2_m2m_set_dst_buffered(ctx->m2m_ctx, 1);
+
 	/* Support request api for output plane */
 	src_vq->supports_requests = true;
 	src_vq->requires_requests = true;
-- 
2.25.1

[PATCH v1, 06/12] media: mtk-vcodec: Refactor get and put capture buffer flow

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

For lat and core decode in parallel, need to get capture buffer
when core start to decode and put put capture buffer to display
list when core decode done.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 .../mtk-vcodec/mtk_vcodec_dec_stateless.c     | 122 ++++++++++++------
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   5 +-
 .../mtk-vcodec/vdec/vdec_h264_req_if.c        |  16 ++-
 3 files changed, 103 insertions(+), 40 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
index 0d19b0f67fd6..f3036c3f223b 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
@@ -108,38 +108,89 @@ static const struct mtk_codec_framesizes mtk_vdec_framesizes[] = {
 
 #define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_vdec_framesizes)
 
-static void mtk_vdec_stateless_set_dst_payload(struct mtk_vcodec_ctx *ctx,
-					       struct vdec_fb *fb)
+static void mtk_vdec_stateless_out_to_done(struct mtk_vcodec_ctx *ctx,
+					   struct mtk_vcodec_mem *bs, int error)
 {
-	struct mtk_video_dec_buf *vdec_frame_buf =
-		container_of(fb, struct mtk_video_dec_buf, frame_buffer);
-	struct vb2_v4l2_buffer *vb = &vdec_frame_buf->m2m_buf.vb;
-	unsigned int cap_y_size = ctx->q_data[MTK_Q_DATA_DST].sizeimage[0];
+	struct mtk_video_dec_buf *out_buf;
+	struct vb2_v4l2_buffer *vb;
 
-	vb2_set_plane_payload(&vb->vb2_buf, 0, cap_y_size);
-	if (ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 2) {
-		unsigned int cap_c_size =
-			ctx->q_data[MTK_Q_DATA_DST].sizeimage[1];
+	if (!bs) {
+		mtk_v4l2_err("Free bitstream buffer fail.");
+		return;
+	}
+	out_buf = container_of(bs, struct mtk_video_dec_buf, bs_buffer);
+	vb = &out_buf->m2m_buf.vb;
+
+	mtk_v4l2_debug(2,
+		"Free bitsteam buffer id = %d to done_list",
+		vb->vb2_buf.index);
+
+	v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+	if (error) {
+		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
+		if (error == -EIO)
+			out_buf->error = true;
+	} else {
+		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_DONE);
+	}
+}
 
-		vb2_set_plane_payload(&vb->vb2_buf, 1, cap_c_size);
+static void mtk_vdec_stateless_cap_to_disp(struct mtk_vcodec_ctx *ctx,
+					   struct vdec_fb *fb, int error)
+{
+	struct mtk_video_dec_buf *vdec_frame_buf;
+	struct vb2_v4l2_buffer *vb;
+	unsigned int cap_y_size, cap_c_size;
+
+	if (!fb) {
+		mtk_v4l2_err("Free frame buffer fail.");
+		return;
 	}
+	vdec_frame_buf = container_of(fb, struct mtk_video_dec_buf,
+				      frame_buffer);
+	vb = &vdec_frame_buf->m2m_buf.vb;
+
+	cap_y_size = ctx->q_data[MTK_Q_DATA_DST].sizeimage[0];
+	cap_c_size = ctx->q_data[MTK_Q_DATA_DST].sizeimage[1];
+
+	v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+
+	vb2_set_plane_payload(&vb->vb2_buf, 0, cap_y_size);
+	if (ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 2)
+		vb2_set_plane_payload(&vb->vb2_buf, 1, cap_c_size);
+
+	mtk_v4l2_debug(2,
+		"Free frame buffer id = %d to done_list",
+		vb->vb2_buf.index);
+	if (error)
+		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_ERROR);
+	else
+		v4l2_m2m_buf_done(vb, VB2_BUF_STATE_DONE);
 }
 
-static struct vdec_fb *vdec_get_cap_buffer(struct mtk_vcodec_ctx *ctx,
-					   struct vb2_v4l2_buffer *vb2_v4l2)
+static struct vdec_fb *vdec_get_cap_buffer(struct mtk_vcodec_ctx *ctx)
 {
-	struct mtk_video_dec_buf *framebuf =
-		container_of(vb2_v4l2, struct mtk_video_dec_buf, m2m_buf.vb);
-	struct vdec_fb *pfb = &framebuf->frame_buffer;
-	struct vb2_buffer *dst_buf = &vb2_v4l2->vb2_buf;
+	struct mtk_video_dec_buf *framebuf;
+	struct vb2_v4l2_buffer *vb2_v4l2;
+	struct vb2_buffer *dst_buf;
+	struct vdec_fb *pfb;
+
+	vb2_v4l2 = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
+	if (!vb2_v4l2) {
+		mtk_v4l2_debug(1, "[%d] dst_buf empty!!", ctx->id);
+		return NULL;
+	}
+
+	dst_buf = &vb2_v4l2->vb2_buf;
+	framebuf = container_of(vb2_v4l2, struct mtk_video_dec_buf, m2m_buf.vb);
 
 	pfb = &framebuf->frame_buffer;
-	pfb->base_y.va = NULL;
+	pfb->base_y.va = vb2_plane_vaddr(dst_buf, 0);
 	pfb->base_y.dma_addr = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
 	pfb->base_y.size = ctx->q_data[MTK_Q_DATA_DST].sizeimage[0];
 
 	if (ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 2) {
-		pfb->base_c.va = NULL;
+		pfb->base_c.va = vb2_plane_vaddr(dst_buf, 1);
 		pfb->base_c.dma_addr =
 			vb2_dma_contig_plane_dma_addr(dst_buf, 1);
 		pfb->base_c.size = ctx->q_data[MTK_Q_DATA_DST].sizeimage[1];
@@ -163,12 +214,11 @@ static void mtk_vdec_worker(struct work_struct *work)
 	struct mtk_vcodec_ctx *ctx =
 		container_of(work, struct mtk_vcodec_ctx, decode_work);
 	struct mtk_vcodec_dev *dev = ctx->dev;
-	struct vb2_v4l2_buffer *vb2_v4l2_src, *vb2_v4l2_dst;
+	struct vb2_v4l2_buffer *vb2_v4l2_src;
 	struct vb2_buffer *vb2_src;
 	struct mtk_vcodec_mem *bs_src;
 	struct mtk_video_dec_buf *dec_buf_src;
 	struct media_request *src_buf_req;
-	struct vdec_fb *dst_buf;
 	bool res_chg = false;
 	int ret;
 
@@ -179,13 +229,6 @@ static void mtk_vdec_worker(struct work_struct *work)
 		return;
 	}
 
-	vb2_v4l2_dst = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
-	if (!vb2_v4l2_dst) {
-		v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
-		mtk_v4l2_debug(1, "[%d] no available destination buffer", ctx->id);
-		return;
-	}
-
 	vb2_src = &vb2_v4l2_src->vb2_buf;
 	dec_buf_src = container_of(vb2_v4l2_src, struct mtk_video_dec_buf,
 				   m2m_buf.vb);
@@ -194,9 +237,15 @@ static void mtk_vdec_worker(struct work_struct *work)
 	mtk_v4l2_debug(3, "[%d] (%d) id=%d, vb=%p", ctx->id,
 		       vb2_src->vb2_queue->type, vb2_src->index, vb2_src);
 
-	bs_src->va = NULL;
+	bs_src->va = vb2_plane_vaddr(vb2_src, 0);
 	bs_src->dma_addr = vb2_dma_contig_plane_dma_addr(vb2_src, 0);
 	bs_src->size = (size_t)vb2_src->planes[0].bytesused;
+	if (!bs_src->va) {
+		v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
+		mtk_v4l2_err("[%d] id=%d source buffer is NULL", ctx->id,
+			vb2_src->index);
+		return;
+	}
 
 	mtk_v4l2_debug(3, "[%d] Bitstream VA=%p DMA=%pad Size=%zx vb=%p",
 		       ctx->id, bs_src->va, &bs_src->dma_addr, bs_src->size, vb2_src);
@@ -207,9 +256,7 @@ static void mtk_vdec_worker(struct work_struct *work)
 	else
 		mtk_v4l2_err("vb2 buffer media request is NULL");
 
-	dst_buf = vdec_get_cap_buffer(ctx, vb2_v4l2_dst);
-	v4l2_m2m_buf_copy_metadata(vb2_v4l2_src, vb2_v4l2_dst, true);
-	ret = vdec_if_decode(ctx, bs_src, dst_buf, &res_chg);
+	ret = vdec_if_decode(ctx, bs_src, NULL, &res_chg);
 	if (ret) {
 		mtk_v4l2_err(" <===[%d], src_buf[%d] sz=0x%zx pts=%llu vdec_if_decode() ret=%d res_chg=%d===>",
 			     ctx->id, vb2_src->index, bs_src->size,
@@ -221,12 +268,9 @@ static void mtk_vdec_worker(struct work_struct *work)
 		}
 	}
 
-	mtk_vdec_stateless_set_dst_payload(ctx, dst_buf);
-
-	v4l2_m2m_buf_done_and_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx,
-					 ret ? VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
-
+	mtk_vdec_stateless_out_to_done(ctx, bs_src, ret);
 	v4l2_ctrl_request_complete(src_buf_req, &ctx->ctrl_hdl);
+	v4l2_m2m_job_finish(dev->m2m_dev_dec, ctx->m2m_ctx);
 }
 
 static void vb2ops_vdec_stateless_buf_queue(struct vb2_buffer *vb)
@@ -359,6 +403,8 @@ const struct mtk_vcodec_dec_pdata mtk_vdec_8183_pdata = {
 	.uses_stateless_api = true,
 	.worker = mtk_vdec_worker,
 	.flush_decoder = mtk_vdec_flush_decoder,
+	.cap_to_disp = mtk_vdec_stateless_cap_to_disp,
+	.get_cap_buffer = vdec_get_cap_buffer,
 	.is_subdev_supported = false,
 	.hw_arch = MTK_VDEC_PURE_SINGLE_CORE,
 };
@@ -377,6 +423,8 @@ const struct mtk_vcodec_dec_pdata mtk_lat_sig_core_pdata = {
 	.uses_stateless_api = true,
 	.worker = mtk_vdec_worker,
 	.flush_decoder = mtk_vdec_flush_decoder,
+	.cap_to_disp = mtk_vdec_stateless_cap_to_disp,
+	.get_cap_buffer = vdec_get_cap_buffer,
 	.is_subdev_supported = true,
 	.hw_arch = MTK_VDEC_LAT_SINGLE_CORE,
 };
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 1938b94eb147..b9c6649da1b1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -362,7 +362,8 @@ enum mtk_vdec_hw_arch {
  * @ctrls_setup: init vcodec dec ctrls
  * @worker: worker to start a decode job
  * @flush_decoder: function that flushes the decoder
- *
+ * @get_cap_buffer: get capture buffer from capture queue
+ * @cap_to_disp: put capture buffer to disp list
  * @vdec_vb2_ops: struct vb2_ops
  *
  * @vdec_formats: supported video decoder formats
@@ -384,6 +385,8 @@ struct mtk_vcodec_dec_pdata {
 	int (*ctrls_setup)(struct mtk_vcodec_ctx *ctx);
 	void (*worker)(struct work_struct *work);
 	int (*flush_decoder)(struct mtk_vcodec_ctx *ctx);
+	struct vdec_fb *(*get_cap_buffer)(struct mtk_vcodec_ctx *ctx);
+	void (*cap_to_disp)(struct mtk_vcodec_ctx *ctx, struct vdec_fb *fb, int error);
 
 	struct vb2_ops *vdec_vb2_ops;
 
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
index d402fc4bda69..870676c2065e 100644
--- a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
@@ -670,32 +670,42 @@ static void vdec_h264_slice_deinit(void *h_vdec)
 }
 
 static int vdec_h264_slice_decode(void *h_vdec, struct mtk_vcodec_mem *bs,
-				  struct vdec_fb *fb, bool *res_chg)
+				  struct vdec_fb *unused, bool *res_chg)
 {
 	struct vdec_h264_slice_inst *inst = h_vdec;
 	const struct v4l2_ctrl_h264_decode_params *dec_params =
 		get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_DECODE_PARAMS);
 	struct vdec_vpu_inst *vpu = &inst->vpu;
+	struct mtk_video_dec_buf *src_buf_info;
+	struct mtk_video_dec_buf *dst_buf_info;
+	struct vdec_fb *fb;
 	u32 data[2];
 	u64 y_fb_dma;
 	u64 c_fb_dma;
 	int err;
 
+	inst->num_nalu++;
 	/* bs NULL means flush decoder */
 	if (!bs)
 		return vpu_dec_reset(vpu);
 
+	fb = inst->ctx->dev->vdec_pdata->get_cap_buffer(inst->ctx);
+	src_buf_info = container_of(bs, struct mtk_video_dec_buf, bs_buffer);
+	dst_buf_info = container_of(fb, struct mtk_video_dec_buf, frame_buffer);
+
 	y_fb_dma = fb ? (u64)fb->base_y.dma_addr : 0;
 	c_fb_dma = fb ? (u64)fb->base_c.dma_addr : 0;
 
 	mtk_vcodec_debug(inst, "+ [%d] FB y_dma=%llx c_dma=%llx va=%p",
-			 ++inst->num_nalu, y_fb_dma, c_fb_dma, fb);
+			 inst->num_nalu, y_fb_dma, c_fb_dma, fb);
 
 	inst->vsi_ctx.dec.bs_dma = (uint64_t)bs->dma_addr;
 	inst->vsi_ctx.dec.y_fb_dma = y_fb_dma;
 	inst->vsi_ctx.dec.c_fb_dma = c_fb_dma;
 	inst->vsi_ctx.dec.vdec_fb_va = (u64)(uintptr_t)fb;
 
+	v4l2_m2m_buf_copy_metadata(&src_buf_info->m2m_buf.vb,
+		&dst_buf_info->m2m_buf.vb, true);
 	get_vdec_decode_parameters(inst);
 	data[0] = bs->size;
 	/*
@@ -734,6 +744,8 @@ static int vdec_h264_slice_decode(void *h_vdec, struct mtk_vcodec_mem *bs,
 
 	memcpy(&inst->vsi_ctx, inst->vpu.vsi, sizeof(inst->vsi_ctx));
 	mtk_vcodec_debug(inst, "\n - NALU[%d]", inst->num_nalu);
+
+	inst->ctx->dev->vdec_pdata->cap_to_disp(inst->ctx, fb, 0);
 	return 0;
 
 err_free_fb_out:
-- 
2.25.1

[PATCH v1, 07/12] media: mtk-vcodec: Refactor supported vdec formats and framesizes

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

Supported output and capture format types for mt8192 are different
with mt8183. Needs to get format types according to decoder capability.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_dec.c      |   8 +-
 .../mtk-vcodec/mtk_vcodec_dec_stateful.c      |  13 +-
 .../mtk-vcodec/mtk_vcodec_dec_stateless.c     | 117 +++++++++++++-----
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  13 +-
 4 files changed, 107 insertions(+), 44 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index ad7cb9d4f280..0b14f816bf58 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -26,7 +26,7 @@ mtk_vdec_find_format(struct v4l2_format *f,
 	const struct mtk_video_fmt *fmt;
 	unsigned int k;
 
-	for (k = 0; k < dec_pdata->num_formats; k++) {
+	for (k = 0; k < *dec_pdata->num_formats; k++) {
 		fmt = &dec_pdata->vdec_formats[k];
 		if (fmt->fourcc == f->fmt.pix_mp.pixelformat)
 			return fmt;
@@ -527,7 +527,7 @@ static int vidioc_enum_framesizes(struct file *file, void *priv,
 	if (fsize->index != 0)
 		return -EINVAL;
 
-	for (i = 0; i < dec_pdata->num_framesizes; ++i) {
+	for (i = 0; i < *dec_pdata->num_framesizes; ++i) {
 		if (fsize->pixel_format != dec_pdata->vdec_framesizes[i].fourcc)
 			continue;
 
@@ -566,7 +566,7 @@ static int vidioc_enum_fmt(struct v4l2_fmtdesc *f, void *priv,
 	const struct mtk_video_fmt *fmt;
 	int i, j = 0;
 
-	for (i = 0; i < dec_pdata->num_formats; i++) {
+	for (i = 0; i < *dec_pdata->num_formats; i++) {
 		if (output_queue &&
 		    dec_pdata->vdec_formats[i].type != MTK_FMT_DEC)
 			continue;
@@ -579,7 +579,7 @@ static int vidioc_enum_fmt(struct v4l2_fmtdesc *f, void *priv,
 		++j;
 	}
 
-	if (i == dec_pdata->num_formats)
+	if (i == *dec_pdata->num_formats)
 		return -EINVAL;
 
 	fmt = &dec_pdata->vdec_formats[i];
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
index 7966c132be8f..3f33beb9c551 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
@@ -37,7 +37,9 @@ static const struct mtk_video_fmt mtk_video_formats[] = {
 	},
 };
 
-#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
+static const unsigned int num_supported_formats =
+	ARRAY_SIZE(mtk_video_formats);
+
 #define DEFAULT_OUT_FMT_IDX 0
 #define DEFAULT_CAP_FMT_IDX 3
 
@@ -59,7 +61,8 @@ static const struct mtk_codec_framesizes mtk_vdec_framesizes[] = {
 	},
 };
 
-#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_vdec_framesizes)
+static const unsigned int num_supported_framesize =
+	ARRAY_SIZE(mtk_vdec_framesizes);
 
 /*
  * This function tries to clean all display buffers, the buffers will return
@@ -235,7 +238,7 @@ static void mtk_vdec_update_fmt(struct mtk_vcodec_ctx *ctx,
 	unsigned int k;
 
 	dst_q_data = &ctx->q_data[MTK_Q_DATA_DST];
-	for (k = 0; k < NUM_FORMATS; k++) {
+	for (k = 0; k < num_supported_formats; k++) {
 		fmt = &mtk_video_formats[k];
 		if (fmt->fourcc == pixelformat) {
 			mtk_v4l2_debug(1, "Update cap fourcc(%d -> %d)",
@@ -617,11 +620,11 @@ const struct mtk_vcodec_dec_pdata mtk_vdec_8173_pdata = {
 	.ctrls_setup = mtk_vcodec_dec_ctrls_setup,
 	.vdec_vb2_ops = &mtk_vdec_frame_vb2_ops,
 	.vdec_formats = mtk_video_formats,
-	.num_formats = NUM_FORMATS,
+	.num_formats = &num_supported_formats,
 	.default_out_fmt = &mtk_video_formats[DEFAULT_OUT_FMT_IDX],
 	.default_cap_fmt = &mtk_video_formats[DEFAULT_CAP_FMT_IDX],
 	.vdec_framesizes = mtk_vdec_framesizes,
-	.num_framesizes = NUM_SUPPORTED_FRAMESIZE,
+	.num_framesizes = &num_supported_framesize,
 	.worker = mtk_vdec_worker,
 	.flush_decoder = mtk_vdec_flush_decoder,
 	.is_subdev_supported = false,
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
index f3036c3f223b..01bb96f3b30e 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
@@ -81,33 +81,23 @@ static const struct mtk_stateless_control mtk_stateless_controls[] = {
 
 #define NUM_CTRLS ARRAY_SIZE(mtk_stateless_controls)
 
-static const struct mtk_video_fmt mtk_video_formats[] = {
-	{
-		.fourcc = V4L2_PIX_FMT_H264_SLICE,
-		.type = MTK_FMT_DEC,
-		.num_planes = 1,
-	},
-	{
-		.fourcc = V4L2_PIX_FMT_MM21,
-		.type = MTK_FMT_FRAME,
-		.num_planes = 2,
-	},
+static struct mtk_video_fmt mtk_video_formats[2];
+static struct mtk_codec_framesizes mtk_vdec_framesizes[1];
+
+static struct mtk_video_fmt default_out_format;
+static struct mtk_video_fmt default_cap_format;
+static unsigned int num_formats = 0;
+static unsigned int num_framesizes = 0;
+
+static struct v4l2_frmsize_stepwise stepwise_fhd = {
+	.min_width = MTK_VDEC_MIN_W,
+	.max_width = MTK_VDEC_MAX_W,
+	.step_width = 16,
+	.min_height = MTK_VDEC_MIN_H,
+	.max_height = MTK_VDEC_MAX_H,
+	.step_height = 16
 };
 
-#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
-#define DEFAULT_OUT_FMT_IDX    0
-#define DEFAULT_CAP_FMT_IDX    1
-
-static const struct mtk_codec_framesizes mtk_vdec_framesizes[] = {
-	{
-		.fourcc	= V4L2_PIX_FMT_H264_SLICE,
-		.stepwise = {  MTK_VDEC_MIN_W, MTK_VDEC_MAX_W, 16,
-				MTK_VDEC_MIN_H, MTK_VDEC_MAX_H, 16 },
-	},
-};
-
-#define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_vdec_framesizes)
-
 static void mtk_vdec_stateless_out_to_done(struct mtk_vcodec_ctx *ctx,
 					   struct mtk_vcodec_mem *bs, int error)
 {
@@ -352,6 +342,63 @@ const struct media_device_ops mtk_vcodec_media_ops = {
 	.req_queue	= v4l2_m2m_request_queue,
 };
 
+static void mtk_vcodec_add_formats(unsigned int fourcc,
+	struct mtk_vcodec_ctx *ctx)
+{
+	struct mtk_vcodec_dev *dev = ctx->dev;
+	const struct mtk_vcodec_dec_pdata *pdata = dev->vdec_pdata;
+	int count_formats = *pdata->num_formats;
+	int count_framesizes = *pdata->num_framesizes;
+
+	switch (fourcc) {
+	case V4L2_PIX_FMT_H264_SLICE:
+		mtk_video_formats[count_formats].fourcc = fourcc;
+		mtk_video_formats[count_formats].type = MTK_FMT_DEC;
+		mtk_video_formats[count_formats].num_planes = 1;
+
+		mtk_vdec_framesizes[count_framesizes].fourcc = fourcc;
+		mtk_vdec_framesizes[count_framesizes].stepwise = stepwise_fhd;
+		num_framesizes++;
+		break;
+	case V4L2_PIX_FMT_MM21:
+		mtk_video_formats[count_formats].fourcc = fourcc;
+		mtk_video_formats[count_formats].type = MTK_FMT_FRAME;
+		mtk_video_formats[count_formats].num_planes = 2;
+		break;
+	default:
+		mtk_v4l2_err("Can not add unsupported format type");
+		return;
+	}
+
+	num_formats++;
+	mtk_v4l2_debug(3, "num_formats: %d num_frames:%d dec_capability: 0x%x",
+		*(pdata->num_formats), *(pdata->num_framesizes),
+		ctx->dev->dec_capability);
+}
+
+static void mtk_vcodec_get_supported_formats(struct mtk_vcodec_ctx *ctx)
+{
+	int cap_format_count = 0, out_format_count = 0;
+
+	if (num_formats && num_framesizes)
+		return;
+
+	if (ctx->dev->dec_capability & MTK_VDEC_FORMAT_MM21) {
+		mtk_vcodec_add_formats(V4L2_PIX_FMT_MM21, ctx);
+		cap_format_count++;
+	}
+	if (ctx->dev->dec_capability & MTK_VDEC_FORMAT_H264_SLICE) {
+		mtk_vcodec_add_formats(V4L2_PIX_FMT_H264_SLICE, ctx);
+		out_format_count++;
+	}
+
+	if (cap_format_count)
+		default_cap_format = mtk_video_formats[cap_format_count - 1];
+	if (out_format_count)
+		default_out_format =
+			mtk_video_formats[cap_format_count + out_format_count - 1];
+}
+
 static void mtk_init_vdec_params(struct mtk_vcodec_ctx *ctx)
 {
 	struct vb2_queue *src_vq;
@@ -361,6 +408,10 @@ static void mtk_init_vdec_params(struct mtk_vcodec_ctx *ctx)
 
 	if (ctx->dev->vdec_pdata->hw_arch != MTK_VDEC_PURE_SINGLE_CORE)
 		v4l2_m2m_set_dst_buffered(ctx->m2m_ctx, 1);
+	else
+		ctx->dev->dec_capability |=
+			MTK_VDEC_FORMAT_H264_SLICE | MTK_VDEC_FORMAT_MM21;
+	mtk_vcodec_get_supported_formats(ctx);
 
 	/* Support request api for output plane */
 	src_vq->supports_requests = true;
@@ -395,11 +446,11 @@ const struct mtk_vcodec_dec_pdata mtk_vdec_8183_pdata = {
 	.ctrls_setup = mtk_vcodec_dec_ctrls_setup,
 	.vdec_vb2_ops = &mtk_vdec_request_vb2_ops,
 	.vdec_formats = mtk_video_formats,
-	.num_formats = NUM_FORMATS,
-	.default_out_fmt = &mtk_video_formats[DEFAULT_OUT_FMT_IDX],
-	.default_cap_fmt = &mtk_video_formats[DEFAULT_CAP_FMT_IDX],
+	.num_formats = &num_formats,
+	.default_out_fmt = &default_out_format,
+	.default_cap_fmt = &default_cap_format,
 	.vdec_framesizes = mtk_vdec_framesizes,
-	.num_framesizes = NUM_SUPPORTED_FRAMESIZE,
+	.num_framesizes = &num_framesizes,
 	.uses_stateless_api = true,
 	.worker = mtk_vdec_worker,
 	.flush_decoder = mtk_vdec_flush_decoder,
@@ -415,11 +466,11 @@ const struct mtk_vcodec_dec_pdata mtk_lat_sig_core_pdata = {
 	.ctrls_setup = mtk_vcodec_dec_ctrls_setup,
 	.vdec_vb2_ops = &mtk_vdec_request_vb2_ops,
 	.vdec_formats = mtk_video_formats,
-	.num_formats = NUM_FORMATS,
-	.default_out_fmt = &mtk_video_formats[DEFAULT_OUT_FMT_IDX],
-	.default_cap_fmt = &mtk_video_formats[DEFAULT_CAP_FMT_IDX],
+	.num_formats = &num_formats,
+	.default_out_fmt = &default_out_format,
+	.default_cap_fmt = &default_cap_format,
 	.vdec_framesizes = mtk_vdec_framesizes,
-	.num_framesizes = NUM_SUPPORTED_FRAMESIZE,
+	.num_framesizes = &num_framesizes,
 	.uses_stateless_api = true,
 	.worker = mtk_vdec_worker,
 	.flush_decoder = mtk_vdec_flush_decoder,
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index b9c6649da1b1..077352453759 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -356,6 +356,15 @@ enum mtk_vdec_hw_arch {
 	MTK_VDEC_LAT_SINGLE_CORE,
 };
 
+/**
+ * struct mtk_vdec_format_types - Structure used to get supported
+ *		  format types according to decoder capability
+ */
+enum mtk_vdec_format_types {
+	MTK_VDEC_FORMAT_MM21 = 0x20,
+	MTK_VDEC_FORMAT_H264_SLICE = 0x100,
+};
+
 /**
  * struct mtk_vcodec_dec_pdata - compatible data for each IC
  * @init_vdec_params: init vdec params
@@ -391,12 +400,12 @@ struct mtk_vcodec_dec_pdata {
 	struct vb2_ops *vdec_vb2_ops;
 
 	const struct mtk_video_fmt *vdec_formats;
-	const int num_formats;
+	const int *num_formats;
 	const struct mtk_video_fmt *default_out_fmt;
 	const struct mtk_video_fmt *default_cap_fmt;
 
 	const struct mtk_codec_framesizes *vdec_framesizes;
-	const int num_framesizes;
+	const int *num_framesizes;
 
 	enum mtk_vdec_hw_arch hw_arch;
 
-- 
2.25.1

[PATCH v1, 08/12] media: mtk-vcodec: Add format to support MT21C

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

Needs to use mediatek compressed mode for mt8192 decoder.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 .../media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c   | 7 ++++++-
 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h         | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
index 01bb96f3b30e..21895fbed4c5 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
@@ -81,7 +81,7 @@ static const struct mtk_stateless_control mtk_stateless_controls[] = {
 
 #define NUM_CTRLS ARRAY_SIZE(mtk_stateless_controls)
 
-static struct mtk_video_fmt mtk_video_formats[2];
+static struct mtk_video_fmt mtk_video_formats[3];
 static struct mtk_codec_framesizes mtk_vdec_framesizes[1];
 
 static struct mtk_video_fmt default_out_format;
@@ -361,6 +361,7 @@ static void mtk_vcodec_add_formats(unsigned int fourcc,
 		num_framesizes++;
 		break;
 	case V4L2_PIX_FMT_MM21:
+	case V4L2_PIX_FMT_MT21C:
 		mtk_video_formats[count_formats].fourcc = fourcc;
 		mtk_video_formats[count_formats].type = MTK_FMT_FRAME;
 		mtk_video_formats[count_formats].num_planes = 2;
@@ -387,6 +388,10 @@ static void mtk_vcodec_get_supported_formats(struct mtk_vcodec_ctx *ctx)
 		mtk_vcodec_add_formats(V4L2_PIX_FMT_MM21, ctx);
 		cap_format_count++;
 	}
+	if (ctx->dev->dec_capability & MTK_VDEC_FORMAT_MT21C) {
+		mtk_vcodec_add_formats(V4L2_PIX_FMT_MT21C, ctx);
+		cap_format_count++;
+	}
 	if (ctx->dev->dec_capability & MTK_VDEC_FORMAT_H264_SLICE) {
 		mtk_vcodec_add_formats(V4L2_PIX_FMT_H264_SLICE, ctx);
 		out_format_count++;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 077352453759..773887578454 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -362,6 +362,7 @@ enum mtk_vdec_hw_arch {
  */
 enum mtk_vdec_format_types {
 	MTK_VDEC_FORMAT_MM21 = 0x20,
+	MTK_VDEC_FORMAT_MT21C = 0x40,
 	MTK_VDEC_FORMAT_H264_SLICE = 0x100,
 };
 
-- 
2.25.1

[PATCH v1, 09/12] media: mtk-vcodec: disable vp8 4K capability

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

For vp8 not support 4K, need to disable it.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 0b14f816bf58..0317a9930de1 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -534,7 +534,8 @@ static int vidioc_enum_framesizes(struct file *file, void *priv,
 		fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
 		fsize->stepwise = dec_pdata->vdec_framesizes[i].stepwise;
 		if (!(ctx->dev->dec_capability &
-				VCODEC_CAPABILITY_4K_DISABLED)) {
+				VCODEC_CAPABILITY_4K_DISABLED) &&
+				fsize->pixel_format != V4L2_PIX_FMT_VP8_FRAME) {
 			mtk_v4l2_debug(3, "4K is enabled");
 			fsize->stepwise.max_width =
 					VCODEC_DEC_4K_CODED_WIDTH;
-- 
2.25.1

[PATCH v1, 10/12] media: mtk-vcodec: Fix v4l2-compliance fail

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

Need to use default pic info when get pic info fail.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 0317a9930de1..4533cb44551d 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -480,11 +480,14 @@ static int vidioc_vdec_s_fmt(struct file *file, void *priv,
 		ctx->picinfo.pic_w = pix_mp->width;
 		ctx->picinfo.pic_h = pix_mp->height;
 
+		/*
+		 * If get pic info fail, need to use the default pic info params, or
+		 * v4l2-compliance will fail
+		 */
 		ret = vdec_if_get_param(ctx, GET_PARAM_PIC_INFO, &ctx->picinfo);
 		if (ret) {
 			mtk_v4l2_err("[%d]Error!! Get GET_PARAM_PICTURE_INFO Fail",
 				     ctx->id);
-			return -EINVAL;
 		}
 
 		ctx->last_decoded_picinfo = ctx->picinfo;
-- 
2.25.1

[PATCH v1, 11/12] media: mtk-vcodec: Extract H264 common code

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

Mt8192 can use some of common code with mt8183. Moves them to
a new file in order to reuse.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile    |   1 +
 .../mtk-vcodec/vdec/vdec_h264_req_common.c    | 303 ++++++++++++++
 .../mtk-vcodec/vdec/vdec_h264_req_common.h    | 247 +++++++++++
 .../mtk-vcodec/vdec/vdec_h264_req_if.c        | 386 +-----------------
 4 files changed, 571 insertions(+), 366 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_common.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_common.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 359619653a0e..3f41d748eee5 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -9,6 +9,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
 		vdec/vdec_vp8_if.o \
 		vdec/vdec_vp9_if.o \
 		vdec/vdec_h264_req_if.o \
+		vdec/vdec_h264_req_common.o \
 		mtk_vcodec_dec_drv.o \
 		vdec_drv_if.o \
 		vdec_vpu_if.o \
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_common.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_common.c
new file mode 100644
index 000000000000..7e248e247628
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_common.c
@@ -0,0 +1,303 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ * Author: Yunfei Dong <yunfei.dong@mediatek.com>
+ */
+
+#include "vdec_h264_req_common.h"
+
+/* get used parameters for sps/pps */
+#define GET_MTK_VDEC_FLAG(cond, flag) \
+	{ dst_param->cond = ((src_param->flags & flag) ? (1) : (0)); }
+#define GET_MTK_VDEC_PARAM(param) \
+	{ dst_param->param = src_param->param; }
+
+void *mtk_vdec_h264_get_ctrl_ptr(struct mtk_vcodec_ctx *ctx, int id)
+{
+	struct v4l2_ctrl *ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, id);
+
+	return ctrl->p_cur.p;
+}
+
+void mtk_vdec_h264_fill_dpb_info(struct mtk_vcodec_ctx *ctx,
+	struct slice_api_h264_decode_param *decode_params,
+	struct mtk_h264_dpb_info *h264_dpb_info)
+{
+	struct vb2_queue *vq;
+	struct vb2_buffer *vb;
+	struct vb2_v4l2_buffer *vb2_v4l2;
+	int index;
+
+	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+
+	for (index = 0; index < V4L2_H264_NUM_DPB_ENTRIES; index++) {
+		const struct slice_h264_dpb_entry *dpb;
+		int vb2_index;
+
+		dpb = &decode_params->dpb[index];
+		if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) {
+			h264_dpb_info[index].reference_flag = 0;
+			continue;
+		}
+
+		vb2_index = vb2_find_timestamp(vq, dpb->reference_ts, 0);
+		if (vb2_index < 0) {
+			dev_err(&ctx->dev->plat_dev->dev,
+				"Reference invalid: dpb_index(%d) reference_ts(%lld)",
+				index, dpb->reference_ts);
+			continue;
+		}
+
+		/* 1 for short term reference, 2 for long term reference */
+		if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM))
+			h264_dpb_info[index].reference_flag = 1;
+		else
+			h264_dpb_info[index].reference_flag = 2;
+
+		vb = vq->bufs[vb2_index];
+		vb2_v4l2 = container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
+		h264_dpb_info[index].field = vb2_v4l2->field;
+
+		h264_dpb_info[index].y_dma_addr =
+			vb2_dma_contig_plane_dma_addr(vb, 0);
+		if (ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 2)
+			h264_dpb_info[index].c_dma_addr =
+				vb2_dma_contig_plane_dma_addr(vb, 1);
+		else
+			h264_dpb_info[index].c_dma_addr =
+				h264_dpb_info[index].y_dma_addr +
+				ctx->picinfo.fb_sz[0];
+	}
+}
+
+void mtk_vdec_h264_copy_sps_params(struct mtk_h264_sps_param *dst_param,
+	const struct v4l2_ctrl_h264_sps *src_param)
+{
+	GET_MTK_VDEC_PARAM(chroma_format_idc);
+	GET_MTK_VDEC_PARAM(bit_depth_luma_minus8);
+	GET_MTK_VDEC_PARAM(bit_depth_chroma_minus8);
+	GET_MTK_VDEC_PARAM(log2_max_frame_num_minus4);
+	GET_MTK_VDEC_PARAM(pic_order_cnt_type);
+	GET_MTK_VDEC_PARAM(log2_max_pic_order_cnt_lsb_minus4);
+	GET_MTK_VDEC_PARAM(max_num_ref_frames);
+	GET_MTK_VDEC_PARAM(pic_width_in_mbs_minus1);
+	GET_MTK_VDEC_PARAM(pic_height_in_map_units_minus1);
+
+	GET_MTK_VDEC_FLAG(separate_colour_plane_flag,
+		V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE);
+	GET_MTK_VDEC_FLAG(qpprime_y_zero_transform_bypass_flag,
+		V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS);
+	GET_MTK_VDEC_FLAG(delta_pic_order_always_zero_flag,
+		V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO);
+	GET_MTK_VDEC_FLAG(frame_mbs_only_flag,
+		V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY);
+	GET_MTK_VDEC_FLAG(mb_adaptive_frame_field_flag,
+		V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD);
+	GET_MTK_VDEC_FLAG(direct_8x8_inference_flag,
+		V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE);
+}
+
+void mtk_vdec_h264_copy_pps_params(struct mtk_h264_pps_param *dst_param,
+	const struct v4l2_ctrl_h264_pps *src_param)
+{
+	GET_MTK_VDEC_PARAM(num_ref_idx_l0_default_active_minus1);
+	GET_MTK_VDEC_PARAM(num_ref_idx_l1_default_active_minus1);
+	GET_MTK_VDEC_PARAM(weighted_bipred_idc);
+	GET_MTK_VDEC_PARAM(pic_init_qp_minus26);
+	GET_MTK_VDEC_PARAM(chroma_qp_index_offset);
+	GET_MTK_VDEC_PARAM(second_chroma_qp_index_offset);
+
+	GET_MTK_VDEC_FLAG(entropy_coding_mode_flag,
+		V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE);
+	GET_MTK_VDEC_FLAG(pic_order_present_flag,
+		V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT);
+	GET_MTK_VDEC_FLAG(weighted_pred_flag,
+		V4L2_H264_PPS_FLAG_WEIGHTED_PRED);
+	GET_MTK_VDEC_FLAG(deblocking_filter_control_present_flag,
+		V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT);
+	GET_MTK_VDEC_FLAG(constrained_intra_pred_flag,
+		V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED);
+	GET_MTK_VDEC_FLAG(redundant_pic_cnt_present_flag,
+		V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT);
+	GET_MTK_VDEC_FLAG(transform_8x8_mode_flag,
+		V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE);
+	GET_MTK_VDEC_FLAG(scaling_matrix_present_flag,
+		V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT);
+}
+
+void mtk_vdec_h264_copy_slice_hd_params(
+	struct mtk_h264_slice_hd_param *dst_param,
+	const struct v4l2_ctrl_h264_slice_params *src_param,
+	const struct v4l2_ctrl_h264_decode_params *dec_param)
+{
+	int temp;
+
+	GET_MTK_VDEC_PARAM(first_mb_in_slice);
+	GET_MTK_VDEC_PARAM(slice_type);
+	GET_MTK_VDEC_PARAM(cabac_init_idc);
+	GET_MTK_VDEC_PARAM(slice_qp_delta);
+	GET_MTK_VDEC_PARAM(disable_deblocking_filter_idc);
+	GET_MTK_VDEC_PARAM(slice_alpha_c0_offset_div2);
+	GET_MTK_VDEC_PARAM(slice_beta_offset_div2);
+	GET_MTK_VDEC_PARAM(num_ref_idx_l0_active_minus1);
+	GET_MTK_VDEC_PARAM(num_ref_idx_l1_active_minus1);
+
+	dst_param->frame_num = dec_param->frame_num;
+	dst_param->pic_order_cnt_lsb = dec_param->pic_order_cnt_lsb;
+
+	dst_param->delta_pic_order_cnt_bottom =
+		dec_param->delta_pic_order_cnt_bottom;
+	dst_param->delta_pic_order_cnt0 =
+		dec_param->delta_pic_order_cnt0;
+	dst_param->delta_pic_order_cnt1 =
+		dec_param->delta_pic_order_cnt1;
+
+	temp = dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC;
+	dst_param->field_pic_flag = temp ? 1 : 0;
+
+	temp = dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_BOTTOM_FIELD;
+	dst_param->bottom_field_flag = temp ? 1 : 0;
+
+	GET_MTK_VDEC_FLAG(direct_spatial_mv_pred_flag,
+		V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED);
+}
+
+void mtk_vdec_h264_copy_scaling_matrix(
+	struct slice_api_h264_scaling_matrix *dst_matrix,
+	const struct v4l2_ctrl_h264_scaling_matrix *src_matrix)
+{
+	memcpy(dst_matrix->scaling_list_4x4, src_matrix->scaling_list_4x4,
+	       sizeof(dst_matrix->scaling_list_4x4));
+
+	memcpy(dst_matrix->scaling_list_8x8, src_matrix->scaling_list_8x8,
+	       sizeof(dst_matrix->scaling_list_8x8));
+}
+
+void mtk_vdec_h264_copy_decode_params(
+	struct slice_api_h264_decode_param *dst_params,
+	const struct v4l2_ctrl_h264_decode_params *src_params,
+	const struct v4l2_h264_dpb_entry dpb[V4L2_H264_NUM_DPB_ENTRIES])
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(dst_params->dpb); i++) {
+		struct slice_h264_dpb_entry *dst_entry = &dst_params->dpb[i];
+		const struct v4l2_h264_dpb_entry *src_entry = &dpb[i];
+
+		dst_entry->reference_ts = src_entry->reference_ts;
+		dst_entry->frame_num = src_entry->frame_num;
+		dst_entry->pic_num = src_entry->pic_num;
+		dst_entry->top_field_order_cnt = src_entry->top_field_order_cnt;
+		dst_entry->bottom_field_order_cnt =
+			src_entry->bottom_field_order_cnt;
+		dst_entry->flags = src_entry->flags;
+	}
+
+	/* num_slices is a leftover from the old H.264 support and is ignored
+	 * by the firmware.
+	 */
+	dst_params->num_slices = 0;
+	dst_params->nal_ref_idc = src_params->nal_ref_idc;
+	dst_params->top_field_order_cnt = src_params->top_field_order_cnt;
+	dst_params->bottom_field_order_cnt = src_params->bottom_field_order_cnt;
+	dst_params->flags = src_params->flags;
+}
+
+static bool mtk_vdec_h264_dpb_entry_match(
+	const struct v4l2_h264_dpb_entry *a,
+	const struct v4l2_h264_dpb_entry *b)
+{
+	return a->top_field_order_cnt == b->top_field_order_cnt &&
+	       a->bottom_field_order_cnt == b->bottom_field_order_cnt;
+}
+
+/*
+ * Move DPB entries of dec_param that refer to a frame already existing in dpb
+ * into the already existing slot in dpb, and move other entries into new slots.
+ *
+ * This function is an adaptation of the similarly-named function in
+ * hantro_h264.c.
+ */
+void mtk_vdec_h264_update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
+	struct v4l2_h264_dpb_entry *dpb)
+{
+	DECLARE_BITMAP(new, ARRAY_SIZE(dec_param->dpb)) = { 0, };
+	DECLARE_BITMAP(in_use, ARRAY_SIZE(dec_param->dpb)) = { 0, };
+	DECLARE_BITMAP(used, ARRAY_SIZE(dec_param->dpb)) = { 0, };
+	unsigned int i, j;
+
+	/* Disable all entries by default, and mark the ones in use. */
+	for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) {
+		if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
+			set_bit(i, in_use);
+		dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE;
+	}
+
+	/* Try to match new DPB entries with existing ones by their POCs. */
+	for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) {
+		const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i];
+
+		if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
+			continue;
+
+		/*
+		 * To cut off some comparisons, iterate only on target DPB
+		 * entries were already used.
+		 */
+		for_each_set_bit(j, in_use, ARRAY_SIZE(dec_param->dpb)) {
+			struct v4l2_h264_dpb_entry *cdpb;
+
+			cdpb = &dpb[j];
+			if (!mtk_vdec_h264_dpb_entry_match(cdpb, ndpb))
+				continue;
+
+			*cdpb = *ndpb;
+			set_bit(j, used);
+			/* Don't reiterate on this one. */
+			clear_bit(j, in_use);
+			break;
+		}
+
+		if (j == ARRAY_SIZE(dec_param->dpb))
+			set_bit(i, new);
+	}
+
+	/* For entries that could not be matched, use remaining free slots. */
+	for_each_set_bit(i, new, ARRAY_SIZE(dec_param->dpb)) {
+		const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i];
+		struct v4l2_h264_dpb_entry *cdpb;
+
+		/*
+		 * Both arrays are of the same sizes, so there is no way
+		 * we can end up with no space in target array, unless
+		 * something is buggy.
+		 */
+		j = find_first_zero_bit(used, ARRAY_SIZE(dec_param->dpb));
+		if (WARN_ON(j >= ARRAY_SIZE(dec_param->dpb)))
+			return;
+
+		cdpb = &dpb[j];
+		*cdpb = *ndpb;
+		set_bit(j, used);
+	}
+}
+
+unsigned int mtk_vdec_h264_get_mv_buf_size(
+	unsigned int width, unsigned int height)
+{
+	int unit_size = (width / MB_UNIT_LEN) * (height / MB_UNIT_LEN) + 8;
+
+	return HW_MB_STORE_SZ * unit_size;
+}
+
+int mtk_vdec_h264_find_start_code(unsigned char *data, unsigned int data_sz)
+{
+	if (data_sz > 3 && data[0] == 0 && data[1] == 0 && data[2] == 1)
+		return 3;
+
+	if (data_sz > 4 && data[0] == 0 && data[1] == 0 && data[2] == 0 &&
+	    data[3] == 1)
+		return 4;
+
+	return -1;
+}
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_common.h b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_common.h
new file mode 100644
index 000000000000..71372fcae168
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_common.h
@@ -0,0 +1,247 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ * Author: Yunfei Dong <yunfei.dong@mediatek.com>
+ */
+
+#ifndef _VDEC_H264_REQ_COMMON_H_
+#define _VDEC_H264_REQ_COMMON_H_
+
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <media/v4l2-h264.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "../mtk_vcodec_drv.h"
+
+#define NAL_NON_IDR_SLICE			0x01
+#define NAL_IDR_SLICE				0x05
+#define NAL_TYPE(value)				((value) & 0x1F)
+
+#define BUF_PREDICTION_SZ			(64 * 4096)
+#define MB_UNIT_LEN				16
+
+/* motion vector size (bytes) for every macro block */
+#define HW_MB_STORE_SZ				64
+
+#define H264_MAX_MV_NUM				32
+
+/**
+ * struct mtk_h264_dpb_info  - h264 dpb information
+ * @y_dma_addr: Y bitstream physical address
+ * @c_dma_addr: CbCr bitstream physical address
+ * @reference_flag: reference picture flag (short/long term reference picture)
+ * @field: field picture flag
+ */
+struct mtk_h264_dpb_info {
+	dma_addr_t y_dma_addr;
+	dma_addr_t c_dma_addr;
+	int reference_flag;
+	int field;
+};
+
+/**
+ * struct mtk_h264_sps_param  - parameters for sps
+ */
+struct mtk_h264_sps_param {
+	unsigned char chroma_format_idc;
+	unsigned char bit_depth_luma_minus8;
+	unsigned char bit_depth_chroma_minus8;
+	unsigned char log2_max_frame_num_minus4;
+	unsigned char pic_order_cnt_type;
+	unsigned char log2_max_pic_order_cnt_lsb_minus4;
+	unsigned char max_num_ref_frames;
+	unsigned char separate_colour_plane_flag;
+	unsigned short pic_width_in_mbs_minus1;
+	unsigned short pic_height_in_map_units_minus1;
+	unsigned int max_frame_nums;
+	unsigned char qpprime_y_zero_transform_bypass_flag;
+	unsigned char delta_pic_order_always_zero_flag;
+	unsigned char frame_mbs_only_flag;
+	unsigned char mb_adaptive_frame_field_flag;
+	unsigned char direct_8x8_inference_flag;
+	unsigned char reserved[3];
+};
+
+/**
+ * struct mtk_h264_pps_param  - parameters for pps
+ */
+struct mtk_h264_pps_param {
+	unsigned char num_ref_idx_l0_default_active_minus1;
+	unsigned char num_ref_idx_l1_default_active_minus1;
+	unsigned char weighted_bipred_idc;
+	char pic_init_qp_minus26;
+	char chroma_qp_index_offset;
+	char second_chroma_qp_index_offset;
+	unsigned char entropy_coding_mode_flag;
+	unsigned char pic_order_present_flag;
+	unsigned char deblocking_filter_control_present_flag;
+	unsigned char constrained_intra_pred_flag;
+	unsigned char weighted_pred_flag;
+	unsigned char redundant_pic_cnt_present_flag;
+	unsigned char transform_8x8_mode_flag;
+	unsigned char scaling_matrix_present_flag;
+	unsigned char reserved[2];
+};
+
+/**
+ * struct mtk_h264_slice_hd_param  - parameters for slice header
+ */
+struct mtk_h264_slice_hd_param {
+	unsigned int first_mb_in_slice;
+	unsigned int field_pic_flag;
+	unsigned int slice_type;
+	unsigned int frame_num;
+	int pic_order_cnt_lsb;
+	int delta_pic_order_cnt_bottom;
+	unsigned int bottom_field_flag;
+	unsigned int direct_spatial_mv_pred_flag;
+	int delta_pic_order_cnt0;
+	int delta_pic_order_cnt1;
+	unsigned int cabac_init_idc;
+	int slice_qp_delta;
+	unsigned int disable_deblocking_filter_idc;
+	int slice_alpha_c0_offset_div2;
+	int slice_beta_offset_div2;
+	unsigned int num_ref_idx_l0_active_minus1;
+	unsigned int num_ref_idx_l1_active_minus1;
+	unsigned int reserved;
+};
+
+struct slice_api_h264_scaling_matrix {
+	unsigned char scaling_list_4x4[6][16];
+	unsigned char scaling_list_8x8[6][64];
+};
+
+struct slice_h264_dpb_entry {
+	unsigned long long reference_ts;
+	unsigned short frame_num;
+	unsigned short pic_num;
+	/* Note that field is indicated by v4l2_buffer.field */
+	int top_field_order_cnt;
+	int bottom_field_order_cnt;
+	unsigned int flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
+};
+
+/**
+ * struct slice_api_h264_decode_param - parameters for decode.
+ */
+struct slice_api_h264_decode_param {
+	struct slice_h264_dpb_entry dpb[V4L2_H264_NUM_DPB_ENTRIES];
+	unsigned short num_slices;
+	unsigned short nal_ref_idc;
+	unsigned char ref_pic_list_p0[32];
+	unsigned char ref_pic_list_b0[32];
+	unsigned char ref_pic_list_b1[32];
+	int top_field_order_cnt;
+	int bottom_field_order_cnt;
+	unsigned int flags; /* V4L2_H264_DECODE_PARAM_FLAG_* */
+};
+
+/**
+ * struct h264_fb - h264 decode frame buffer information
+ * @vdec_fb_va  : virtual address of struct vdec_fb
+ * @y_fb_dma    : dma address of Y frame buffer (luma)
+ * @c_fb_dma    : dma address of C frame buffer (chroma)
+ * @poc         : picture order count of frame buffer
+ * @reserved    : for 8 bytes alignment
+ */
+struct h264_fb {
+	uint64_t vdec_fb_va;
+	uint64_t y_fb_dma;
+	uint64_t c_fb_dma;
+	int32_t poc;
+	uint32_t reserved;
+};
+
+/**
+ * mtk_vdec_h264_get_ctrl_ptr - get each CID contrl address.
+ * @ctx: v4l2 ctx
+ * @id: CID control ID
+ */
+void *mtk_vdec_h264_get_ctrl_ptr(struct mtk_vcodec_ctx *ctx, int id);
+
+/**
+ * mtk_vdec_h264_fill_dpb_info - get each CID contrl address.
+ * @ctx: v4l2 ctx
+ * @decode_params: slice decode params
+ * @h264_dpb_info: dpb buffer information
+ */
+void mtk_vdec_h264_fill_dpb_info(struct mtk_vcodec_ctx *ctx,
+	struct slice_api_h264_decode_param *decode_params,
+	struct mtk_h264_dpb_info *h264_dpb_info);
+
+/**
+ * mtk_vdec_h264_copy_sps_params - get sps params.
+ * @dst_params: sps params for hw decoder
+ * @src_params: sps params from user driver
+ */
+void mtk_vdec_h264_copy_sps_params(struct mtk_h264_sps_param *dst_param,
+	const struct v4l2_ctrl_h264_sps *src_param);
+
+/**
+ * mtk_vdec_h264_copy_pps_params - get pps params.
+ * @dst_params: pps params for hw decoder
+ * @src_params: pps params from user driver
+ */
+void mtk_vdec_h264_copy_pps_params(struct mtk_h264_pps_param *dst_param,
+	const struct v4l2_ctrl_h264_pps *src_param);
+
+/**
+ * mtk_vdec_h264_copy_slice_hd_params - get slice header params.
+ * @dst_params: slice params for hw decoder
+ * @src_params: slice params from user driver
+ * @dec_param: decode params from user driver
+ */
+void mtk_vdec_h264_copy_slice_hd_params(
+	struct mtk_h264_slice_hd_param *dst_param,
+	const struct v4l2_ctrl_h264_slice_params *src_param,
+	const struct v4l2_ctrl_h264_decode_params *dec_param);
+
+/**
+ * mtk_vdec_h264_copy_scaling_matrix - get each CID contrl address.
+ * @dst_matrix: scaling list params for hw decoder
+ * @src_matrix: scaling list params from user driver
+ */
+void mtk_vdec_h264_copy_scaling_matrix(
+	struct slice_api_h264_scaling_matrix *dst_matrix,
+	const struct v4l2_ctrl_h264_scaling_matrix *src_matrix);
+
+/**
+ * mtk_vdec_h264_copy_decode_params - get decode params.
+ * @dst_params: dst params for hw decoder
+ * @src_params: decode params from user driver
+ * @dpb: dpb information
+ */
+void mtk_vdec_h264_copy_decode_params(
+	struct slice_api_h264_decode_param *dst_params,
+	const struct v4l2_ctrl_h264_decode_params *src_params,
+	const struct v4l2_h264_dpb_entry dpb[V4L2_H264_NUM_DPB_ENTRIES]);
+
+
+/**
+ * mtk_vdec_h264_update_dpb - updata dpb list.
+ * @dec_param: v4l2 control decode params
+ * @dpb: dpb entry informaton
+ */
+void mtk_vdec_h264_update_dpb(
+	const struct v4l2_ctrl_h264_decode_params *dec_param,
+	struct v4l2_h264_dpb_entry *dpb);
+
+/**
+ * mtk_vdec_h264_find_start_code - find h264 start code using sofeware.
+ * @data: input buffer address
+ * @data_sz: input buffer size
+ */
+int mtk_vdec_h264_find_start_code(unsigned char *data, unsigned int data_sz);
+
+/**
+ * mtk_vdec_h264_get_mv_buf_size - get mv buffer size.
+ * @width: picture width
+ * @height: picture height
+ */
+unsigned int mtk_vdec_h264_get_mv_buf_size(
+	unsigned int width, unsigned int height);
+
+#endif
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
index 870676c2065e..985a1b900257 100644
--- a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c
@@ -12,109 +12,7 @@
 #include "../vdec_drv_base.h"
 #include "../vdec_drv_if.h"
 #include "../vdec_vpu_if.h"
-
-#define BUF_PREDICTION_SZ			(64 * 4096)
-#define MB_UNIT_LEN				16
-
-/* get used parameters for sps/pps */
-#define GET_MTK_VDEC_FLAG(cond, flag) \
-	{ dst_param->cond = ((src_param->flags & (flag)) ? (1) : (0)); }
-#define GET_MTK_VDEC_PARAM(param) \
-	{ dst_param->param = src_param->param; }
-/* motion vector size (bytes) for every macro block */
-#define HW_MB_STORE_SZ				64
-
-#define H264_MAX_FB_NUM				17
-#define H264_MAX_MV_NUM				32
-#define HDR_PARSING_BUF_SZ			1024
-
-/**
- * struct mtk_h264_dpb_info  - h264 dpb information
- * @y_dma_addr: Y bitstream physical address
- * @c_dma_addr: CbCr bitstream physical address
- * @reference_flag: reference picture flag (short/long term reference picture)
- * @field: field picture flag
- */
-struct mtk_h264_dpb_info {
-	dma_addr_t y_dma_addr;
-	dma_addr_t c_dma_addr;
-	int reference_flag;
-	int field;
-};
-
-/*
- * struct mtk_h264_sps_param  - parameters for sps
- */
-struct mtk_h264_sps_param {
-	unsigned char chroma_format_idc;
-	unsigned char bit_depth_luma_minus8;
-	unsigned char bit_depth_chroma_minus8;
-	unsigned char log2_max_frame_num_minus4;
-	unsigned char pic_order_cnt_type;
-	unsigned char log2_max_pic_order_cnt_lsb_minus4;
-	unsigned char max_num_ref_frames;
-	unsigned char separate_colour_plane_flag;
-	unsigned short pic_width_in_mbs_minus1;
-	unsigned short pic_height_in_map_units_minus1;
-	unsigned int max_frame_nums;
-	unsigned char qpprime_y_zero_transform_bypass_flag;
-	unsigned char delta_pic_order_always_zero_flag;
-	unsigned char frame_mbs_only_flag;
-	unsigned char mb_adaptive_frame_field_flag;
-	unsigned char direct_8x8_inference_flag;
-	unsigned char reserved[3];
-};
-
-/*
- * struct mtk_h264_pps_param  - parameters for pps
- */
-struct mtk_h264_pps_param {
-	unsigned char num_ref_idx_l0_default_active_minus1;
-	unsigned char num_ref_idx_l1_default_active_minus1;
-	unsigned char weighted_bipred_idc;
-	char pic_init_qp_minus26;
-	char chroma_qp_index_offset;
-	char second_chroma_qp_index_offset;
-	unsigned char entropy_coding_mode_flag;
-	unsigned char pic_order_present_flag;
-	unsigned char deblocking_filter_control_present_flag;
-	unsigned char constrained_intra_pred_flag;
-	unsigned char weighted_pred_flag;
-	unsigned char redundant_pic_cnt_present_flag;
-	unsigned char transform_8x8_mode_flag;
-	unsigned char scaling_matrix_present_flag;
-	unsigned char reserved[2];
-};
-
-struct slice_api_h264_scaling_matrix {
-	unsigned char scaling_list_4x4[6][16];
-	unsigned char scaling_list_8x8[6][64];
-};
-
-struct slice_h264_dpb_entry {
-	unsigned long long reference_ts;
-	unsigned short frame_num;
-	unsigned short pic_num;
-	/* Note that field is indicated by v4l2_buffer.field */
-	int top_field_order_cnt;
-	int bottom_field_order_cnt;
-	unsigned int flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */
-};
-
-/*
- * struct slice_api_h264_decode_param - parameters for decode.
- */
-struct slice_api_h264_decode_param {
-	struct slice_h264_dpb_entry dpb[16];
-	unsigned short num_slices;
-	unsigned short nal_ref_idc;
-	unsigned char ref_pic_list_p0[32];
-	unsigned char ref_pic_list_b0[32];
-	unsigned char ref_pic_list_b1[32];
-	int top_field_order_cnt;
-	int bottom_field_order_cnt;
-	unsigned int flags; /* V4L2_H264_DECODE_PARAM_FLAG_* */
-};
+#include "vdec_h264_req_common.h"
 
 /*
  * struct mtk_h264_dec_slice_param  - parameters for decode current frame
@@ -127,22 +25,6 @@ struct mtk_h264_dec_slice_param {
 	struct mtk_h264_dpb_info h264_dpb_info[16];
 };
 
-/**
- * struct h264_fb - h264 decode frame buffer information
- * @vdec_fb_va  : virtual address of struct vdec_fb
- * @y_fb_dma    : dma address of Y frame buffer (luma)
- * @c_fb_dma    : dma address of C frame buffer (chroma)
- * @poc         : picture order count of frame buffer
- * @reserved    : for 8 bytes alignment
- */
-struct h264_fb {
-	u64 vdec_fb_va;
-	u64 y_fb_dma;
-	u64 c_fb_dma;
-	s32 poc;
-	u32 reserved;
-};
-
 /**
  * struct vdec_h264_dec_info - decode information
  * @dpb_sz		: decoding picture buffer size
@@ -212,233 +94,6 @@ struct vdec_h264_slice_inst {
 	struct v4l2_h264_dpb_entry dpb[16];
 };
 
-static void *get_ctrl_ptr(struct mtk_vcodec_ctx *ctx, int id)
-{
-	struct v4l2_ctrl *ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, id);
-
-	return ctrl->p_cur.p;
-}
-
-static void get_h264_dpb_list(struct vdec_h264_slice_inst *inst,
-			      struct mtk_h264_dec_slice_param *slice_param)
-{
-	struct vb2_queue *vq;
-	struct vb2_buffer *vb;
-	struct vb2_v4l2_buffer *vb2_v4l2;
-	u64 index;
-
-	vq = v4l2_m2m_get_vq(inst->ctx->m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
-
-	for (index = 0; index < ARRAY_SIZE(slice_param->decode_params.dpb); index++) {
-		const struct slice_h264_dpb_entry *dpb;
-		int vb2_index;
-
-		dpb = &slice_param->decode_params.dpb[index];
-		if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) {
-			slice_param->h264_dpb_info[index].reference_flag = 0;
-			continue;
-		}
-
-		vb2_index = vb2_find_timestamp(vq, dpb->reference_ts, 0);
-		if (vb2_index < 0) {
-			mtk_vcodec_err(inst, "Reference invalid: dpb_index(%lld) reference_ts(%lld)",
-				       index, dpb->reference_ts);
-			continue;
-		}
-		/* 1 for short term reference, 2 for long term reference */
-		if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM))
-			slice_param->h264_dpb_info[index].reference_flag = 1;
-		else
-			slice_param->h264_dpb_info[index].reference_flag = 2;
-
-		vb = vq->bufs[vb2_index];
-		vb2_v4l2 = container_of(vb, struct vb2_v4l2_buffer, vb2_buf);
-		slice_param->h264_dpb_info[index].field = vb2_v4l2->field;
-
-		slice_param->h264_dpb_info[index].y_dma_addr =
-			vb2_dma_contig_plane_dma_addr(vb, 0);
-		if (inst->ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 2) {
-			slice_param->h264_dpb_info[index].c_dma_addr =
-				vb2_dma_contig_plane_dma_addr(vb, 1);
-		}
-	}
-}
-
-static void get_h264_sps_parameters(struct mtk_h264_sps_param *dst_param,
-				    const struct v4l2_ctrl_h264_sps *src_param)
-{
-	GET_MTK_VDEC_PARAM(chroma_format_idc);
-	GET_MTK_VDEC_PARAM(bit_depth_luma_minus8);
-	GET_MTK_VDEC_PARAM(bit_depth_chroma_minus8);
-	GET_MTK_VDEC_PARAM(log2_max_frame_num_minus4);
-	GET_MTK_VDEC_PARAM(pic_order_cnt_type);
-	GET_MTK_VDEC_PARAM(log2_max_pic_order_cnt_lsb_minus4);
-	GET_MTK_VDEC_PARAM(max_num_ref_frames);
-	GET_MTK_VDEC_PARAM(pic_width_in_mbs_minus1);
-	GET_MTK_VDEC_PARAM(pic_height_in_map_units_minus1);
-
-	GET_MTK_VDEC_FLAG(separate_colour_plane_flag,
-			  V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE);
-	GET_MTK_VDEC_FLAG(qpprime_y_zero_transform_bypass_flag,
-			  V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS);
-	GET_MTK_VDEC_FLAG(delta_pic_order_always_zero_flag,
-			  V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO);
-	GET_MTK_VDEC_FLAG(frame_mbs_only_flag,
-			  V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY);
-	GET_MTK_VDEC_FLAG(mb_adaptive_frame_field_flag,
-			  V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD);
-	GET_MTK_VDEC_FLAG(direct_8x8_inference_flag,
-			  V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE);
-}
-
-static void get_h264_pps_parameters(struct mtk_h264_pps_param *dst_param,
-				    const struct v4l2_ctrl_h264_pps *src_param)
-{
-	GET_MTK_VDEC_PARAM(num_ref_idx_l0_default_active_minus1);
-	GET_MTK_VDEC_PARAM(num_ref_idx_l1_default_active_minus1);
-	GET_MTK_VDEC_PARAM(weighted_bipred_idc);
-	GET_MTK_VDEC_PARAM(pic_init_qp_minus26);
-	GET_MTK_VDEC_PARAM(chroma_qp_index_offset);
-	GET_MTK_VDEC_PARAM(second_chroma_qp_index_offset);
-
-	GET_MTK_VDEC_FLAG(entropy_coding_mode_flag,
-			  V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE);
-	GET_MTK_VDEC_FLAG(pic_order_present_flag,
-			  V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT);
-	GET_MTK_VDEC_FLAG(weighted_pred_flag,
-			  V4L2_H264_PPS_FLAG_WEIGHTED_PRED);
-	GET_MTK_VDEC_FLAG(deblocking_filter_control_present_flag,
-			  V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT);
-	GET_MTK_VDEC_FLAG(constrained_intra_pred_flag,
-			  V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED);
-	GET_MTK_VDEC_FLAG(redundant_pic_cnt_present_flag,
-			  V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT);
-	GET_MTK_VDEC_FLAG(transform_8x8_mode_flag,
-			  V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE);
-	GET_MTK_VDEC_FLAG(scaling_matrix_present_flag,
-			  V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT);
-}
-
-static void
-get_h264_scaling_matrix(struct slice_api_h264_scaling_matrix *dst_matrix,
-			const struct v4l2_ctrl_h264_scaling_matrix *src_matrix)
-{
-	memcpy(dst_matrix->scaling_list_4x4, src_matrix->scaling_list_4x4,
-	       sizeof(dst_matrix->scaling_list_4x4));
-
-	memcpy(dst_matrix->scaling_list_8x8, src_matrix->scaling_list_8x8,
-	       sizeof(dst_matrix->scaling_list_8x8));
-}
-
-static void
-get_h264_decode_parameters(struct slice_api_h264_decode_param *dst_params,
-			   const struct v4l2_ctrl_h264_decode_params *src_params,
-			   const struct v4l2_h264_dpb_entry dpb[V4L2_H264_NUM_DPB_ENTRIES])
-{
-	int i;
-
-	for (i = 0; i < ARRAY_SIZE(dst_params->dpb); i++) {
-		struct slice_h264_dpb_entry *dst_entry = &dst_params->dpb[i];
-		const struct v4l2_h264_dpb_entry *src_entry = &dpb[i];
-
-		dst_entry->reference_ts = src_entry->reference_ts;
-		dst_entry->frame_num = src_entry->frame_num;
-		dst_entry->pic_num = src_entry->pic_num;
-		dst_entry->top_field_order_cnt = src_entry->top_field_order_cnt;
-		dst_entry->bottom_field_order_cnt =
-			src_entry->bottom_field_order_cnt;
-		dst_entry->flags = src_entry->flags;
-	}
-
-	/*
-	 * num_slices is a leftover from the old H.264 support and is ignored
-	 * by the firmware.
-	 */
-	dst_params->num_slices = 0;
-	dst_params->nal_ref_idc = src_params->nal_ref_idc;
-	dst_params->top_field_order_cnt = src_params->top_field_order_cnt;
-	dst_params->bottom_field_order_cnt = src_params->bottom_field_order_cnt;
-	dst_params->flags = src_params->flags;
-}
-
-static bool dpb_entry_match(const struct v4l2_h264_dpb_entry *a,
-			    const struct v4l2_h264_dpb_entry *b)
-{
-	return a->top_field_order_cnt == b->top_field_order_cnt &&
-	       a->bottom_field_order_cnt == b->bottom_field_order_cnt;
-}
-
-/*
- * Move DPB entries of dec_param that refer to a frame already existing in dpb
- * into the already existing slot in dpb, and move other entries into new slots.
- *
- * This function is an adaptation of the similarly-named function in
- * hantro_h264.c.
- */
-static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
-		       struct v4l2_h264_dpb_entry *dpb)
-{
-	DECLARE_BITMAP(new, ARRAY_SIZE(dec_param->dpb)) = { 0, };
-	DECLARE_BITMAP(in_use, ARRAY_SIZE(dec_param->dpb)) = { 0, };
-	DECLARE_BITMAP(used, ARRAY_SIZE(dec_param->dpb)) = { 0, };
-	unsigned int i, j;
-
-	/* Disable all entries by default, and mark the ones in use. */
-	for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) {
-		if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
-			set_bit(i, in_use);
-		dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE;
-	}
-
-	/* Try to match new DPB entries with existing ones by their POCs. */
-	for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) {
-		const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i];
-
-		if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
-			continue;
-
-		/*
-		 * To cut off some comparisons, iterate only on target DPB
-		 * entries were already used.
-		 */
-		for_each_set_bit(j, in_use, ARRAY_SIZE(dec_param->dpb)) {
-			struct v4l2_h264_dpb_entry *cdpb;
-
-			cdpb = &dpb[j];
-			if (!dpb_entry_match(cdpb, ndpb))
-				continue;
-
-			*cdpb = *ndpb;
-			set_bit(j, used);
-			/* Don't reiterate on this one. */
-			clear_bit(j, in_use);
-			break;
-		}
-
-		if (j == ARRAY_SIZE(dec_param->dpb))
-			set_bit(i, new);
-	}
-
-	/* For entries that could not be matched, use remaining free slots. */
-	for_each_set_bit(i, new, ARRAY_SIZE(dec_param->dpb)) {
-		const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i];
-		struct v4l2_h264_dpb_entry *cdpb;
-
-		/*
-		 * Both arrays are of the same sizes, so there is no way
-		 * we can end up with no space in target array, unless
-		 * something is buggy.
-		 */
-		j = find_first_zero_bit(used, ARRAY_SIZE(dec_param->dpb));
-		if (WARN_ON(j >= ARRAY_SIZE(dec_param->dpb)))
-			return;
-
-		cdpb = &dpb[j];
-		*cdpb = *ndpb;
-		set_bit(j, used);
-	}
-}
-
 /*
  * The firmware expects unused reflist entries to have the value 0x20.
  */
@@ -450,27 +105,32 @@ static void fixup_ref_list(u8 *ref_list, size_t num_valid)
 static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
 {
 	const struct v4l2_ctrl_h264_decode_params *dec_params =
-		get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_DECODE_PARAMS);
+		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
+		V4L2_CID_STATELESS_H264_DECODE_PARAMS);
 	const struct v4l2_ctrl_h264_sps *sps =
-		get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_SPS);
+		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
+		V4L2_CID_STATELESS_H264_SPS);
 	const struct v4l2_ctrl_h264_pps *pps =
-		get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_PPS);
+		mtk_vdec_h264_get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_PPS);
 	const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix =
-		get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_SCALING_MATRIX);
+		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
+		V4L2_CID_STATELESS_H264_SCALING_MATRIX);
 	struct mtk_h264_dec_slice_param *slice_param = &inst->h264_slice_param;
 	struct v4l2_h264_reflist_builder reflist_builder;
 	u8 *p0_reflist = slice_param->decode_params.ref_pic_list_p0;
 	u8 *b0_reflist = slice_param->decode_params.ref_pic_list_b0;
 	u8 *b1_reflist = slice_param->decode_params.ref_pic_list_b1;
 
-	update_dpb(dec_params, inst->dpb);
+	mtk_vdec_h264_update_dpb(dec_params, inst->dpb);
 
-	get_h264_sps_parameters(&slice_param->sps, sps);
-	get_h264_pps_parameters(&slice_param->pps, pps);
-	get_h264_scaling_matrix(&slice_param->scaling_matrix, scaling_matrix);
-	get_h264_decode_parameters(&slice_param->decode_params, dec_params,
-				   inst->dpb);
-	get_h264_dpb_list(inst, slice_param);
+	mtk_vdec_h264_copy_sps_params(&slice_param->sps, sps);
+	mtk_vdec_h264_copy_pps_params(&slice_param->pps, pps);
+	mtk_vdec_h264_copy_scaling_matrix(&slice_param->scaling_matrix,
+		scaling_matrix);
+	mtk_vdec_h264_copy_decode_params(&slice_param->decode_params,
+		dec_params, inst->dpb);
+	mtk_vdec_h264_fill_dpb_info(inst->ctx, &slice_param->decode_params,
+		slice_param->h264_dpb_info);
 
 	/* Build the reference lists */
 	v4l2_h264_init_reflist_builder(&reflist_builder, dec_params, sps,
@@ -486,13 +146,6 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
 	       sizeof(inst->vsi_ctx.h264_slice_params));
 }
 
-static unsigned int get_mv_buf_size(unsigned int width, unsigned int height)
-{
-	int unit_size = (width / MB_UNIT_LEN) * (height / MB_UNIT_LEN) + 8;
-
-	return HW_MB_STORE_SZ * unit_size;
-}
-
 static int allocate_predication_buf(struct vdec_h264_slice_inst *inst)
 {
 	int err;
@@ -525,7 +178,7 @@ static int alloc_mv_buf(struct vdec_h264_slice_inst *inst,
 	int i;
 	int err;
 	struct mtk_vcodec_mem *mem = NULL;
-	unsigned int buf_sz = get_mv_buf_size(pic->buf_w, pic->buf_h);
+	unsigned int buf_sz = mtk_vdec_h264_get_mv_buf_size(pic->buf_w, pic->buf_h);
 
 	mtk_v4l2_debug(3, "size = 0x%lx", buf_sz);
 	for (i = 0; i < H264_MAX_MV_NUM; i++) {
@@ -674,7 +327,8 @@ static int vdec_h264_slice_decode(void *h_vdec, struct mtk_vcodec_mem *bs,
 {
 	struct vdec_h264_slice_inst *inst = h_vdec;
 	const struct v4l2_ctrl_h264_decode_params *dec_params =
-		get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_DECODE_PARAMS);
+		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
+		V4L2_CID_STATELESS_H264_DECODE_PARAMS);
 	struct vdec_vpu_inst *vpu = &inst->vpu;
 	struct mtk_video_dec_buf *src_buf_info;
 	struct mtk_video_dec_buf *dst_buf_info;
-- 
2.25.1

[PATCH v1, 12/12] media: mtk-vcodec: Add h264 slice api driver for mt8192

[Yunfei Dong]

From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>

Adds h264 lat and core driver for mt8192.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile    |   1 +
 .../mtk-vcodec/vdec/vdec_h264_req_lat_if.c    | 620 ++++++++++++++++++
 .../media/platform/mtk-vcodec/vdec_drv_if.c   |   8 +-
 .../media/platform/mtk-vcodec/vdec_drv_if.h   |   1 +
 include/linux/remoteproc/mtk_scp.h            |   2 +
 5 files changed, 631 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_lat_if.c

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 3f41d748eee5..1777d7606f0d 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -10,6 +10,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
 		vdec/vdec_vp9_if.o \
 		vdec/vdec_h264_req_if.o \
 		vdec/vdec_h264_req_common.o \
+		vdec/vdec_h264_req_lat_if.o \
 		mtk_vcodec_dec_drv.o \
 		vdec_drv_if.o \
 		vdec_vpu_if.o \
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_lat_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_lat_if.c
new file mode 100644
index 000000000000..403d7df00e1d
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_lat_if.c
@@ -0,0 +1,620 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ * Author: Yunfei Dong <yunfei.dong@mediatek.com>
+ */
+
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <media/v4l2-h264.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "../mtk_vcodec_util.h"
+#include "../mtk_vcodec_dec.h"
+#include "../mtk_vcodec_intr.h"
+#include "../vdec_drv_base.h"
+#include "../vdec_drv_if.h"
+#include "../vdec_vpu_if.h"
+#include "vdec_h264_req_common.h"
+
+/**
+ * enum vdec_h264_core_dec_err_type  - core decode error type
+ */
+enum vdec_h264_core_dec_err_type {
+	TRANS_BUFFER_FULL = 1,
+	SLICE_HEADER_FULL,
+};
+
+/**
+ * struct vdec_h264_slice_lat_dec_param  - parameters for decode current frame
+ * @sps : h264 sps syntax parameters
+ * @pps : h264 pps syntax parameters
+ * @slice_header: h264 slice header syntax parameters
+ * @scaling_matrix : h264 scaling list parameters
+ * @decode_params : decoder parameters of each frame used for hardware decode
+ * @h264_dpb_info : dpb reference list
+ */
+struct vdec_h264_slice_lat_dec_param {
+	struct mtk_h264_sps_param sps;
+	struct mtk_h264_pps_param pps;
+	struct mtk_h264_slice_hd_param slice_header;
+	struct slice_api_h264_scaling_matrix scaling_matrix;
+	struct slice_api_h264_decode_param decode_params;
+	struct mtk_h264_dpb_info h264_dpb_info[V4L2_H264_NUM_DPB_ENTRIES];
+};
+
+/**
+ * struct vdec_h264_slice_info - decode information
+ * @nal_info    : nal info of current picture
+ * @timeout     : Decode timeout: 1 timeout, 0 no timeount
+ * @bs_buf_size : bitstream size
+ * @bs_buf_addr : bitstream buffer dma address
+ * @y_fb_dma    : Y frame buffer dma address
+ * @c_fb_dma    : C frame buffer dma address
+ * @vdec_fb_va  : VDEC frame buffer struct virtual address
+ * @crc         : Used to check whether hardware's status is right
+ */
+struct vdec_h264_slice_info {
+	uint16_t nal_info;
+	uint16_t timeout;
+	uint32_t bs_buf_size;
+	uint64_t bs_buf_addr;
+	uint64_t y_fb_dma;
+	uint64_t c_fb_dma;
+	uint64_t vdec_fb_va;
+	uint32_t crc[8];
+};
+
+/**
+ * struct vdec_h264_slice_vsi - shared memory for decode information exchange
+ *        between VPU and Host. The memory is allocated by VPU then mapping to
+ *        Host in vdec_h264_slice_init() and freed in vdec_h264_slice_deinit()
+ *        by VPU. AP-W/R : AP is writer/reader on this item. VPU-W/R: VPU is
+ *        write/reader on this item.
+ * @wdma_err_addr       : wdma error dma address
+ * @wdma_start_addr     : wdma start dma address
+ * @wdma_end_addr       : wdma end dma address
+ * @slice_bc_start_addr : slice bc start dma address
+ * @slice_bc_end_addr   : slice bc end dma address
+ * @row_info_start_addr : row info start dma address
+ * @row_info_end_addr   : row info end dma address
+ * @trans_start         : trans start dma address
+ * @trans_end           : trans end dma address
+ * @wdma_end_addr_offset: wdma end address offset
+ * @mv_buf_dma          : HW working motion vector buffer
+ *                        dma address (AP-W, VPU-R)
+ * @dec                 : decode information (AP-R, VPU-W)
+ * @h264_slice_params   : decode parameters for hw used
+ */
+struct vdec_h264_slice_vsi {
+	/* LAT dec addr */
+	uint64_t wdma_err_addr;
+	uint64_t wdma_start_addr;
+	uint64_t wdma_end_addr;
+	uint64_t slice_bc_start_addr;
+	uint64_t slice_bc_end_addr;
+	uint64_t row_info_start_addr;
+	uint64_t row_info_end_addr;
+	uint64_t trans_start;
+	uint64_t trans_end;
+	uint64_t wdma_end_addr_offset;
+
+	uint64_t mv_buf_dma[H264_MAX_MV_NUM];
+	struct vdec_h264_slice_info dec;
+	struct vdec_h264_slice_lat_dec_param h264_slice_params;
+};
+
+/**
+ * struct vdec_h264_slice_share_info - shared information used to exchange
+ *                                     message between lat and core
+ * @sps	              : sequence header information from user space
+ * @dec_params        : decoder params from user space
+ * @h264_slice_params : decoder params used for hardware
+ * @trans_start       : trans start dma address
+ * @trans_end         : trans end dma address
+ * @nal_info          : nal info of current picture
+ */
+struct vdec_h264_slice_share_info {
+	struct v4l2_ctrl_h264_sps sps;
+	struct v4l2_ctrl_h264_decode_params dec_params;
+	struct vdec_h264_slice_lat_dec_param h264_slice_params;
+	uint64_t trans_start;
+	uint64_t trans_end;
+	uint16_t nal_info;
+};
+
+/**
+ * struct vdec_h264_slice_inst - h264 decoder instance
+ * @num_nalu            : how many nalus be decoded
+ * @ctx                 : point to mtk_vcodec_ctx
+ * @pred_buf            : HW working predication buffer
+ * @mv_buf              : HW working motion vector buffer
+ * @vpu                 : VPU instance
+ * @vsi                 : vsi used for lat
+ * @vsi_core            : vsi used for core
+ * @resolution_changed  : resolution changed
+ * @realloc_mv_buf      : reallocate mv buffer
+ * @cap_num_planes      : number of capture queue plane
+ */
+struct vdec_h264_slice_inst {
+	unsigned int num_nalu;
+	struct mtk_vcodec_ctx *ctx;
+	struct mtk_vcodec_mem pred_buf;
+	struct mtk_vcodec_mem mv_buf[H264_MAX_MV_NUM];
+	struct vdec_vpu_inst vpu;
+	struct vdec_h264_slice_vsi *vsi;
+	struct vdec_h264_slice_vsi *vsi_core;
+
+	unsigned int resolution_changed;
+	unsigned int realloc_mv_buf;
+	unsigned int cap_num_planes;
+
+	struct v4l2_h264_dpb_entry dpb[16];
+};
+
+static void vdec_h264_slice_fill_decode_parameters(
+	struct vdec_h264_slice_inst *inst,
+	struct vdec_h264_slice_share_info *share_info)
+{
+	struct vdec_h264_slice_lat_dec_param *slice_param =
+		&inst->vsi->h264_slice_params;
+	const struct v4l2_ctrl_h264_decode_params *dec_params =
+		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
+			V4L2_CID_STATELESS_H264_DECODE_PARAMS);
+	const struct v4l2_ctrl_h264_scaling_matrix *src_matrix =
+		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
+			V4L2_CID_STATELESS_H264_SCALING_MATRIX);
+	const struct v4l2_ctrl_h264_sps *sps =
+		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
+			V4L2_CID_STATELESS_H264_SPS);
+	const struct v4l2_ctrl_h264_pps *pps =
+		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
+			V4L2_CID_STATELESS_H264_PPS);
+
+	mtk_vdec_h264_copy_sps_params(&slice_param->sps,sps);
+	mtk_vdec_h264_copy_pps_params(&slice_param->pps, pps);
+	mtk_vdec_h264_copy_scaling_matrix(
+		&slice_param->scaling_matrix, src_matrix);
+
+	memcpy(&share_info->sps, sps, sizeof(*sps));
+	memcpy(&share_info->dec_params, dec_params, sizeof(*dec_params));
+}
+
+/*
+ * The firmware expects unused reflist entries to have the value 0x20.
+ */
+static void fixup_ref_list(u8 *ref_list, size_t num_valid)
+{
+	memset(&ref_list[num_valid], 0x20, 32 - num_valid);
+}
+
+static void vdec_h264_slice_fill_decode_reflist(
+	struct vdec_h264_slice_inst *inst,
+	struct vdec_h264_slice_lat_dec_param *slice_param,
+	struct vdec_h264_slice_share_info *share_info)
+{
+	struct v4l2_ctrl_h264_decode_params *dec_params = &share_info->dec_params;
+	struct v4l2_ctrl_h264_sps *sps = &share_info->sps;
+	struct v4l2_h264_reflist_builder reflist_builder;
+	u8 *p0_reflist = slice_param->decode_params.ref_pic_list_p0;
+	u8 *b0_reflist = slice_param->decode_params.ref_pic_list_b0;
+	u8 *b1_reflist = slice_param->decode_params.ref_pic_list_b1;
+
+	mtk_vdec_h264_update_dpb(dec_params, inst->dpb);
+
+	mtk_vdec_h264_copy_decode_params(&slice_param->decode_params, dec_params,
+		inst->dpb);
+	mtk_vdec_h264_fill_dpb_info(inst->ctx, &slice_param->decode_params,
+		slice_param->h264_dpb_info);
+
+	mtk_v4l2_debug(3, "cur poc = %d\n", dec_params->bottom_field_order_cnt);
+	/* Build the reference lists */
+	v4l2_h264_init_reflist_builder(&reflist_builder, dec_params, sps,
+				       inst->dpb);
+	v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
+	v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
+
+	/* Adapt the built lists to the firmware's expectations */
+	fixup_ref_list(p0_reflist, reflist_builder.num_valid);
+	fixup_ref_list(b0_reflist, reflist_builder.num_valid);
+	fixup_ref_list(b1_reflist, reflist_builder.num_valid);
+}
+
+static int vdec_h264_slice_alloc_mv_buf(struct vdec_h264_slice_inst *inst,
+	struct vdec_pic_info *pic)
+{
+	int i;
+	int err;
+	struct mtk_vcodec_mem *mem;
+	unsigned int buf_sz = mtk_vdec_h264_get_mv_buf_size(
+		pic->buf_w, pic->buf_h);
+
+	mtk_v4l2_debug(3, "size = 0x%x", buf_sz);
+	for (i = 0; i < H264_MAX_MV_NUM; i++) {
+		mem = &inst->mv_buf[i];
+		if (mem->va)
+			mtk_vcodec_mem_free(inst->ctx, mem);
+		mem->size = buf_sz;
+		err = mtk_vcodec_mem_alloc(inst->ctx, mem);
+		if (err) {
+			mtk_vcodec_err(inst, "failed to allocate mv buf");
+			return err;
+		}
+	}
+
+	return 0;
+}
+
+static void vdec_h264_slice_free_mv_buf(struct vdec_h264_slice_inst *inst)
+{
+	int i;
+	struct mtk_vcodec_mem *mem;
+
+	for (i = 0; i < H264_MAX_MV_NUM; i++) {
+		mem = &inst->mv_buf[i];
+		if (mem->va)
+			mtk_vcodec_mem_free(inst->ctx, mem);
+	}
+}
+
+static void vdec_h264_slice_get_pic_info(struct vdec_h264_slice_inst *inst)
+{
+	struct mtk_vcodec_ctx *ctx = inst->ctx;
+	unsigned int data[3];
+
+	data[0] = ctx->picinfo.pic_w;
+	data[1] = ctx->picinfo.pic_h;
+	data[2] = ctx->capture_fourcc;
+	vpu_dec_get_param(&inst->vpu, data, 3, GET_PARAM_PIC_INFO);
+
+	ctx->picinfo.buf_w = ALIGN(ctx->picinfo.pic_w, 64);
+	ctx->picinfo.buf_h = ALIGN(ctx->picinfo.pic_h, 64);
+	ctx->picinfo.fb_sz[0] = inst->vpu.fb_sz[0];
+	ctx->picinfo.fb_sz[1] = inst->vpu.fb_sz[1];
+	inst->cap_num_planes =
+		ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes;
+
+	mtk_vcodec_debug(inst, "pic(%d, %d), buf(%d, %d)",
+			 ctx->picinfo.pic_w, ctx->picinfo.pic_h,
+			 ctx->picinfo.buf_w, ctx->picinfo.buf_h);
+	mtk_vcodec_debug(inst, "Y/C(%d, %d)", ctx->picinfo.fb_sz[0],
+		ctx->picinfo.fb_sz[1]);
+
+	if ((ctx->last_decoded_picinfo.pic_w != ctx->picinfo.pic_w) ||
+		(ctx->last_decoded_picinfo.pic_h != ctx->picinfo.pic_h)) {
+		inst->resolution_changed = true;
+		if ((ctx->last_decoded_picinfo.buf_w != ctx->picinfo.buf_w) ||
+			(ctx->last_decoded_picinfo.buf_h != ctx->picinfo.buf_h))
+			inst->realloc_mv_buf = true;
+
+		mtk_v4l2_debug(1, "resChg: (%d %d) : old(%d, %d) -> new(%d, %d)",
+			inst->resolution_changed,
+			inst->realloc_mv_buf,
+			ctx->last_decoded_picinfo.pic_w,
+			ctx->last_decoded_picinfo.pic_h,
+			ctx->picinfo.pic_w, ctx->picinfo.pic_h);
+	}
+}
+
+static void vdec_h264_slice_get_crop_info(struct vdec_h264_slice_inst *inst,
+	struct v4l2_rect *cr)
+{
+	cr->left = 0;
+	cr->top = 0;
+	cr->width = inst->ctx->picinfo.pic_w;
+	cr->height = inst->ctx->picinfo.pic_h;
+
+	mtk_vcodec_debug(inst, "l=%d, t=%d, w=%d, h=%d",
+			 cr->left, cr->top, cr->width, cr->height);
+}
+
+static int vdec_h264_slice_init(struct mtk_vcodec_ctx *ctx)
+{
+	struct vdec_h264_slice_inst *inst;
+	int err, vsi_size;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	inst->ctx = ctx;
+
+	inst->vpu.id = SCP_IPI_VDEC_LAT;
+	inst->vpu.core_id = SCP_IPI_VDEC_CORE;
+	inst->vpu.ctx = ctx;
+	inst->vpu.codec_type = ctx->current_codec;
+	inst->vpu.capture_type = ctx->capture_fourcc;
+
+	err = vpu_dec_init(&inst->vpu);
+	if (err) {
+		mtk_vcodec_err(inst, "vdec_h264 init err=%d", err);
+		goto error_free_inst;
+	}
+
+	vsi_size = round_up(sizeof(struct vdec_h264_slice_vsi), 64);
+	inst->vsi = inst->vpu.vsi;
+	inst->vsi_core =
+		(struct vdec_h264_slice_vsi *)(((char *)inst->vpu.vsi) + vsi_size);
+	inst->resolution_changed = true;
+	inst->realloc_mv_buf = true;
+
+	mtk_vcodec_debug(inst, "lat struct size = %d,%d,%d,%d vsi: %d\n",
+		(int)sizeof(struct mtk_h264_sps_param),
+		(int)sizeof(struct mtk_h264_pps_param),
+		(int)sizeof(struct vdec_h264_slice_lat_dec_param),
+		(int)sizeof(struct mtk_h264_dpb_info),
+		vsi_size);
+	mtk_vcodec_debug(inst, "lat H264 instance >> %p, codec_type = 0x%x",
+		inst, inst->vpu.codec_type);
+
+	ctx->drv_handle = inst;
+	return 0;
+
+error_free_inst:
+	kfree(inst);
+	return err;
+}
+
+static void vdec_h264_slice_deinit(void *h_vdec)
+{
+	struct vdec_h264_slice_inst *inst = h_vdec;
+
+	mtk_vcodec_debug_enter(inst);
+
+	vpu_dec_deinit(&inst->vpu);
+	vdec_h264_slice_free_mv_buf(inst);
+	vdec_msg_queue_deinit(&inst->ctx->msg_queue, inst->ctx);
+
+	kfree(inst);
+}
+
+static int vdec_h264_slice_core_decode(struct vdec_lat_buf *lat_buf)
+{
+	struct vdec_fb *fb;
+	uint64_t vdec_fb_va;
+	uint64_t y_fb_dma, c_fb_dma;
+	int err, timeout, i, dec_err;
+	struct vdec_vpu_inst *vpu;
+	struct mtk_vcodec_ctx *ctx = lat_buf->ctx;
+	struct vdec_h264_slice_inst *inst = ctx->drv_handle;
+	struct vb2_v4l2_buffer *vb2_v4l2;
+	struct vdec_h264_slice_share_info *share_info = lat_buf->private_data;
+	struct mtk_vcodec_mem *mem;
+
+	mtk_vcodec_debug(inst, "[h264-core] vdec_h264 core decode");
+	memcpy(&inst->vsi_core->h264_slice_params, &share_info->h264_slice_params,
+		sizeof(share_info->h264_slice_params));
+	fb = ctx->dev->vdec_pdata->get_cap_buffer(ctx);
+	vpu = &inst->vpu;
+	vdec_fb_va = (unsigned long)fb;
+	y_fb_dma = fb ? (u64)fb->base_y.dma_addr : 0;
+
+	if (ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 1)
+		c_fb_dma =
+			y_fb_dma + inst->ctx->picinfo.buf_w * inst->ctx->picinfo.buf_h;
+	else
+		c_fb_dma = fb ? (u64)fb->base_c.dma_addr : 0;
+
+	mtk_vcodec_debug(inst, "[h264-core] y/c addr = 0x%llx 0x%llx", y_fb_dma,
+		c_fb_dma);
+
+	inst->vsi_core->dec.y_fb_dma = y_fb_dma;
+	inst->vsi_core->dec.c_fb_dma = c_fb_dma;
+	inst->vsi_core->dec.vdec_fb_va = vdec_fb_va;
+	inst->vsi_core->dec.nal_info = share_info->nal_info;
+	inst->vsi_core->wdma_start_addr =
+		lat_buf->ctx->msg_queue.wdma_addr.dma_addr;
+	inst->vsi_core->wdma_end_addr =
+		lat_buf->ctx->msg_queue.wdma_addr.dma_addr +
+		lat_buf->ctx->msg_queue.wdma_addr.size;
+	inst->vsi_core->wdma_err_addr = lat_buf->wdma_err_addr.dma_addr;
+	inst->vsi_core->slice_bc_start_addr = lat_buf->slice_bc_addr.dma_addr;
+	inst->vsi_core->slice_bc_end_addr = lat_buf->slice_bc_addr.dma_addr +
+		lat_buf->slice_bc_addr.size;
+	inst->vsi_core->trans_start = share_info->trans_start;
+	inst->vsi_core->trans_end = share_info->trans_end;
+	for (i = 0; i < H264_MAX_MV_NUM; i++) {
+		mem = &inst->mv_buf[i];
+		inst->vsi_core->mv_buf_dma[i] = mem->dma_addr;
+	}
+
+	vb2_v4l2 = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
+	vb2_v4l2->vb2_buf.timestamp = lat_buf->ts_info.vb2_buf.timestamp;
+	vb2_v4l2->timecode = lat_buf->ts_info.timecode;
+	vb2_v4l2->field = lat_buf->ts_info.field;
+	vb2_v4l2->flags = lat_buf->ts_info.flags;
+	vb2_v4l2->vb2_buf.copied_timestamp =
+		lat_buf->ts_info.vb2_buf.copied_timestamp;
+
+	vdec_h264_slice_fill_decode_reflist(inst,
+		&inst->vsi_core->h264_slice_params, share_info);
+
+	err = vpu_dec_core(vpu);
+	if (err) {
+		dec_err = 1;
+		mtk_vcodec_err(inst, "core decode err=%d", err);
+		goto vdec_dec_end;
+	} else {
+		dec_err = 0;
+	}
+
+	/* wait decoder done interrupt */
+	timeout = mtk_vcodec_wait_for_done_ctx(
+		inst->ctx, MTK_INST_IRQ_RECEIVED,
+		WAIT_INTR_TIMEOUT_MS, MTK_VDEC_CORE);
+	if (timeout)
+		mtk_vcodec_err(inst, "core decode timeout: pic_%d",
+			ctx->decoded_frame_cnt);
+	inst->vsi_core->dec.timeout = !!timeout;
+
+	vpu_dec_core_end(vpu);
+	mtk_vcodec_debug(inst, "pic[%d] crc: 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x",
+		ctx->decoded_frame_cnt,
+		inst->vsi_core->dec.crc[0],	inst->vsi_core->dec.crc[1],
+		inst->vsi_core->dec.crc[2],	inst->vsi_core->dec.crc[3],
+		inst->vsi_core->dec.crc[4],	inst->vsi_core->dec.crc[5],
+		inst->vsi_core->dec.crc[6],	inst->vsi_core->dec.crc[7]);
+vdec_dec_end:
+	vdec_msg_queue_update_ube_rptr(&lat_buf->ctx->msg_queue,
+		inst->vsi_core->trans_end);
+	ctx->dev->vdec_pdata->cap_to_disp(ctx, fb, dec_err);
+	mtk_vcodec_debug(inst, "core decode done err=%d", err);
+	ctx->decoded_frame_cnt++;
+
+	return 0;
+}
+
+static int vdec_h264_slice_decode(void *h_vdec, struct mtk_vcodec_mem *bs,
+	struct vdec_fb *fb, bool *res_chg)
+{
+	struct vdec_h264_slice_inst *inst = h_vdec;
+	struct vdec_vpu_inst *vpu = &inst->vpu;
+	struct mtk_video_dec_buf *src_buf_info;
+	int nal_start_idx, err, timeout = 0, i;
+	unsigned int nal_type, data[2];
+	struct vdec_lat_buf *lat_buf;
+	struct vdec_h264_slice_share_info *share_info;
+	unsigned char *buf;
+	struct mtk_vcodec_mem *mem;
+
+	mtk_vcodec_debug(inst, "+ [%d] ", ++inst->num_nalu);
+
+	if (vdec_msg_queue_init(&inst->ctx->msg_queue, inst->ctx,
+		vdec_h264_slice_core_decode, sizeof(*share_info)))
+	return -ENOMEM;
+
+	/* bs NULL means flush decoder */
+	if (!bs) {
+		vdec_msg_queue_wait_lat_buf_full(&inst->ctx->msg_queue);
+		return vpu_dec_reset(vpu);
+	}
+
+	lat_buf = vdec_msg_queue_dqbuf(&inst->ctx->msg_queue.lat_ctx);
+	if (!lat_buf) {
+		mtk_vcodec_err(inst, "failed to get lat buffer");
+		return -EINVAL;
+	}
+	share_info = lat_buf->private_data;
+	src_buf_info = container_of(bs, struct mtk_video_dec_buf, bs_buffer);
+
+	buf = (unsigned char *)bs->va;
+	nal_start_idx = mtk_vdec_h264_find_start_code(buf, bs->size);
+	if (nal_start_idx < 0) {
+		err = -EINVAL;
+		goto err_free_fb_out;
+	}
+
+	inst->vsi->dec.nal_info = buf[nal_start_idx];
+	nal_type = NAL_TYPE(buf[nal_start_idx]);
+	mtk_vcodec_debug(inst, "\n + NALU[%d] type %d +\n", inst->num_nalu,
+			 nal_type);
+
+	inst->vsi->dec.bs_buf_addr = (uint64_t)bs->dma_addr;
+	inst->vsi->dec.bs_buf_size = bs->size;
+
+	v4l2_m2m_buf_copy_metadata(&src_buf_info->m2m_buf.vb,
+		&lat_buf->ts_info, true);
+
+	vdec_h264_slice_fill_decode_parameters(inst, share_info);
+	*res_chg = inst->resolution_changed;
+	if (inst->resolution_changed) {
+		mtk_vcodec_debug(inst, "- resolution changed -");
+		if (inst->realloc_mv_buf) {
+			err = vdec_h264_slice_alloc_mv_buf(inst, &inst->ctx->picinfo);
+			inst->realloc_mv_buf = false;
+			if (err)
+				goto err_free_fb_out;
+		}
+		inst->resolution_changed = false;
+	}
+	for (i = 0; i < H264_MAX_MV_NUM; i++) {
+		mem = &inst->mv_buf[i];
+		inst->vsi->mv_buf_dma[i] = mem->dma_addr;
+	}
+	inst->vsi->wdma_start_addr = lat_buf->ctx->msg_queue.wdma_addr.dma_addr;
+	inst->vsi->wdma_end_addr = lat_buf->ctx->msg_queue.wdma_addr.dma_addr +
+		lat_buf->ctx->msg_queue.wdma_addr.size;
+	inst->vsi->wdma_err_addr = lat_buf->wdma_err_addr.dma_addr;
+	inst->vsi->slice_bc_start_addr = lat_buf->slice_bc_addr.dma_addr;
+	inst->vsi->slice_bc_end_addr = lat_buf->slice_bc_addr.dma_addr +
+		lat_buf->slice_bc_addr.size;
+
+	inst->vsi->trans_end = inst->ctx->msg_queue.wdma_rptr_addr;
+	inst->vsi->trans_start = inst->ctx->msg_queue.wdma_wptr_addr;
+	mtk_vcodec_debug(inst, "lat:trans(0x%llx 0x%llx)err:0x%llx",
+		inst->vsi->wdma_start_addr,
+		inst->vsi->wdma_end_addr,
+		inst->vsi->wdma_err_addr);
+
+	mtk_vcodec_debug(inst, "slice(0x%llx 0x%llx) rprt((0x%llx 0x%llx))",
+		inst->vsi->slice_bc_start_addr,
+		inst->vsi->slice_bc_end_addr,
+		inst->vsi->trans_start,
+		inst->vsi->trans_end);
+	err = vpu_dec_start(vpu, data, 2);
+	if (err) {
+		mtk_vcodec_debug(inst, "lat decode err: %d", err);
+		goto err_free_fb_out;
+	}
+
+	if (nal_type == NAL_NON_IDR_SLICE || nal_type == NAL_IDR_SLICE) {
+		/* wait decoder done interrupt */
+		timeout = mtk_vcodec_wait_for_done_ctx(
+			inst->ctx, MTK_INST_IRQ_RECEIVED,
+			WAIT_INTR_TIMEOUT_MS, MTK_VDEC_LAT0);
+		inst->vsi->dec.timeout = !!timeout;
+	}
+	err = vpu_dec_end(vpu);
+	if (err == SLICE_HEADER_FULL || timeout || err == TRANS_BUFFER_FULL) {
+		err = -EINVAL;
+		goto err_free_fb_out;
+	}
+
+	share_info->trans_end = inst->ctx->msg_queue.wdma_addr.dma_addr +
+		inst->vsi->wdma_end_addr_offset;
+	share_info->trans_start = inst->ctx->msg_queue.wdma_wptr_addr;
+	share_info->nal_info = inst->vsi->dec.nal_info;
+	vdec_msg_queue_update_ube_wptr(&lat_buf->ctx->msg_queue,
+		share_info->trans_end);
+
+	memcpy(&share_info->h264_slice_params, &inst->vsi->h264_slice_params,
+		sizeof(share_info->h264_slice_params));
+	vdec_msg_queue_qbuf(&inst->ctx->dev->msg_queue_core_ctx, lat_buf);
+	mtk_vcodec_debug(inst, "- NALU[%d] type=%d -\n", inst->num_nalu,
+			 nal_type);
+	return 0;
+
+err_free_fb_out:
+	mtk_vcodec_err(inst, "- NALU[%d] err=%d -\n", inst->num_nalu, err);
+	return err;
+}
+
+static int vdec_h264_slice_get_param(void *h_vdec,
+	enum vdec_get_param_type type, void *out)
+{
+	struct vdec_h264_slice_inst *inst = h_vdec;
+
+	switch (type) {
+	case GET_PARAM_PIC_INFO:
+		vdec_h264_slice_get_pic_info(inst);
+		break;
+	case GET_PARAM_DPB_SIZE:
+		*(unsigned int *)out = 6;
+		break;
+	case GET_PARAM_CROP_INFO:
+		vdec_h264_slice_get_crop_info(inst, out);
+		break;
+	default:
+		mtk_vcodec_err(inst, "invalid get parameter type=%d", type);
+		return -EINVAL;
+	}
+	return 0;
+}
+
+const struct vdec_common_if vdec_h264_slice_lat_if = {
+	.init		= vdec_h264_slice_init,
+	.decode		= vdec_h264_slice_decode,
+	.get_param	= vdec_h264_slice_get_param,
+	.deinit		= vdec_h264_slice_deinit,
+};
diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
index c93dd0ea3537..c17a7815e1bb 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
@@ -20,7 +20,13 @@ int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 
 	switch (fourcc) {
 	case V4L2_PIX_FMT_H264_SLICE:
-		ctx->dec_if = &vdec_h264_slice_if;
+		if (ctx->dev->vdec_pdata->hw_arch == MTK_VDEC_PURE_SINGLE_CORE) {
+			ctx->dec_if = &vdec_h264_slice_if;
+			ctx->hw_id = MTK_VDEC_CORE;
+		} else {
+			ctx->dec_if = &vdec_h264_slice_lat_if;
+			ctx->hw_id = MTK_VDEC_LAT0;
+		}
 		break;
 	case V4L2_PIX_FMT_H264:
 		ctx->dec_if = &vdec_h264_if;
diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.h b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
index d467e8af4a84..6ce848e74167 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
@@ -56,6 +56,7 @@ struct vdec_fb_node {
 
 extern const struct vdec_common_if vdec_h264_if;
 extern const struct vdec_common_if vdec_h264_slice_if;
+extern const struct vdec_common_if vdec_h264_slice_lat_if;
 extern const struct vdec_common_if vdec_vp8_if;
 extern const struct vdec_common_if vdec_vp9_if;
 
diff --git a/include/linux/remoteproc/mtk_scp.h b/include/linux/remoteproc/mtk_scp.h
index b47416f7aeb8..a32ffd736e7b 100644
--- a/include/linux/remoteproc/mtk_scp.h
+++ b/include/linux/remoteproc/mtk_scp.h
@@ -41,6 +41,8 @@ enum scp_ipi_id {
 	SCP_IPI_ISP_FRAME,
 	SCP_IPI_FD_CMD,
 	SCP_IPI_CROS_HOST_CMD,
+	SCP_IPI_VDEC_LAT,
+	SPC_IPI_VDEC_CORE,
 	SCP_IPI_NS_SERVICE = 0xFF,
 	SCP_IPI_MAX = 0x100,
 };
-- 
2.25.1

Re: [PATCH v1, 12/12] media: mtk-vcodec: Add h264 slice api driver for mt8192

[Nicolas Dufresne]

Hi Yunfei,

Le mercredi 15 décembre 2021 à 14:59 +0800, Yunfei Dong a écrit :
> From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>
> 
> Adds h264 lat and core driver for mt8192.

This is purely a nit, but I have first notice the usage of "slice" in the
namespace and the title, which lead me to think this new platform was
V4L2_STATELESS_H264_DECODE_MODE_SLICE_BASED. I think some structure which are
clearly frame_based should probably be renamed (its the namespace that is
confusing) to reduce the confusion.

p.s. Note that adding slice_based mode would be amazing for streaming with ultra
low latency (think remote video games)

regards,
Nicolas

> 
> Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> ---
>  drivers/media/platform/mtk-vcodec/Makefile    |   1 +
>  .../mtk-vcodec/vdec/vdec_h264_req_lat_if.c    | 620 ++++++++++++++++++
>  .../media/platform/mtk-vcodec/vdec_drv_if.c   |   8 +-
>  .../media/platform/mtk-vcodec/vdec_drv_if.h   |   1 +
>  include/linux/remoteproc/mtk_scp.h            |   2 +
>  5 files changed, 631 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_lat_if.c
> 
> diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> index 3f41d748eee5..1777d7606f0d 100644
> --- a/drivers/media/platform/mtk-vcodec/Makefile
> +++ b/drivers/media/platform/mtk-vcodec/Makefile
> @@ -10,6 +10,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
>  		vdec/vdec_vp9_if.o \
>  		vdec/vdec_h264_req_if.o \
>  		vdec/vdec_h264_req_common.o \
> +		vdec/vdec_h264_req_lat_if.o \
>  		mtk_vcodec_dec_drv.o \
>  		vdec_drv_if.o \
>  		vdec_vpu_if.o \
> diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_lat_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_lat_if.c
> new file mode 100644
> index 000000000000..403d7df00e1d
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_lat_if.c
> @@ -0,0 +1,620 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2021 MediaTek Inc.
> + * Author: Yunfei Dong <yunfei.dong@mediatek.com>
> + */
> +
> +#include <linux/module.h>
> +#include <linux/slab.h>
> +#include <media/v4l2-h264.h>
> +#include <media/v4l2-mem2mem.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "../mtk_vcodec_util.h"
> +#include "../mtk_vcodec_dec.h"
> +#include "../mtk_vcodec_intr.h"
> +#include "../vdec_drv_base.h"
> +#include "../vdec_drv_if.h"
> +#include "../vdec_vpu_if.h"
> +#include "vdec_h264_req_common.h"
> +
> +/**
> + * enum vdec_h264_core_dec_err_type  - core decode error type
> + */
> +enum vdec_h264_core_dec_err_type {
> +	TRANS_BUFFER_FULL = 1,
> +	SLICE_HEADER_FULL,
> +};
> +
> +/**
> + * struct vdec_h264_slice_lat_dec_param  - parameters for decode current frame
> + * @sps : h264 sps syntax parameters
> + * @pps : h264 pps syntax parameters
> + * @slice_header: h264 slice header syntax parameters
> + * @scaling_matrix : h264 scaling list parameters
> + * @decode_params : decoder parameters of each frame used for hardware decode
> + * @h264_dpb_info : dpb reference list
> + */
> +struct vdec_h264_slice_lat_dec_param {
> +	struct mtk_h264_sps_param sps;
> +	struct mtk_h264_pps_param pps;
> +	struct mtk_h264_slice_hd_param slice_header;
> +	struct slice_api_h264_scaling_matrix scaling_matrix;
> +	struct slice_api_h264_decode_param decode_params;
> +	struct mtk_h264_dpb_info h264_dpb_info[V4L2_H264_NUM_DPB_ENTRIES];
> +};
> +
> +/**
> + * struct vdec_h264_slice_info - decode information
> + * @nal_info    : nal info of current picture
> + * @timeout     : Decode timeout: 1 timeout, 0 no timeount
> + * @bs_buf_size : bitstream size
> + * @bs_buf_addr : bitstream buffer dma address
> + * @y_fb_dma    : Y frame buffer dma address
> + * @c_fb_dma    : C frame buffer dma address
> + * @vdec_fb_va  : VDEC frame buffer struct virtual address
> + * @crc         : Used to check whether hardware's status is right
> + */
> +struct vdec_h264_slice_info {
> +	uint16_t nal_info;
> +	uint16_t timeout;
> +	uint32_t bs_buf_size;
> +	uint64_t bs_buf_addr;
> +	uint64_t y_fb_dma;
> +	uint64_t c_fb_dma;
> +	uint64_t vdec_fb_va;
> +	uint32_t crc[8];
> +};
> +
> +/**
> + * struct vdec_h264_slice_vsi - shared memory for decode information exchange
> + *        between VPU and Host. The memory is allocated by VPU then mapping to
> + *        Host in vdec_h264_slice_init() and freed in vdec_h264_slice_deinit()
> + *        by VPU. AP-W/R : AP is writer/reader on this item. VPU-W/R: VPU is
> + *        write/reader on this item.
> + * @wdma_err_addr       : wdma error dma address
> + * @wdma_start_addr     : wdma start dma address
> + * @wdma_end_addr       : wdma end dma address
> + * @slice_bc_start_addr : slice bc start dma address
> + * @slice_bc_end_addr   : slice bc end dma address
> + * @row_info_start_addr : row info start dma address
> + * @row_info_end_addr   : row info end dma address
> + * @trans_start         : trans start dma address
> + * @trans_end           : trans end dma address
> + * @wdma_end_addr_offset: wdma end address offset
> + * @mv_buf_dma          : HW working motion vector buffer
> + *                        dma address (AP-W, VPU-R)
> + * @dec                 : decode information (AP-R, VPU-W)
> + * @h264_slice_params   : decode parameters for hw used
> + */
> +struct vdec_h264_slice_vsi {
> +	/* LAT dec addr */
> +	uint64_t wdma_err_addr;
> +	uint64_t wdma_start_addr;
> +	uint64_t wdma_end_addr;
> +	uint64_t slice_bc_start_addr;
> +	uint64_t slice_bc_end_addr;
> +	uint64_t row_info_start_addr;
> +	uint64_t row_info_end_addr;
> +	uint64_t trans_start;
> +	uint64_t trans_end;
> +	uint64_t wdma_end_addr_offset;
> +
> +	uint64_t mv_buf_dma[H264_MAX_MV_NUM];
> +	struct vdec_h264_slice_info dec;
> +	struct vdec_h264_slice_lat_dec_param h264_slice_params;
> +};
> +
> +/**
> + * struct vdec_h264_slice_share_info - shared information used to exchange
> + *                                     message between lat and core
> + * @sps	              : sequence header information from user space
> + * @dec_params        : decoder params from user space
> + * @h264_slice_params : decoder params used for hardware
> + * @trans_start       : trans start dma address
> + * @trans_end         : trans end dma address
> + * @nal_info          : nal info of current picture
> + */
> +struct vdec_h264_slice_share_info {
> +	struct v4l2_ctrl_h264_sps sps;
> +	struct v4l2_ctrl_h264_decode_params dec_params;
> +	struct vdec_h264_slice_lat_dec_param h264_slice_params;
> +	uint64_t trans_start;
> +	uint64_t trans_end;
> +	uint16_t nal_info;
> +};
> +
> +/**
> + * struct vdec_h264_slice_inst - h264 decoder instance
> + * @num_nalu            : how many nalus be decoded
> + * @ctx                 : point to mtk_vcodec_ctx
> + * @pred_buf            : HW working predication buffer
> + * @mv_buf              : HW working motion vector buffer
> + * @vpu                 : VPU instance
> + * @vsi                 : vsi used for lat
> + * @vsi_core            : vsi used for core
> + * @resolution_changed  : resolution changed
> + * @realloc_mv_buf      : reallocate mv buffer
> + * @cap_num_planes      : number of capture queue plane
> + */
> +struct vdec_h264_slice_inst {
> +	unsigned int num_nalu;
> +	struct mtk_vcodec_ctx *ctx;
> +	struct mtk_vcodec_mem pred_buf;
> +	struct mtk_vcodec_mem mv_buf[H264_MAX_MV_NUM];
> +	struct vdec_vpu_inst vpu;
> +	struct vdec_h264_slice_vsi *vsi;
> +	struct vdec_h264_slice_vsi *vsi_core;
> +
> +	unsigned int resolution_changed;
> +	unsigned int realloc_mv_buf;
> +	unsigned int cap_num_planes;
> +
> +	struct v4l2_h264_dpb_entry dpb[16];
> +};
> +
> +static void vdec_h264_slice_fill_decode_parameters(
> +	struct vdec_h264_slice_inst *inst,
> +	struct vdec_h264_slice_share_info *share_info)
> +{
> +	struct vdec_h264_slice_lat_dec_param *slice_param =
> +		&inst->vsi->h264_slice_params;
> +	const struct v4l2_ctrl_h264_decode_params *dec_params =
> +		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
> +			V4L2_CID_STATELESS_H264_DECODE_PARAMS);
> +	const struct v4l2_ctrl_h264_scaling_matrix *src_matrix =
> +		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
> +			V4L2_CID_STATELESS_H264_SCALING_MATRIX);
> +	const struct v4l2_ctrl_h264_sps *sps =
> +		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
> +			V4L2_CID_STATELESS_H264_SPS);
> +	const struct v4l2_ctrl_h264_pps *pps =
> +		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
> +			V4L2_CID_STATELESS_H264_PPS);
> +
> +	mtk_vdec_h264_copy_sps_params(&slice_param->sps,sps);
> +	mtk_vdec_h264_copy_pps_params(&slice_param->pps, pps);
> +	mtk_vdec_h264_copy_scaling_matrix(
> +		&slice_param->scaling_matrix, src_matrix);
> +
> +	memcpy(&share_info->sps, sps, sizeof(*sps));
> +	memcpy(&share_info->dec_params, dec_params, sizeof(*dec_params));
> +}
> +
> +/*
> + * The firmware expects unused reflist entries to have the value 0x20.
> + */
> +static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> +{
> +	memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> +}
> +
> +static void vdec_h264_slice_fill_decode_reflist(
> +	struct vdec_h264_slice_inst *inst,
> +	struct vdec_h264_slice_lat_dec_param *slice_param,
> +	struct vdec_h264_slice_share_info *share_info)
> +{
> +	struct v4l2_ctrl_h264_decode_params *dec_params = &share_info->dec_params;
> +	struct v4l2_ctrl_h264_sps *sps = &share_info->sps;
> +	struct v4l2_h264_reflist_builder reflist_builder;
> +	u8 *p0_reflist = slice_param->decode_params.ref_pic_list_p0;
> +	u8 *b0_reflist = slice_param->decode_params.ref_pic_list_b0;
> +	u8 *b1_reflist = slice_param->decode_params.ref_pic_list_b1;
> +
> +	mtk_vdec_h264_update_dpb(dec_params, inst->dpb);
> +
> +	mtk_vdec_h264_copy_decode_params(&slice_param->decode_params, dec_params,
> +		inst->dpb);
> +	mtk_vdec_h264_fill_dpb_info(inst->ctx, &slice_param->decode_params,
> +		slice_param->h264_dpb_info);
> +
> +	mtk_v4l2_debug(3, "cur poc = %d\n", dec_params->bottom_field_order_cnt);
> +	/* Build the reference lists */
> +	v4l2_h264_init_reflist_builder(&reflist_builder, dec_params, sps,
> +				       inst->dpb);
> +	v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> +	v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> +
> +	/* Adapt the built lists to the firmware's expectations */
> +	fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> +	fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> +	fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> +}
> +
> +static int vdec_h264_slice_alloc_mv_buf(struct vdec_h264_slice_inst *inst,
> +	struct vdec_pic_info *pic)
> +{
> +	int i;
> +	int err;
> +	struct mtk_vcodec_mem *mem;
> +	unsigned int buf_sz = mtk_vdec_h264_get_mv_buf_size(
> +		pic->buf_w, pic->buf_h);
> +
> +	mtk_v4l2_debug(3, "size = 0x%x", buf_sz);
> +	for (i = 0; i < H264_MAX_MV_NUM; i++) {
> +		mem = &inst->mv_buf[i];
> +		if (mem->va)
> +			mtk_vcodec_mem_free(inst->ctx, mem);
> +		mem->size = buf_sz;
> +		err = mtk_vcodec_mem_alloc(inst->ctx, mem);
> +		if (err) {
> +			mtk_vcodec_err(inst, "failed to allocate mv buf");
> +			return err;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static void vdec_h264_slice_free_mv_buf(struct vdec_h264_slice_inst *inst)
> +{
> +	int i;
> +	struct mtk_vcodec_mem *mem;
> +
> +	for (i = 0; i < H264_MAX_MV_NUM; i++) {
> +		mem = &inst->mv_buf[i];
> +		if (mem->va)
> +			mtk_vcodec_mem_free(inst->ctx, mem);
> +	}
> +}
> +
> +static void vdec_h264_slice_get_pic_info(struct vdec_h264_slice_inst *inst)
> +{
> +	struct mtk_vcodec_ctx *ctx = inst->ctx;
> +	unsigned int data[3];
> +
> +	data[0] = ctx->picinfo.pic_w;
> +	data[1] = ctx->picinfo.pic_h;
> +	data[2] = ctx->capture_fourcc;
> +	vpu_dec_get_param(&inst->vpu, data, 3, GET_PARAM_PIC_INFO);
> +
> +	ctx->picinfo.buf_w = ALIGN(ctx->picinfo.pic_w, 64);
> +	ctx->picinfo.buf_h = ALIGN(ctx->picinfo.pic_h, 64);
> +	ctx->picinfo.fb_sz[0] = inst->vpu.fb_sz[0];
> +	ctx->picinfo.fb_sz[1] = inst->vpu.fb_sz[1];
> +	inst->cap_num_planes =
> +		ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes;
> +
> +	mtk_vcodec_debug(inst, "pic(%d, %d), buf(%d, %d)",
> +			 ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> +			 ctx->picinfo.buf_w, ctx->picinfo.buf_h);
> +	mtk_vcodec_debug(inst, "Y/C(%d, %d)", ctx->picinfo.fb_sz[0],
> +		ctx->picinfo.fb_sz[1]);
> +
> +	if ((ctx->last_decoded_picinfo.pic_w != ctx->picinfo.pic_w) ||
> +		(ctx->last_decoded_picinfo.pic_h != ctx->picinfo.pic_h)) {
> +		inst->resolution_changed = true;
> +		if ((ctx->last_decoded_picinfo.buf_w != ctx->picinfo.buf_w) ||
> +			(ctx->last_decoded_picinfo.buf_h != ctx->picinfo.buf_h))
> +			inst->realloc_mv_buf = true;
> +
> +		mtk_v4l2_debug(1, "resChg: (%d %d) : old(%d, %d) -> new(%d, %d)",
> +			inst->resolution_changed,
> +			inst->realloc_mv_buf,
> +			ctx->last_decoded_picinfo.pic_w,
> +			ctx->last_decoded_picinfo.pic_h,
> +			ctx->picinfo.pic_w, ctx->picinfo.pic_h);
> +	}
> +}
> +
> +static void vdec_h264_slice_get_crop_info(struct vdec_h264_slice_inst *inst,
> +	struct v4l2_rect *cr)
> +{
> +	cr->left = 0;
> +	cr->top = 0;
> +	cr->width = inst->ctx->picinfo.pic_w;
> +	cr->height = inst->ctx->picinfo.pic_h;
> +
> +	mtk_vcodec_debug(inst, "l=%d, t=%d, w=%d, h=%d",
> +			 cr->left, cr->top, cr->width, cr->height);
> +}
> +
> +static int vdec_h264_slice_init(struct mtk_vcodec_ctx *ctx)
> +{
> +	struct vdec_h264_slice_inst *inst;
> +	int err, vsi_size;
> +
> +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
> +	if (!inst)
> +		return -ENOMEM;
> +
> +	inst->ctx = ctx;
> +
> +	inst->vpu.id = SCP_IPI_VDEC_LAT;
> +	inst->vpu.core_id = SCP_IPI_VDEC_CORE;
> +	inst->vpu.ctx = ctx;
> +	inst->vpu.codec_type = ctx->current_codec;
> +	inst->vpu.capture_type = ctx->capture_fourcc;
> +
> +	err = vpu_dec_init(&inst->vpu);
> +	if (err) {
> +		mtk_vcodec_err(inst, "vdec_h264 init err=%d", err);
> +		goto error_free_inst;
> +	}
> +
> +	vsi_size = round_up(sizeof(struct vdec_h264_slice_vsi), 64);
> +	inst->vsi = inst->vpu.vsi;
> +	inst->vsi_core =
> +		(struct vdec_h264_slice_vsi *)(((char *)inst->vpu.vsi) + vsi_size);
> +	inst->resolution_changed = true;
> +	inst->realloc_mv_buf = true;
> +
> +	mtk_vcodec_debug(inst, "lat struct size = %d,%d,%d,%d vsi: %d\n",
> +		(int)sizeof(struct mtk_h264_sps_param),
> +		(int)sizeof(struct mtk_h264_pps_param),
> +		(int)sizeof(struct vdec_h264_slice_lat_dec_param),
> +		(int)sizeof(struct mtk_h264_dpb_info),
> +		vsi_size);
> +	mtk_vcodec_debug(inst, "lat H264 instance >> %p, codec_type = 0x%x",
> +		inst, inst->vpu.codec_type);
> +
> +	ctx->drv_handle = inst;
> +	return 0;
> +
> +error_free_inst:
> +	kfree(inst);
> +	return err;
> +}
> +
> +static void vdec_h264_slice_deinit(void *h_vdec)
> +{
> +	struct vdec_h264_slice_inst *inst = h_vdec;
> +
> +	mtk_vcodec_debug_enter(inst);
> +
> +	vpu_dec_deinit(&inst->vpu);
> +	vdec_h264_slice_free_mv_buf(inst);
> +	vdec_msg_queue_deinit(&inst->ctx->msg_queue, inst->ctx);
> +
> +	kfree(inst);
> +}
> +
> +static int vdec_h264_slice_core_decode(struct vdec_lat_buf *lat_buf)
> +{
> +	struct vdec_fb *fb;
> +	uint64_t vdec_fb_va;
> +	uint64_t y_fb_dma, c_fb_dma;
> +	int err, timeout, i, dec_err;
> +	struct vdec_vpu_inst *vpu;
> +	struct mtk_vcodec_ctx *ctx = lat_buf->ctx;
> +	struct vdec_h264_slice_inst *inst = ctx->drv_handle;
> +	struct vb2_v4l2_buffer *vb2_v4l2;
> +	struct vdec_h264_slice_share_info *share_info = lat_buf->private_data;
> +	struct mtk_vcodec_mem *mem;
> +
> +	mtk_vcodec_debug(inst, "[h264-core] vdec_h264 core decode");
> +	memcpy(&inst->vsi_core->h264_slice_params, &share_info->h264_slice_params,
> +		sizeof(share_info->h264_slice_params));
> +	fb = ctx->dev->vdec_pdata->get_cap_buffer(ctx);
> +	vpu = &inst->vpu;
> +	vdec_fb_va = (unsigned long)fb;
> +	y_fb_dma = fb ? (u64)fb->base_y.dma_addr : 0;
> +
> +	if (ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 1)
> +		c_fb_dma =
> +			y_fb_dma + inst->ctx->picinfo.buf_w * inst->ctx->picinfo.buf_h;
> +	else
> +		c_fb_dma = fb ? (u64)fb->base_c.dma_addr : 0;
> +
> +	mtk_vcodec_debug(inst, "[h264-core] y/c addr = 0x%llx 0x%llx", y_fb_dma,
> +		c_fb_dma);
> +
> +	inst->vsi_core->dec.y_fb_dma = y_fb_dma;
> +	inst->vsi_core->dec.c_fb_dma = c_fb_dma;
> +	inst->vsi_core->dec.vdec_fb_va = vdec_fb_va;
> +	inst->vsi_core->dec.nal_info = share_info->nal_info;
> +	inst->vsi_core->wdma_start_addr =
> +		lat_buf->ctx->msg_queue.wdma_addr.dma_addr;
> +	inst->vsi_core->wdma_end_addr =
> +		lat_buf->ctx->msg_queue.wdma_addr.dma_addr +
> +		lat_buf->ctx->msg_queue.wdma_addr.size;
> +	inst->vsi_core->wdma_err_addr = lat_buf->wdma_err_addr.dma_addr;
> +	inst->vsi_core->slice_bc_start_addr = lat_buf->slice_bc_addr.dma_addr;
> +	inst->vsi_core->slice_bc_end_addr = lat_buf->slice_bc_addr.dma_addr +
> +		lat_buf->slice_bc_addr.size;
> +	inst->vsi_core->trans_start = share_info->trans_start;
> +	inst->vsi_core->trans_end = share_info->trans_end;
> +	for (i = 0; i < H264_MAX_MV_NUM; i++) {
> +		mem = &inst->mv_buf[i];
> +		inst->vsi_core->mv_buf_dma[i] = mem->dma_addr;
> +	}
> +
> +	vb2_v4l2 = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
> +	vb2_v4l2->vb2_buf.timestamp = lat_buf->ts_info.vb2_buf.timestamp;
> +	vb2_v4l2->timecode = lat_buf->ts_info.timecode;
> +	vb2_v4l2->field = lat_buf->ts_info.field;
> +	vb2_v4l2->flags = lat_buf->ts_info.flags;
> +	vb2_v4l2->vb2_buf.copied_timestamp =
> +		lat_buf->ts_info.vb2_buf.copied_timestamp;
> +
> +	vdec_h264_slice_fill_decode_reflist(inst,
> +		&inst->vsi_core->h264_slice_params, share_info);
> +
> +	err = vpu_dec_core(vpu);
> +	if (err) {
> +		dec_err = 1;
> +		mtk_vcodec_err(inst, "core decode err=%d", err);
> +		goto vdec_dec_end;
> +	} else {
> +		dec_err = 0;
> +	}
> +
> +	/* wait decoder done interrupt */
> +	timeout = mtk_vcodec_wait_for_done_ctx(
> +		inst->ctx, MTK_INST_IRQ_RECEIVED,
> +		WAIT_INTR_TIMEOUT_MS, MTK_VDEC_CORE);
> +	if (timeout)
> +		mtk_vcodec_err(inst, "core decode timeout: pic_%d",
> +			ctx->decoded_frame_cnt);
> +	inst->vsi_core->dec.timeout = !!timeout;
> +
> +	vpu_dec_core_end(vpu);
> +	mtk_vcodec_debug(inst, "pic[%d] crc: 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x",
> +		ctx->decoded_frame_cnt,
> +		inst->vsi_core->dec.crc[0],	inst->vsi_core->dec.crc[1],
> +		inst->vsi_core->dec.crc[2],	inst->vsi_core->dec.crc[3],
> +		inst->vsi_core->dec.crc[4],	inst->vsi_core->dec.crc[5],
> +		inst->vsi_core->dec.crc[6],	inst->vsi_core->dec.crc[7]);
> +vdec_dec_end:
> +	vdec_msg_queue_update_ube_rptr(&lat_buf->ctx->msg_queue,
> +		inst->vsi_core->trans_end);
> +	ctx->dev->vdec_pdata->cap_to_disp(ctx, fb, dec_err);
> +	mtk_vcodec_debug(inst, "core decode done err=%d", err);
> +	ctx->decoded_frame_cnt++;
> +
> +	return 0;
> +}
> +
> +static int vdec_h264_slice_decode(void *h_vdec, struct mtk_vcodec_mem *bs,
> +	struct vdec_fb *fb, bool *res_chg)
> +{
> +	struct vdec_h264_slice_inst *inst = h_vdec;
> +	struct vdec_vpu_inst *vpu = &inst->vpu;
> +	struct mtk_video_dec_buf *src_buf_info;
> +	int nal_start_idx, err, timeout = 0, i;
> +	unsigned int nal_type, data[2];
> +	struct vdec_lat_buf *lat_buf;
> +	struct vdec_h264_slice_share_info *share_info;
> +	unsigned char *buf;
> +	struct mtk_vcodec_mem *mem;
> +
> +	mtk_vcodec_debug(inst, "+ [%d] ", ++inst->num_nalu);
> +
> +	if (vdec_msg_queue_init(&inst->ctx->msg_queue, inst->ctx,
> +		vdec_h264_slice_core_decode, sizeof(*share_info)))
> +	return -ENOMEM;
> +
> +	/* bs NULL means flush decoder */
> +	if (!bs) {
> +		vdec_msg_queue_wait_lat_buf_full(&inst->ctx->msg_queue);
> +		return vpu_dec_reset(vpu);
> +	}
> +
> +	lat_buf = vdec_msg_queue_dqbuf(&inst->ctx->msg_queue.lat_ctx);
> +	if (!lat_buf) {
> +		mtk_vcodec_err(inst, "failed to get lat buffer");
> +		return -EINVAL;
> +	}
> +	share_info = lat_buf->private_data;
> +	src_buf_info = container_of(bs, struct mtk_video_dec_buf, bs_buffer);
> +
> +	buf = (unsigned char *)bs->va;
> +	nal_start_idx = mtk_vdec_h264_find_start_code(buf, bs->size);
> +	if (nal_start_idx < 0) {
> +		err = -EINVAL;
> +		goto err_free_fb_out;
> +	}
> +
> +	inst->vsi->dec.nal_info = buf[nal_start_idx];
> +	nal_type = NAL_TYPE(buf[nal_start_idx]);
> +	mtk_vcodec_debug(inst, "\n + NALU[%d] type %d +\n", inst->num_nalu,
> +			 nal_type);
> +
> +	inst->vsi->dec.bs_buf_addr = (uint64_t)bs->dma_addr;
> +	inst->vsi->dec.bs_buf_size = bs->size;
> +
> +	v4l2_m2m_buf_copy_metadata(&src_buf_info->m2m_buf.vb,
> +		&lat_buf->ts_info, true);
> +
> +	vdec_h264_slice_fill_decode_parameters(inst, share_info);
> +	*res_chg = inst->resolution_changed;
> +	if (inst->resolution_changed) {
> +		mtk_vcodec_debug(inst, "- resolution changed -");
> +		if (inst->realloc_mv_buf) {
> +			err = vdec_h264_slice_alloc_mv_buf(inst, &inst->ctx->picinfo);
> +			inst->realloc_mv_buf = false;
> +			if (err)
> +				goto err_free_fb_out;
> +		}
> +		inst->resolution_changed = false;
> +	}
> +	for (i = 0; i < H264_MAX_MV_NUM; i++) {
> +		mem = &inst->mv_buf[i];
> +		inst->vsi->mv_buf_dma[i] = mem->dma_addr;
> +	}
> +	inst->vsi->wdma_start_addr = lat_buf->ctx->msg_queue.wdma_addr.dma_addr;
> +	inst->vsi->wdma_end_addr = lat_buf->ctx->msg_queue.wdma_addr.dma_addr +
> +		lat_buf->ctx->msg_queue.wdma_addr.size;
> +	inst->vsi->wdma_err_addr = lat_buf->wdma_err_addr.dma_addr;
> +	inst->vsi->slice_bc_start_addr = lat_buf->slice_bc_addr.dma_addr;
> +	inst->vsi->slice_bc_end_addr = lat_buf->slice_bc_addr.dma_addr +
> +		lat_buf->slice_bc_addr.size;
> +
> +	inst->vsi->trans_end = inst->ctx->msg_queue.wdma_rptr_addr;
> +	inst->vsi->trans_start = inst->ctx->msg_queue.wdma_wptr_addr;
> +	mtk_vcodec_debug(inst, "lat:trans(0x%llx 0x%llx)err:0x%llx",
> +		inst->vsi->wdma_start_addr,
> +		inst->vsi->wdma_end_addr,
> +		inst->vsi->wdma_err_addr);
> +
> +	mtk_vcodec_debug(inst, "slice(0x%llx 0x%llx) rprt((0x%llx 0x%llx))",
> +		inst->vsi->slice_bc_start_addr,
> +		inst->vsi->slice_bc_end_addr,
> +		inst->vsi->trans_start,
> +		inst->vsi->trans_end);
> +	err = vpu_dec_start(vpu, data, 2);
> +	if (err) {
> +		mtk_vcodec_debug(inst, "lat decode err: %d", err);
> +		goto err_free_fb_out;
> +	}
> +
> +	if (nal_type == NAL_NON_IDR_SLICE || nal_type == NAL_IDR_SLICE) {
> +		/* wait decoder done interrupt */
> +		timeout = mtk_vcodec_wait_for_done_ctx(
> +			inst->ctx, MTK_INST_IRQ_RECEIVED,
> +			WAIT_INTR_TIMEOUT_MS, MTK_VDEC_LAT0);
> +		inst->vsi->dec.timeout = !!timeout;
> +	}
> +	err = vpu_dec_end(vpu);
> +	if (err == SLICE_HEADER_FULL || timeout || err == TRANS_BUFFER_FULL) {
> +		err = -EINVAL;
> +		goto err_free_fb_out;
> +	}
> +
> +	share_info->trans_end = inst->ctx->msg_queue.wdma_addr.dma_addr +
> +		inst->vsi->wdma_end_addr_offset;
> +	share_info->trans_start = inst->ctx->msg_queue.wdma_wptr_addr;
> +	share_info->nal_info = inst->vsi->dec.nal_info;
> +	vdec_msg_queue_update_ube_wptr(&lat_buf->ctx->msg_queue,
> +		share_info->trans_end);
> +
> +	memcpy(&share_info->h264_slice_params, &inst->vsi->h264_slice_params,
> +		sizeof(share_info->h264_slice_params));
> +	vdec_msg_queue_qbuf(&inst->ctx->dev->msg_queue_core_ctx, lat_buf);
> +	mtk_vcodec_debug(inst, "- NALU[%d] type=%d -\n", inst->num_nalu,
> +			 nal_type);
> +	return 0;
> +
> +err_free_fb_out:
> +	mtk_vcodec_err(inst, "- NALU[%d] err=%d -\n", inst->num_nalu, err);
> +	return err;
> +}
> +
> +static int vdec_h264_slice_get_param(void *h_vdec,
> +	enum vdec_get_param_type type, void *out)
> +{
> +	struct vdec_h264_slice_inst *inst = h_vdec;
> +
> +	switch (type) {
> +	case GET_PARAM_PIC_INFO:
> +		vdec_h264_slice_get_pic_info(inst);
> +		break;
> +	case GET_PARAM_DPB_SIZE:
> +		*(unsigned int *)out = 6;
> +		break;
> +	case GET_PARAM_CROP_INFO:
> +		vdec_h264_slice_get_crop_info(inst, out);
> +		break;
> +	default:
> +		mtk_vcodec_err(inst, "invalid get parameter type=%d", type);
> +		return -EINVAL;
> +	}
> +	return 0;
> +}
> +
> +const struct vdec_common_if vdec_h264_slice_lat_if = {
> +	.init		= vdec_h264_slice_init,
> +	.decode		= vdec_h264_slice_decode,
> +	.get_param	= vdec_h264_slice_get_param,
> +	.deinit		= vdec_h264_slice_deinit,
> +};
> diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> index c93dd0ea3537..c17a7815e1bb 100644
> --- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> @@ -20,7 +20,13 @@ int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
>  
>  	switch (fourcc) {
>  	case V4L2_PIX_FMT_H264_SLICE:
> -		ctx->dec_if = &vdec_h264_slice_if;
> +		if (ctx->dev->vdec_pdata->hw_arch == MTK_VDEC_PURE_SINGLE_CORE) {
> +			ctx->dec_if = &vdec_h264_slice_if;
> +			ctx->hw_id = MTK_VDEC_CORE;
> +		} else {
> +			ctx->dec_if = &vdec_h264_slice_lat_if;
> +			ctx->hw_id = MTK_VDEC_LAT0;
> +		}
>  		break;
>  	case V4L2_PIX_FMT_H264:
>  		ctx->dec_if = &vdec_h264_if;
> diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.h b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> index d467e8af4a84..6ce848e74167 100644
> --- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> @@ -56,6 +56,7 @@ struct vdec_fb_node {
>  
>  extern const struct vdec_common_if vdec_h264_if;
>  extern const struct vdec_common_if vdec_h264_slice_if;
> +extern const struct vdec_common_if vdec_h264_slice_lat_if;
>  extern const struct vdec_common_if vdec_vp8_if;
>  extern const struct vdec_common_if vdec_vp9_if;
>  
> diff --git a/include/linux/remoteproc/mtk_scp.h b/include/linux/remoteproc/mtk_scp.h
> index b47416f7aeb8..a32ffd736e7b 100644
> --- a/include/linux/remoteproc/mtk_scp.h
> +++ b/include/linux/remoteproc/mtk_scp.h
> @@ -41,6 +41,8 @@ enum scp_ipi_id {
>  	SCP_IPI_ISP_FRAME,
>  	SCP_IPI_FD_CMD,
>  	SCP_IPI_CROS_HOST_CMD,
> +	SCP_IPI_VDEC_LAT,
> +	SPC_IPI_VDEC_CORE,
>  	SCP_IPI_NS_SERVICE = 0xFF,
>  	SCP_IPI_MAX = 0x100,
>  };

Re: [PATCH v1, 12/12] media: mtk-vcodec: Add h264 slice api driver for mt8192

[yunfei.dong]

Hi Nicolas,

Thanks for your suggestion.
On Wed, 2021-12-15 at 10:27 -0500, Nicolas Dufresne wrote:
> Hi Yunfei,
> 
> Le mercredi 15 décembre 2021 à 14:59 +0800, Yunfei Dong a écrit :
> > From: Yunfei Dong <yunfei.dong@mediatek.corp-partner.google.com>
> > 
> > Adds h264 lat and core driver for mt8192.
> 
> This is purely a nit, but I have first notice the usage of "slice" in
> the
> namespace and the title, which lead me to think this new platform was
> V4L2_STATELESS_H264_DECODE_MODE_SLICE_BASED. I think some structure
> which are

> clearly frame_based should probably be renamed (its the namespace
> that is
> confusing) to reduce the confusion.
> 
This driver is frame_based used for mt8182 lat and core architecture.
I will fix the commit in next patch.

Thanks for your remind.

Best Regards,
Yunfei Dong
> p.s. Note that adding slice_based mode would be amazing for streaming
> with ultra
> low latency (think remote video games)
> 
> regards,
> Nicolas
> 
> > 
> > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > ---
> >  drivers/media/platform/mtk-vcodec/Makefile    |   1 +
> >  .../mtk-vcodec/vdec/vdec_h264_req_lat_if.c    | 620
> > ++++++++++++++++++
> >  .../media/platform/mtk-vcodec/vdec_drv_if.c   |   8 +-
> >  .../media/platform/mtk-vcodec/vdec_drv_if.h   |   1 +
> >  include/linux/remoteproc/mtk_scp.h            |   2 +
> >  5 files changed, 631 insertions(+), 1 deletion(-)
> >  create mode 100644 drivers/media/platform/mtk-
> > vcodec/vdec/vdec_h264_req_lat_if.c
> > 
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile
> > b/drivers/media/platform/mtk-vcodec/Makefile
> > index 3f41d748eee5..1777d7606f0d 100644
> > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -10,6 +10,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
> >  		vdec/vdec_vp9_if.o \
> >  		vdec/vdec_h264_req_if.o \
> >  		vdec/vdec_h264_req_common.o \
> > +		vdec/vdec_h264_req_lat_if.o \
> >  		mtk_vcodec_dec_drv.o \
> >  		vdec_drv_if.o \
> >  		vdec_vpu_if.o \
> > diff --git a/drivers/media/platform/mtk-
> > vcodec/vdec/vdec_h264_req_lat_if.c b/drivers/media/platform/mtk-
> > vcodec/vdec/vdec_h264_req_lat_if.c
> > new file mode 100644
> > index 000000000000..403d7df00e1d
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_lat_if.c
> > @@ -0,0 +1,620 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2021 MediaTek Inc.
> > + * Author: Yunfei Dong <yunfei.dong@mediatek.com>
> > + */
> > +
> > +#include <linux/module.h>
> > +#include <linux/slab.h>
> > +#include <media/v4l2-h264.h>
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "../mtk_vcodec_util.h"
> > +#include "../mtk_vcodec_dec.h"
> > +#include "../mtk_vcodec_intr.h"
> > +#include "../vdec_drv_base.h"
> > +#include "../vdec_drv_if.h"
> > +#include "../vdec_vpu_if.h"
> > +#include "vdec_h264_req_common.h"
> > +
> > +/**
> > + * enum vdec_h264_core_dec_err_type  - core decode error type
> > + */
> > +enum vdec_h264_core_dec_err_type {
> > +	TRANS_BUFFER_FULL = 1,
> > +	SLICE_HEADER_FULL,
> > +};
> > +
> > +/**
> > + * struct vdec_h264_slice_lat_dec_param  - parameters for decode
> > current frame
> > + * @sps : h264 sps syntax parameters
> > + * @pps : h264 pps syntax parameters
> > + * @slice_header: h264 slice header syntax parameters
> > + * @scaling_matrix : h264 scaling list parameters
> > + * @decode_params : decoder parameters of each frame used for
> > hardware decode
> > + * @h264_dpb_info : dpb reference list
> > + */
> > +struct vdec_h264_slice_lat_dec_param {
> > +	struct mtk_h264_sps_param sps;
> > +	struct mtk_h264_pps_param pps;
> > +	struct mtk_h264_slice_hd_param slice_header;
> > +	struct slice_api_h264_scaling_matrix scaling_matrix;
> > +	struct slice_api_h264_decode_param decode_params;
> > +	struct mtk_h264_dpb_info
> > h264_dpb_info[V4L2_H264_NUM_DPB_ENTRIES];
> > +};
> > +
> > +/**
> > + * struct vdec_h264_slice_info - decode information
> > + * @nal_info    : nal info of current picture
> > + * @timeout     : Decode timeout: 1 timeout, 0 no timeount
> > + * @bs_buf_size : bitstream size
> > + * @bs_buf_addr : bitstream buffer dma address
> > + * @y_fb_dma    : Y frame buffer dma address
> > + * @c_fb_dma    : C frame buffer dma address
> > + * @vdec_fb_va  : VDEC frame buffer struct virtual address
> > + * @crc         : Used to check whether hardware's status is right
> > + */
> > +struct vdec_h264_slice_info {
> > +	uint16_t nal_info;
> > +	uint16_t timeout;
> > +	uint32_t bs_buf_size;
> > +	uint64_t bs_buf_addr;
> > +	uint64_t y_fb_dma;
> > +	uint64_t c_fb_dma;
> > +	uint64_t vdec_fb_va;
> > +	uint32_t crc[8];
> > +};
> > +
> > +/**
> > + * struct vdec_h264_slice_vsi - shared memory for decode
> > information exchange
> > + *        between VPU and Host. The memory is allocated by VPU
> > then mapping to
> > + *        Host in vdec_h264_slice_init() and freed in
> > vdec_h264_slice_deinit()
> > + *        by VPU. AP-W/R : AP is writer/reader on this item. VPU-
> > W/R: VPU is
> > + *        write/reader on this item.
> > + * @wdma_err_addr       : wdma error dma address
> > + * @wdma_start_addr     : wdma start dma address
> > + * @wdma_end_addr       : wdma end dma address
> > + * @slice_bc_start_addr : slice bc start dma address
> > + * @slice_bc_end_addr   : slice bc end dma address
> > + * @row_info_start_addr : row info start dma address
> > + * @row_info_end_addr   : row info end dma address
> > + * @trans_start         : trans start dma address
> > + * @trans_end           : trans end dma address
> > + * @wdma_end_addr_offset: wdma end address offset
> > + * @mv_buf_dma          : HW working motion vector buffer
> > + *                        dma address (AP-W, VPU-R)
> > + * @dec                 : decode information (AP-R, VPU-W)
> > + * @h264_slice_params   : decode parameters for hw used
> > + */
> > +struct vdec_h264_slice_vsi {
> > +	/* LAT dec addr */
> > +	uint64_t wdma_err_addr;
> > +	uint64_t wdma_start_addr;
> > +	uint64_t wdma_end_addr;
> > +	uint64_t slice_bc_start_addr;
> > +	uint64_t slice_bc_end_addr;
> > +	uint64_t row_info_start_addr;
> > +	uint64_t row_info_end_addr;
> > +	uint64_t trans_start;
> > +	uint64_t trans_end;
> > +	uint64_t wdma_end_addr_offset;
> > +
> > +	uint64_t mv_buf_dma[H264_MAX_MV_NUM];
> > +	struct vdec_h264_slice_info dec;
> > +	struct vdec_h264_slice_lat_dec_param h264_slice_params;
> > +};
> > +
> > +/**
> > + * struct vdec_h264_slice_share_info - shared information used to
> > exchange
> > + *                                     message between lat and
> > core
> > + * @sps	              : sequence header information from user
> > space
> > + * @dec_params        : decoder params from user space
> > + * @h264_slice_params : decoder params used for hardware
> > + * @trans_start       : trans start dma address
> > + * @trans_end         : trans end dma address
> > + * @nal_info          : nal info of current picture
> > + */
> > +struct vdec_h264_slice_share_info {
> > +	struct v4l2_ctrl_h264_sps sps;
> > +	struct v4l2_ctrl_h264_decode_params dec_params;
> > +	struct vdec_h264_slice_lat_dec_param h264_slice_params;
> > +	uint64_t trans_start;
> > +	uint64_t trans_end;
> > +	uint16_t nal_info;
> > +};
> > +
> > +/**
> > + * struct vdec_h264_slice_inst - h264 decoder instance
> > + * @num_nalu            : how many nalus be decoded
> > + * @ctx                 : point to mtk_vcodec_ctx
> > + * @pred_buf            : HW working predication buffer
> > + * @mv_buf              : HW working motion vector buffer
> > + * @vpu                 : VPU instance
> > + * @vsi                 : vsi used for lat
> > + * @vsi_core            : vsi used for core
> > + * @resolution_changed  : resolution changed
> > + * @realloc_mv_buf      : reallocate mv buffer
> > + * @cap_num_planes      : number of capture queue plane
> > + */
> > +struct vdec_h264_slice_inst {
> > +	unsigned int num_nalu;
> > +	struct mtk_vcodec_ctx *ctx;
> > +	struct mtk_vcodec_mem pred_buf;
> > +	struct mtk_vcodec_mem mv_buf[H264_MAX_MV_NUM];
> > +	struct vdec_vpu_inst vpu;
> > +	struct vdec_h264_slice_vsi *vsi;
> > +	struct vdec_h264_slice_vsi *vsi_core;
> > +
> > +	unsigned int resolution_changed;
> > +	unsigned int realloc_mv_buf;
> > +	unsigned int cap_num_planes;
> > +
> > +	struct v4l2_h264_dpb_entry dpb[16];
> > +};
> > +
> > +static void vdec_h264_slice_fill_decode_parameters(
> > +	struct vdec_h264_slice_inst *inst,
> > +	struct vdec_h264_slice_share_info *share_info)
> > +{
> > +	struct vdec_h264_slice_lat_dec_param *slice_param =
> > +		&inst->vsi->h264_slice_params;
> > +	const struct v4l2_ctrl_h264_decode_params *dec_params =
> > +		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
> > +			V4L2_CID_STATELESS_H264_DECODE_PARAMS);
> > +	const struct v4l2_ctrl_h264_scaling_matrix *src_matrix =
> > +		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
> > +			V4L2_CID_STATELESS_H264_SCALING_MATRIX);
> > +	const struct v4l2_ctrl_h264_sps *sps =
> > +		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
> > +			V4L2_CID_STATELESS_H264_SPS);
> > +	const struct v4l2_ctrl_h264_pps *pps =
> > +		mtk_vdec_h264_get_ctrl_ptr(inst->ctx,
> > +			V4L2_CID_STATELESS_H264_PPS);
> > +
> > +	mtk_vdec_h264_copy_sps_params(&slice_param->sps,sps);
> > +	mtk_vdec_h264_copy_pps_params(&slice_param->pps, pps);
> > +	mtk_vdec_h264_copy_scaling_matrix(
> > +		&slice_param->scaling_matrix, src_matrix);
> > +
> > +	memcpy(&share_info->sps, sps, sizeof(*sps));
> > +	memcpy(&share_info->dec_params, dec_params,
> > sizeof(*dec_params));
> > +}
> > +
> > +/*
> > + * The firmware expects unused reflist entries to have the value
> > 0x20.
> > + */
> > +static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> > +{
> > +	memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> > +}
> > +
> > +static void vdec_h264_slice_fill_decode_reflist(
> > +	struct vdec_h264_slice_inst *inst,
> > +	struct vdec_h264_slice_lat_dec_param *slice_param,
> > +	struct vdec_h264_slice_share_info *share_info)
> > +{
> > +	struct v4l2_ctrl_h264_decode_params *dec_params = &share_info-
> > >dec_params;
> > +	struct v4l2_ctrl_h264_sps *sps = &share_info->sps;
> > +	struct v4l2_h264_reflist_builder reflist_builder;
> > +	u8 *p0_reflist = slice_param->decode_params.ref_pic_list_p0;
> > +	u8 *b0_reflist = slice_param->decode_params.ref_pic_list_b0;
> > +	u8 *b1_reflist = slice_param->decode_params.ref_pic_list_b1;
> > +
> > +	mtk_vdec_h264_update_dpb(dec_params, inst->dpb);
> > +
> > +	mtk_vdec_h264_copy_decode_params(&slice_param->decode_params,
> > dec_params,
> > +		inst->dpb);
> > +	mtk_vdec_h264_fill_dpb_info(inst->ctx, &slice_param-
> > >decode_params,
> > +		slice_param->h264_dpb_info);
> > +
> > +	mtk_v4l2_debug(3, "cur poc = %d\n", dec_params-
> > >bottom_field_order_cnt);
> > +	/* Build the reference lists */
> > +	v4l2_h264_init_reflist_builder(&reflist_builder, dec_params,
> > sps,
> > +				       inst->dpb);
> > +	v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> > +	v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist,
> > b1_reflist);
> > +
> > +	/* Adapt the built lists to the firmware's expectations */
> > +	fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> > +	fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> > +	fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> > +}
> > +
> > +static int vdec_h264_slice_alloc_mv_buf(struct
> > vdec_h264_slice_inst *inst,
> > +	struct vdec_pic_info *pic)
> > +{
> > +	int i;
> > +	int err;
> > +	struct mtk_vcodec_mem *mem;
> > +	unsigned int buf_sz = mtk_vdec_h264_get_mv_buf_size(
> > +		pic->buf_w, pic->buf_h);
> > +
> > +	mtk_v4l2_debug(3, "size = 0x%x", buf_sz);
> > +	for (i = 0; i < H264_MAX_MV_NUM; i++) {
> > +		mem = &inst->mv_buf[i];
> > +		if (mem->va)
> > +			mtk_vcodec_mem_free(inst->ctx, mem);
> > +		mem->size = buf_sz;
> > +		err = mtk_vcodec_mem_alloc(inst->ctx, mem);
> > +		if (err) {
> > +			mtk_vcodec_err(inst, "failed to allocate mv
> > buf");
> > +			return err;
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static void vdec_h264_slice_free_mv_buf(struct
> > vdec_h264_slice_inst *inst)
> > +{
> > +	int i;
> > +	struct mtk_vcodec_mem *mem;
> > +
> > +	for (i = 0; i < H264_MAX_MV_NUM; i++) {
> > +		mem = &inst->mv_buf[i];
> > +		if (mem->va)
> > +			mtk_vcodec_mem_free(inst->ctx, mem);
> > +	}
> > +}
> > +
> > +static void vdec_h264_slice_get_pic_info(struct
> > vdec_h264_slice_inst *inst)
> > +{
> > +	struct mtk_vcodec_ctx *ctx = inst->ctx;
> > +	unsigned int data[3];
> > +
> > +	data[0] = ctx->picinfo.pic_w;
> > +	data[1] = ctx->picinfo.pic_h;
> > +	data[2] = ctx->capture_fourcc;
> > +	vpu_dec_get_param(&inst->vpu, data, 3, GET_PARAM_PIC_INFO);
> > +
> > +	ctx->picinfo.buf_w = ALIGN(ctx->picinfo.pic_w, 64);
> > +	ctx->picinfo.buf_h = ALIGN(ctx->picinfo.pic_h, 64);
> > +	ctx->picinfo.fb_sz[0] = inst->vpu.fb_sz[0];
> > +	ctx->picinfo.fb_sz[1] = inst->vpu.fb_sz[1];
> > +	inst->cap_num_planes =
> > +		ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes;
> > +
> > +	mtk_vcodec_debug(inst, "pic(%d, %d), buf(%d, %d)",
> > +			 ctx->picinfo.pic_w, ctx->picinfo.pic_h,
> > +			 ctx->picinfo.buf_w, ctx->picinfo.buf_h);
> > +	mtk_vcodec_debug(inst, "Y/C(%d, %d)", ctx->picinfo.fb_sz[0],
> > +		ctx->picinfo.fb_sz[1]);
> > +
> > +	if ((ctx->last_decoded_picinfo.pic_w != ctx->picinfo.pic_w) ||
> > +		(ctx->last_decoded_picinfo.pic_h != ctx-
> > >picinfo.pic_h)) {
> > +		inst->resolution_changed = true;
> > +		if ((ctx->last_decoded_picinfo.buf_w != ctx-
> > >picinfo.buf_w) ||
> > +			(ctx->last_decoded_picinfo.buf_h != ctx-
> > >picinfo.buf_h))
> > +			inst->realloc_mv_buf = true;
> > +
> > +		mtk_v4l2_debug(1, "resChg: (%d %d) : old(%d, %d) ->
> > new(%d, %d)",
> > +			inst->resolution_changed,
> > +			inst->realloc_mv_buf,
> > +			ctx->last_decoded_picinfo.pic_w,
> > +			ctx->last_decoded_picinfo.pic_h,
> > +			ctx->picinfo.pic_w, ctx->picinfo.pic_h);
> > +	}
> > +}
> > +
> > +static void vdec_h264_slice_get_crop_info(struct
> > vdec_h264_slice_inst *inst,
> > +	struct v4l2_rect *cr)
> > +{
> > +	cr->left = 0;
> > +	cr->top = 0;
> > +	cr->width = inst->ctx->picinfo.pic_w;
> > +	cr->height = inst->ctx->picinfo.pic_h;
> > +
> > +	mtk_vcodec_debug(inst, "l=%d, t=%d, w=%d, h=%d",
> > +			 cr->left, cr->top, cr->width, cr->height);
> > +}
> > +
> > +static int vdec_h264_slice_init(struct mtk_vcodec_ctx *ctx)
> > +{
> > +	struct vdec_h264_slice_inst *inst;
> > +	int err, vsi_size;
> > +
> > +	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
> > +	if (!inst)
> > +		return -ENOMEM;
> > +
> > +	inst->ctx = ctx;
> > +
> > +	inst->vpu.id = SCP_IPI_VDEC_LAT;
> > +	inst->vpu.core_id = SCP_IPI_VDEC_CORE;
> > +	inst->vpu.ctx = ctx;
> > +	inst->vpu.codec_type = ctx->current_codec;
> > +	inst->vpu.capture_type = ctx->capture_fourcc;
> > +
> > +	err = vpu_dec_init(&inst->vpu);
> > +	if (err) {
> > +		mtk_vcodec_err(inst, "vdec_h264 init err=%d", err);
> > +		goto error_free_inst;
> > +	}
> > +
> > +	vsi_size = round_up(sizeof(struct vdec_h264_slice_vsi), 64);
> > +	inst->vsi = inst->vpu.vsi;
> > +	inst->vsi_core =
> > +		(struct vdec_h264_slice_vsi *)(((char *)inst->vpu.vsi)
> > + vsi_size);
> > +	inst->resolution_changed = true;
> > +	inst->realloc_mv_buf = true;
> > +
> > +	mtk_vcodec_debug(inst, "lat struct size = %d,%d,%d,%d vsi:
> > %d\n",
> > +		(int)sizeof(struct mtk_h264_sps_param),
> > +		(int)sizeof(struct mtk_h264_pps_param),
> > +		(int)sizeof(struct vdec_h264_slice_lat_dec_param),
> > +		(int)sizeof(struct mtk_h264_dpb_info),
> > +		vsi_size);
> > +	mtk_vcodec_debug(inst, "lat H264 instance >> %p, codec_type =
> > 0x%x",
> > +		inst, inst->vpu.codec_type);
> > +
> > +	ctx->drv_handle = inst;
> > +	return 0;
> > +
> > +error_free_inst:
> > +	kfree(inst);
> > +	return err;
> > +}
> > +
> > +static void vdec_h264_slice_deinit(void *h_vdec)
> > +{
> > +	struct vdec_h264_slice_inst *inst = h_vdec;
> > +
> > +	mtk_vcodec_debug_enter(inst);
> > +
> > +	vpu_dec_deinit(&inst->vpu);
> > +	vdec_h264_slice_free_mv_buf(inst);
> > +	vdec_msg_queue_deinit(&inst->ctx->msg_queue, inst->ctx);
> > +
> > +	kfree(inst);
> > +}
> > +
> > +static int vdec_h264_slice_core_decode(struct vdec_lat_buf
> > *lat_buf)
> > +{
> > +	struct vdec_fb *fb;
> > +	uint64_t vdec_fb_va;
> > +	uint64_t y_fb_dma, c_fb_dma;
> > +	int err, timeout, i, dec_err;
> > +	struct vdec_vpu_inst *vpu;
> > +	struct mtk_vcodec_ctx *ctx = lat_buf->ctx;
> > +	struct vdec_h264_slice_inst *inst = ctx->drv_handle;
> > +	struct vb2_v4l2_buffer *vb2_v4l2;
> > +	struct vdec_h264_slice_share_info *share_info = lat_buf-
> > >private_data;
> > +	struct mtk_vcodec_mem *mem;
> > +
> > +	mtk_vcodec_debug(inst, "[h264-core] vdec_h264 core decode");
> > +	memcpy(&inst->vsi_core->h264_slice_params, &share_info-
> > >h264_slice_params,
> > +		sizeof(share_info->h264_slice_params));
> > +	fb = ctx->dev->vdec_pdata->get_cap_buffer(ctx);
> > +	vpu = &inst->vpu;
> > +	vdec_fb_va = (unsigned long)fb;
> > +	y_fb_dma = fb ? (u64)fb->base_y.dma_addr : 0;
> > +
> > +	if (ctx->q_data[MTK_Q_DATA_DST].fmt->num_planes == 1)
> > +		c_fb_dma =
> > +			y_fb_dma + inst->ctx->picinfo.buf_w * inst-
> > >ctx->picinfo.buf_h;
> > +	else
> > +		c_fb_dma = fb ? (u64)fb->base_c.dma_addr : 0;
> > +
> > +	mtk_vcodec_debug(inst, "[h264-core] y/c addr = 0x%llx 0x%llx",
> > y_fb_dma,
> > +		c_fb_dma);
> > +
> > +	inst->vsi_core->dec.y_fb_dma = y_fb_dma;
> > +	inst->vsi_core->dec.c_fb_dma = c_fb_dma;
> > +	inst->vsi_core->dec.vdec_fb_va = vdec_fb_va;
> > +	inst->vsi_core->dec.nal_info = share_info->nal_info;
> > +	inst->vsi_core->wdma_start_addr =
> > +		lat_buf->ctx->msg_queue.wdma_addr.dma_addr;
> > +	inst->vsi_core->wdma_end_addr =
> > +		lat_buf->ctx->msg_queue.wdma_addr.dma_addr +
> > +		lat_buf->ctx->msg_queue.wdma_addr.size;
> > +	inst->vsi_core->wdma_err_addr = lat_buf-
> > >wdma_err_addr.dma_addr;
> > +	inst->vsi_core->slice_bc_start_addr = lat_buf-
> > >slice_bc_addr.dma_addr;
> > +	inst->vsi_core->slice_bc_end_addr = lat_buf-
> > >slice_bc_addr.dma_addr +
> > +		lat_buf->slice_bc_addr.size;
> > +	inst->vsi_core->trans_start = share_info->trans_start;
> > +	inst->vsi_core->trans_end = share_info->trans_end;
> > +	for (i = 0; i < H264_MAX_MV_NUM; i++) {
> > +		mem = &inst->mv_buf[i];
> > +		inst->vsi_core->mv_buf_dma[i] = mem->dma_addr;
> > +	}
> > +
> > +	vb2_v4l2 = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
> > +	vb2_v4l2->vb2_buf.timestamp = lat_buf-
> > >ts_info.vb2_buf.timestamp;
> > +	vb2_v4l2->timecode = lat_buf->ts_info.timecode;
> > +	vb2_v4l2->field = lat_buf->ts_info.field;
> > +	vb2_v4l2->flags = lat_buf->ts_info.flags;
> > +	vb2_v4l2->vb2_buf.copied_timestamp =
> > +		lat_buf->ts_info.vb2_buf.copied_timestamp;
> > +
> > +	vdec_h264_slice_fill_decode_reflist(inst,
> > +		&inst->vsi_core->h264_slice_params, share_info);
> > +
> > +	err = vpu_dec_core(vpu);
> > +	if (err) {
> > +		dec_err = 1;
> > +		mtk_vcodec_err(inst, "core decode err=%d", err);
> > +		goto vdec_dec_end;
> > +	} else {
> > +		dec_err = 0;
> > +	}
> > +
> > +	/* wait decoder done interrupt */
> > +	timeout = mtk_vcodec_wait_for_done_ctx(
> > +		inst->ctx, MTK_INST_IRQ_RECEIVED,
> > +		WAIT_INTR_TIMEOUT_MS, MTK_VDEC_CORE);
> > +	if (timeout)
> > +		mtk_vcodec_err(inst, "core decode timeout: pic_%d",
> > +			ctx->decoded_frame_cnt);
> > +	inst->vsi_core->dec.timeout = !!timeout;
> > +
> > +	vpu_dec_core_end(vpu);
> > +	mtk_vcodec_debug(inst, "pic[%d] crc: 0x%x 0x%x 0x%x 0x%x 0x%x
> > 0x%x 0x%x 0x%x",
> > +		ctx->decoded_frame_cnt,
> > +		inst->vsi_core->dec.crc[0],	inst->vsi_core->dec.crc[1],
> > +		inst->vsi_core->dec.crc[2],	inst->vsi_core->dec.crc[3],
> > +		inst->vsi_core->dec.crc[4],	inst->vsi_core->dec.crc[5],
> > +		inst->vsi_core->dec.crc[6],	inst->vsi_core-
> > >dec.crc[7]);
> > +vdec_dec_end:
> > +	vdec_msg_queue_update_ube_rptr(&lat_buf->ctx->msg_queue,
> > +		inst->vsi_core->trans_end);
> > +	ctx->dev->vdec_pdata->cap_to_disp(ctx, fb, dec_err);
> > +	mtk_vcodec_debug(inst, "core decode done err=%d", err);
> > +	ctx->decoded_frame_cnt++;
> > +
> > +	return 0;
> > +}
> > +
> > +static int vdec_h264_slice_decode(void *h_vdec, struct
> > mtk_vcodec_mem *bs,
> > +	struct vdec_fb *fb, bool *res_chg)
> > +{
> > +	struct vdec_h264_slice_inst *inst = h_vdec;
> > +	struct vdec_vpu_inst *vpu = &inst->vpu;
> > +	struct mtk_video_dec_buf *src_buf_info;
> > +	int nal_start_idx, err, timeout = 0, i;
> > +	unsigned int nal_type, data[2];
> > +	struct vdec_lat_buf *lat_buf;
> > +	struct vdec_h264_slice_share_info *share_info;
> > +	unsigned char *buf;
> > +	struct mtk_vcodec_mem *mem;
> > +
> > +	mtk_vcodec_debug(inst, "+ [%d] ", ++inst->num_nalu);
> > +
> > +	if (vdec_msg_queue_init(&inst->ctx->msg_queue, inst->ctx,
> > +		vdec_h264_slice_core_decode, sizeof(*share_info)))
> > +	return -ENOMEM;
> > +
> > +	/* bs NULL means flush decoder */
> > +	if (!bs) {
> > +		vdec_msg_queue_wait_lat_buf_full(&inst->ctx-
> > >msg_queue);
> > +		return vpu_dec_reset(vpu);
> > +	}
> > +
> > +	lat_buf = vdec_msg_queue_dqbuf(&inst->ctx->msg_queue.lat_ctx);
> > +	if (!lat_buf) {
> > +		mtk_vcodec_err(inst, "failed to get lat buffer");
> > +		return -EINVAL;
> > +	}
> > +	share_info = lat_buf->private_data;
> > +	src_buf_info = container_of(bs, struct mtk_video_dec_buf,
> > bs_buffer);
> > +
> > +	buf = (unsigned char *)bs->va;
> > +	nal_start_idx = mtk_vdec_h264_find_start_code(buf, bs->size);
> > +	if (nal_start_idx < 0) {
> > +		err = -EINVAL;
> > +		goto err_free_fb_out;
> > +	}
> > +
> > +	inst->vsi->dec.nal_info = buf[nal_start_idx];
> > +	nal_type = NAL_TYPE(buf[nal_start_idx]);
> > +	mtk_vcodec_debug(inst, "\n + NALU[%d] type %d +\n", inst-
> > >num_nalu,
> > +			 nal_type);
> > +
> > +	inst->vsi->dec.bs_buf_addr = (uint64_t)bs->dma_addr;
> > +	inst->vsi->dec.bs_buf_size = bs->size;
> > +
> > +	v4l2_m2m_buf_copy_metadata(&src_buf_info->m2m_buf.vb,
> > +		&lat_buf->ts_info, true);
> > +
> > +	vdec_h264_slice_fill_decode_parameters(inst, share_info);
> > +	*res_chg = inst->resolution_changed;
> > +	if (inst->resolution_changed) {
> > +		mtk_vcodec_debug(inst, "- resolution changed -");
> > +		if (inst->realloc_mv_buf) {
> > +			err = vdec_h264_slice_alloc_mv_buf(inst, &inst-
> > >ctx->picinfo);
> > +			inst->realloc_mv_buf = false;
> > +			if (err)
> > +				goto err_free_fb_out;
> > +		}
> > +		inst->resolution_changed = false;
> > +	}
> > +	for (i = 0; i < H264_MAX_MV_NUM; i++) {
> > +		mem = &inst->mv_buf[i];
> > +		inst->vsi->mv_buf_dma[i] = mem->dma_addr;
> > +	}
> > +	inst->vsi->wdma_start_addr = lat_buf->ctx-
> > >msg_queue.wdma_addr.dma_addr;
> > +	inst->vsi->wdma_end_addr = lat_buf->ctx-
> > >msg_queue.wdma_addr.dma_addr +
> > +		lat_buf->ctx->msg_queue.wdma_addr.size;
> > +	inst->vsi->wdma_err_addr = lat_buf->wdma_err_addr.dma_addr;
> > +	inst->vsi->slice_bc_start_addr = lat_buf-
> > >slice_bc_addr.dma_addr;
> > +	inst->vsi->slice_bc_end_addr = lat_buf->slice_bc_addr.dma_addr
> > +
> > +		lat_buf->slice_bc_addr.size;
> > +
> > +	inst->vsi->trans_end = inst->ctx->msg_queue.wdma_rptr_addr;
> > +	inst->vsi->trans_start = inst->ctx->msg_queue.wdma_wptr_addr;
> > +	mtk_vcodec_debug(inst, "lat:trans(0x%llx 0x%llx)err:0x%llx",
> > +		inst->vsi->wdma_start_addr,
> > +		inst->vsi->wdma_end_addr,
> > +		inst->vsi->wdma_err_addr);
> > +
> > +	mtk_vcodec_debug(inst, "slice(0x%llx 0x%llx) rprt((0x%llx
> > 0x%llx))",
> > +		inst->vsi->slice_bc_start_addr,
> > +		inst->vsi->slice_bc_end_addr,
> > +		inst->vsi->trans_start,
> > +		inst->vsi->trans_end);
> > +	err = vpu_dec_start(vpu, data, 2);
> > +	if (err) {
> > +		mtk_vcodec_debug(inst, "lat decode err: %d", err);
> > +		goto err_free_fb_out;
> > +	}
> > +
> > +	if (nal_type == NAL_NON_IDR_SLICE || nal_type == NAL_IDR_SLICE)
> > {
> > +		/* wait decoder done interrupt */
> > +		timeout = mtk_vcodec_wait_for_done_ctx(
> > +			inst->ctx, MTK_INST_IRQ_RECEIVED,
> > +			WAIT_INTR_TIMEOUT_MS, MTK_VDEC_LAT0);
> > +		inst->vsi->dec.timeout = !!timeout;
> > +	}
> > +	err = vpu_dec_end(vpu);
> > +	if (err == SLICE_HEADER_FULL || timeout || err ==
> > TRANS_BUFFER_FULL) {
> > +		err = -EINVAL;
> > +		goto err_free_fb_out;
> > +	}
> > +
> > +	share_info->trans_end = inst->ctx->msg_queue.wdma_addr.dma_addr 
> > +
> > +		inst->vsi->wdma_end_addr_offset;
> > +	share_info->trans_start = inst->ctx->msg_queue.wdma_wptr_addr;
> > +	share_info->nal_info = inst->vsi->dec.nal_info;
> > +	vdec_msg_queue_update_ube_wptr(&lat_buf->ctx->msg_queue,
> > +		share_info->trans_end);
> > +
> > +	memcpy(&share_info->h264_slice_params, &inst->vsi-
> > >h264_slice_params,
> > +		sizeof(share_info->h264_slice_params));
> > +	vdec_msg_queue_qbuf(&inst->ctx->dev->msg_queue_core_ctx,
> > lat_buf);
> > +	mtk_vcodec_debug(inst, "- NALU[%d] type=%d -\n", inst-
> > >num_nalu,
> > +			 nal_type);
> > +	return 0;
> > +
> > +err_free_fb_out:
> > +	mtk_vcodec_err(inst, "- NALU[%d] err=%d -\n", inst->num_nalu,
> > err);
> > +	return err;
> > +}
> > +
> > +static int vdec_h264_slice_get_param(void *h_vdec,
> > +	enum vdec_get_param_type type, void *out)
> > +{
> > +	struct vdec_h264_slice_inst *inst = h_vdec;
> > +
> > +	switch (type) {
> > +	case GET_PARAM_PIC_INFO:
> > +		vdec_h264_slice_get_pic_info(inst);
> > +		break;
> > +	case GET_PARAM_DPB_SIZE:
> > +		*(unsigned int *)out = 6;
> > +		break;
> > +	case GET_PARAM_CROP_INFO:
> > +		vdec_h264_slice_get_crop_info(inst, out);
> > +		break;
> > +	default:
> > +		mtk_vcodec_err(inst, "invalid get parameter type=%d",
> > type);
> > +		return -EINVAL;
> > +	}
> > +	return 0;
> > +}
> > +
> > +const struct vdec_common_if vdec_h264_slice_lat_if = {
> > +	.init		= vdec_h264_slice_init,
> > +	.decode		= vdec_h264_slice_decode,
> > +	.get_param	= vdec_h264_slice_get_param,
> > +	.deinit		= vdec_h264_slice_deinit,
> > +};
> > diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> > b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> > index c93dd0ea3537..c17a7815e1bb 100644
> > --- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> > +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> > @@ -20,7 +20,13 @@ int vdec_if_init(struct mtk_vcodec_ctx *ctx,
> > unsigned int fourcc)
> >  
> >  	switch (fourcc) {
> >  	case V4L2_PIX_FMT_H264_SLICE:
> > -		ctx->dec_if = &vdec_h264_slice_if;
> > +		if (ctx->dev->vdec_pdata->hw_arch ==
> > MTK_VDEC_PURE_SINGLE_CORE) {
> > +			ctx->dec_if = &vdec_h264_slice_if;
> > +			ctx->hw_id = MTK_VDEC_CORE;
> > +		} else {
> > +			ctx->dec_if = &vdec_h264_slice_lat_if;
> > +			ctx->hw_id = MTK_VDEC_LAT0;
> > +		}
> >  		break;
> >  	case V4L2_PIX_FMT_H264:
> >  		ctx->dec_if = &vdec_h264_if;
> > diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> > b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> > index d467e8af4a84..6ce848e74167 100644
> > --- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> > +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.h
> > @@ -56,6 +56,7 @@ struct vdec_fb_node {
> >  
> >  extern const struct vdec_common_if vdec_h264_if;
> >  extern const struct vdec_common_if vdec_h264_slice_if;
> > +extern const struct vdec_common_if vdec_h264_slice_lat_if;
> >  extern const struct vdec_common_if vdec_vp8_if;
> >  extern const struct vdec_common_if vdec_vp9_if;
> >  
> > diff --git a/include/linux/remoteproc/mtk_scp.h
> > b/include/linux/remoteproc/mtk_scp.h
> > index b47416f7aeb8..a32ffd736e7b 100644
> > --- a/include/linux/remoteproc/mtk_scp.h
> > +++ b/include/linux/remoteproc/mtk_scp.h
> > @@ -41,6 +41,8 @@ enum scp_ipi_id {
> >  	SCP_IPI_ISP_FRAME,
> >  	SCP_IPI_FD_CMD,
> >  	SCP_IPI_CROS_HOST_CMD,
> > +	SCP_IPI_VDEC_LAT,
> > +	SPC_IPI_VDEC_CORE,
> >  	SCP_IPI_NS_SERVICE = 0xFF,
> >  	SCP_IPI_MAX = 0x100,
> >  };
> 
>