[PATCH v3 00/16] mtk-vcodec: venc: support for MT8183 and v4l2-compliance fixes

[PATCH v3 00/16] mtk-vcodec: venc: support for MT8183 and v4l2-compliance fixes

[Alexandre Courbot]

This is the third version of the series adding support for the MT8183
encoder (1-9) and fixing v4l2-compliance tests (10-16).

Changes since v2:
* Carried new Acked-bys from mtk-vcodec maintainer,
* Removed 2 patches that addressed what was in fact a bug in
  v4l2-compliance,
* Moved DT bindings patches before the code implementing them as

Changes since v1:
* Checked that no regressions against v4l2-compliance were introduced,
* Fixed 9 failing v4l2-compliance tests,
* Removed 1 cleanup patch of contested utility,
* Carried Acked-bys and Reviewed-bys from mtk-vcodec maintainer.

Alexandre Courbot (12):
  dt-bindings: media: mtk-vcodec: document SCP node
  media: mtk-vcodec: venc: handle firmware version field
  media: mtk-vcodec: venc: specify bitrate range per-chip
  media: mtk-vcodec: venc: specify supported formats per-chip
  dt-bindings: media: document mediatek,mt8183-vcodec-enc
  Revert "media: mtk-vcodec: Remove extra area allocation in an input
    buffer on encoding"
  media: mtk-vcodec: venc support MIN_OUTPUT_BUFFERS control
  media: mtk-vcodec: venc: set OUTPUT buffers field to V4L2_FIELD_NONE
  media: mtk-vcodec: venc: use platform data for ENUM_FRAMESIZES
  media: mtk-vcodec: venc: support ENUM_FRAMESIZES on OUTPUT formats
  media: mtk-vcodec: venc: set default time per frame
  media: mtk-vcodec: venc: fix invalid time per frame in S_PARM

Yunfei Dong (4):
  media: mtk-vcodec: abstract firmware interface
  media: mtk-vcodec: add SCP firmware ops
  media: mtk-vcodec: venc: support SCP firmware
  media: mtk-vcodec: add support for MT8183 encoder

 .../bindings/media/mediatek-vcodec.txt        |   9 +-
 drivers/media/platform/Kconfig                |   1 +
 drivers/media/platform/mtk-vcodec/Makefile    |   4 +-
 .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  53 ++--
 .../platform/mtk-vcodec/mtk_vcodec_dec_pm.c   |   1 -
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  38 ++-
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 211 ++++++++--------
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 149 ++++++++----
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   |   2 -
 .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 228 ++++++++++++++++++
 .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  38 +++
 .../platform/mtk-vcodec/mtk_vcodec_util.c     |   1 -
 .../platform/mtk-vcodec/vdec/vdec_h264_if.c   |   1 -
 .../platform/mtk-vcodec/vdec/vdec_vp8_if.c    |   1 -
 .../platform/mtk-vcodec/vdec/vdec_vp9_if.c    |   1 -
 .../media/platform/mtk-vcodec/vdec_drv_base.h |   2 -
 .../media/platform/mtk-vcodec/vdec_drv_if.c   |   1 -
 .../media/platform/mtk-vcodec/vdec_vpu_if.c   |  12 +-
 .../media/platform/mtk-vcodec/vdec_vpu_if.h   |  11 +-
 .../platform/mtk-vcodec/venc/venc_h264_if.c   |  80 ++++--
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |  11 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   |   1 -
 .../media/platform/mtk-vcodec/venc_drv_if.h   |   6 +
 .../media/platform/mtk-vcodec/venc_ipi_msg.h  |  24 +-
 .../media/platform/mtk-vcodec/venc_vpu_if.c   | 141 ++++++++---
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |   8 +-
 26 files changed, 756 insertions(+), 279 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h

--
2.27.0.383.g050319c2ae-goog

[PATCH v3 01/16] media: mtk-vcodec: abstract firmware interface

[Alexandre Courbot]

From: Yunfei Dong <yunfei.dong@mediatek.com>

MT8183's codec firwmare is run by a different remote processor from
MT8173. While the firmware interface is basically the same, the way to
invoke it differs. Abstract all firmware calls under a layer that will
allow us to handle both firmware types transparently.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
[pihsun: fix error path and add mtk_vcodec_fw_release]
Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
Reviewed-by: Tiffany Lin <tiffany.lin@mediatek.com>
Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/Makefile    |   4 +-
 .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  50 ++---
 .../platform/mtk-vcodec/mtk_vcodec_dec_pm.c   |   1 -
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   5 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  47 ++---
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   |   2 -
 .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 172 ++++++++++++++++++
 .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  36 ++++
 .../platform/mtk-vcodec/mtk_vcodec_util.c     |   1 -
 .../platform/mtk-vcodec/vdec/vdec_h264_if.c   |   1 -
 .../platform/mtk-vcodec/vdec/vdec_vp8_if.c    |   1 -
 .../platform/mtk-vcodec/vdec/vdec_vp9_if.c    |   1 -
 .../media/platform/mtk-vcodec/vdec_drv_base.h |   2 -
 .../media/platform/mtk-vcodec/vdec_drv_if.c   |   1 -
 .../media/platform/mtk-vcodec/vdec_vpu_if.c   |  12 +-
 .../media/platform/mtk-vcodec/vdec_vpu_if.h   |  11 +-
 .../platform/mtk-vcodec/venc/venc_h264_if.c   |  15 +-
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   8 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   |   1 -
 .../media/platform/mtk-vcodec/venc_vpu_if.c   |  17 +-
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |   5 +-
 21 files changed, 290 insertions(+), 103 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h

diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
index 37b94b555fa1..b8636119ed0a 100644
--- a/drivers/media/platform/mtk-vcodec/Makefile
+++ b/drivers/media/platform/mtk-vcodec/Makefile
@@ -12,7 +12,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
 		vdec_vpu_if.o \
 		mtk_vcodec_dec.o \
 		mtk_vcodec_dec_pm.o \
-
+		mtk_vcodec_fw.o
 
 mtk-vcodec-enc-y := venc/venc_vp8_if.o \
 		venc/venc_h264_if.o \
@@ -25,5 +25,3 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
 
 mtk-vcodec-common-y := mtk_vcodec_intr.o \
 		mtk_vcodec_util.o\
-
-ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index 97a1b6664c20..4f07a5fcce7f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -20,7 +20,7 @@
 #include "mtk_vcodec_dec_pm.h"
 #include "mtk_vcodec_intr.h"
 #include "mtk_vcodec_util.h"
-#include "mtk_vpu.h"
+#include "mtk_vcodec_fw.h"
 
 #define VDEC_HW_ACTIVE	0x10
 #define VDEC_IRQ_CFG	0x11
@@ -77,22 +77,6 @@ static irqreturn_t mtk_vcodec_dec_irq_handler(int irq, void *priv)
 	return IRQ_HANDLED;
 }
 
-static void mtk_vcodec_dec_reset_handler(void *priv)
-{
-	struct mtk_vcodec_dev *dev = priv;
-	struct mtk_vcodec_ctx *ctx;
-
-	mtk_v4l2_err("Watchdog timeout!!");
-
-	mutex_lock(&dev->dev_mutex);
-	list_for_each_entry(ctx, &dev->ctx_list, list) {
-		ctx->state = MTK_STATE_ABORT;
-		mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR",
-				ctx->id);
-	}
-	mutex_unlock(&dev->dev_mutex);
-}
-
 static int fops_vcodec_open(struct file *file)
 {
 	struct mtk_vcodec_dev *dev = video_drvdata(file);
@@ -144,21 +128,20 @@ static int fops_vcodec_open(struct file *file)
 	if (v4l2_fh_is_singular(&ctx->fh)) {
 		mtk_vcodec_dec_pw_on(&dev->pm);
 		/*
-		 * vpu_load_firmware checks if it was loaded already and
-		 * does nothing in that case
+		 * Does nothing if firmware was already loaded.
 		 */
-		ret = vpu_load_firmware(dev->vpu_plat_dev);
+		ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
 		if (ret < 0) {
 			/*
 			 * Return 0 if downloading firmware successfully,
 			 * otherwise it is failed
 			 */
-			mtk_v4l2_err("vpu_load_firmware failed!");
+			mtk_v4l2_err("failed to load firmware!");
 			goto err_load_fw;
 		}
 
 		dev->dec_capability =
-			vpu_get_vdec_hw_capa(dev->vpu_plat_dev);
+			mtk_vcodec_fw_get_vdec_capa(dev->fw_handler);
 		mtk_v4l2_debug(0, "decoder capability %x", dev->dec_capability);
 	}
 
@@ -228,6 +211,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	struct mtk_vcodec_dev *dev;
 	struct video_device *vfd_dec;
 	struct resource *res;
+	phandle rproc_phandle;
+	enum mtk_vcodec_fw_type fw_type;
 	int i, ret;
 
 	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
@@ -237,19 +222,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	INIT_LIST_HEAD(&dev->ctx_list);
 	dev->plat_dev = pdev;
 
-	dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
-	if (dev->vpu_plat_dev == NULL) {
-		mtk_v4l2_err("[VPU] vpu device in not ready");
-		return -EPROBE_DEFER;
+	if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
+				  &rproc_phandle)) {
+		fw_type = VPU;
+	} else {
+		mtk_v4l2_err("Could not get vdec IPI device");
+		return -ENODEV;
 	}
-
-	vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_dec_reset_handler,
-			dev, VPU_RST_DEC);
+	dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_DEC);
+	if (IS_ERR(dev->fw_handler))
+		return PTR_ERR(dev->fw_handler);
 
 	ret = mtk_vcodec_init_dec_pm(dev);
 	if (ret < 0) {
 		dev_err(&pdev->dev, "Failed to get mt vcodec clock source");
-		return ret;
+		goto err_dec_pm;
 	}
 
 	for (i = 0; i < NUM_MAX_VDEC_REG_BASE; i++) {
@@ -352,6 +339,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	v4l2_device_unregister(&dev->v4l2_dev);
 err_res:
 	mtk_vcodec_release_dec_pm(dev);
+err_dec_pm:
+	mtk_vcodec_fw_release(dev->fw_handler);
 	return ret;
 }
 
@@ -376,6 +365,7 @@ static int mtk_vcodec_dec_remove(struct platform_device *pdev)
 
 	v4l2_device_unregister(&dev->v4l2_dev);
 	mtk_vcodec_release_dec_pm(dev);
+	mtk_vcodec_fw_release(dev->fw_handler);
 	return 0;
 }
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
index 5a6ec8fb52da..36dfe3fc056a 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
@@ -12,7 +12,6 @@
 
 #include "mtk_vcodec_dec_pm.h"
 #include "mtk_vcodec_util.h"
-#include "mtk_vpu.h"
 
 int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *mtkdev)
 {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 9fd56dee7fd1..e132c4ec463a 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -309,13 +309,13 @@ struct mtk_vcodec_ctx {
  * @m2m_dev_dec: m2m device for decoder
  * @m2m_dev_enc: m2m device for encoder.
  * @plat_dev: platform device
- * @vpu_plat_dev: mtk vpu platform device
  * @ctx_list: list of struct mtk_vcodec_ctx
  * @irqlock: protect data access by irq handler and work thread
  * @curr_ctx: The context that is waiting for codec hardware
  *
  * @reg_base: Mapped address of MTK Vcodec registers.
  *
+ * @fw_handler: used to communicate with the firmware.
  * @id_counter: used to identify current opened instance
  *
  * @encode_workqueue: encode work queue
@@ -344,12 +344,13 @@ struct mtk_vcodec_dev {
 	struct v4l2_m2m_dev *m2m_dev_dec;
 	struct v4l2_m2m_dev *m2m_dev_enc;
 	struct platform_device *plat_dev;
-	struct platform_device *vpu_plat_dev;
 	struct list_head ctx_list;
 	spinlock_t irqlock;
 	struct mtk_vcodec_ctx *curr_ctx;
 	void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
 
+	struct mtk_vcodec_fw *fw_handler;
+
 	unsigned long id_counter;
 
 	struct workqueue_struct *decode_workqueue;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 4d31f1ed113f..4340ea10afd0 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -21,7 +21,7 @@
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_intr.h"
 #include "mtk_vcodec_util.h"
-#include "mtk_vpu.h"
+#include "mtk_vcodec_fw.h"
 
 module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
 module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
@@ -101,22 +101,6 @@ static irqreturn_t mtk_vcodec_enc_lt_irq_handler(int irq, void *priv)
 	return IRQ_HANDLED;
 }
 
-static void mtk_vcodec_enc_reset_handler(void *priv)
-{
-	struct mtk_vcodec_dev *dev = priv;
-	struct mtk_vcodec_ctx *ctx;
-
-	mtk_v4l2_debug(0, "Watchdog timeout!!");
-
-	mutex_lock(&dev->dev_mutex);
-	list_for_each_entry(ctx, &dev->ctx_list, list) {
-		ctx->state = MTK_STATE_ABORT;
-		mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
-				ctx->id);
-	}
-	mutex_unlock(&dev->dev_mutex);
-}
-
 static int fops_vcodec_open(struct file *file)
 {
 	struct mtk_vcodec_dev *dev = video_drvdata(file);
@@ -159,10 +143,10 @@ static int fops_vcodec_open(struct file *file)
 
 	if (v4l2_fh_is_singular(&ctx->fh)) {
 		/*
-		 * vpu_load_firmware checks if it was loaded already and
+		 * load fireware to checks if it was loaded already and
 		 * does nothing in that case
 		 */
-		ret = vpu_load_firmware(dev->vpu_plat_dev);
+		ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
 		if (ret < 0) {
 			/*
 			 * Return 0 if downloading firmware successfully,
@@ -173,7 +157,7 @@ static int fops_vcodec_open(struct file *file)
 		}
 
 		dev->enc_capability =
-			vpu_get_venc_hw_capa(dev->vpu_plat_dev);
+			mtk_vcodec_fw_get_venc_capa(dev->fw_handler);
 		mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
 	}
 
@@ -235,6 +219,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	struct mtk_vcodec_dev *dev;
 	struct video_device *vfd_enc;
 	struct resource *res;
+	phandle rproc_phandle;
+	enum mtk_vcodec_fw_type fw_type;
 	int i, j, ret;
 
 	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
@@ -244,19 +230,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	INIT_LIST_HEAD(&dev->ctx_list);
 	dev->plat_dev = pdev;
 
-	dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
-	if (dev->vpu_plat_dev == NULL) {
-		mtk_v4l2_err("[VPU] vpu device in not ready");
-		return -EPROBE_DEFER;
+	if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
+				  &rproc_phandle)) {
+		fw_type = VPU;
+	} else {
+		mtk_v4l2_err("Could not get venc IPI device");
+		return -ENODEV;
 	}
-
-	vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler,
-				dev, VPU_RST_ENC);
+	dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_ENC);
+	if (IS_ERR(dev->fw_handler))
+		return PTR_ERR(dev->fw_handler);
 
 	ret = mtk_vcodec_init_enc_pm(dev);
 	if (ret < 0) {
 		dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
-		return ret;
+		goto err_enc_pm;
 	}
 
 	for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
@@ -377,6 +365,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	v4l2_device_unregister(&dev->v4l2_dev);
 err_res:
 	mtk_vcodec_release_enc_pm(dev);
+err_enc_pm:
+	mtk_vcodec_fw_release(dev->fw_handler);
 	return ret;
 }
 
@@ -401,6 +391,7 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
 
 	v4l2_device_unregister(&dev->v4l2_dev);
 	mtk_vcodec_release_enc_pm(dev);
+	mtk_vcodec_fw_release(dev->fw_handler);
 	return 0;
 }
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
index 3e2bfded79a6..ee22902aaa71 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
@@ -12,8 +12,6 @@
 
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
-#include "mtk_vpu.h"
-
 
 int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
 {
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
new file mode 100644
index 000000000000..967bb100a990
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
@@ -0,0 +1,172 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include "mtk_vcodec_fw.h"
+#include "mtk_vcodec_util.h"
+#include "mtk_vcodec_drv.h"
+
+struct mtk_vcodec_fw_ops {
+	int (*load_firmware)(struct mtk_vcodec_fw *fw);
+	unsigned int (*get_vdec_capa)(struct mtk_vcodec_fw *fw);
+	unsigned int (*get_venc_capa)(struct mtk_vcodec_fw *fw);
+	void * (*map_dm_addr)(struct mtk_vcodec_fw *fw, u32 dtcm_dmem_addr);
+	int (*ipi_register)(struct mtk_vcodec_fw *fw, int id,
+		mtk_vcodec_ipi_handler handler, const char *name, void *priv);
+	int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
+		unsigned int len, unsigned int wait);
+};
+
+struct mtk_vcodec_fw {
+	enum mtk_vcodec_fw_type type;
+	const struct mtk_vcodec_fw_ops *ops;
+	struct platform_device *pdev;
+};
+
+static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
+{
+	return vpu_load_firmware(fw->pdev);
+}
+
+static unsigned int mtk_vcodec_vpu_get_vdec_capa(struct mtk_vcodec_fw *fw)
+{
+	return vpu_get_vdec_hw_capa(fw->pdev);
+}
+
+static unsigned int mtk_vcodec_vpu_get_venc_capa(struct mtk_vcodec_fw *fw)
+{
+	return vpu_get_venc_hw_capa(fw->pdev);
+}
+
+static void *mtk_vcodec_vpu_map_dm_addr(struct mtk_vcodec_fw *fw,
+					u32 dtcm_dmem_addr)
+{
+	return vpu_mapping_dm_addr(fw->pdev, dtcm_dmem_addr);
+}
+
+static int mtk_vcodec_vpu_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
+		mtk_vcodec_ipi_handler handler, const char *name, void *priv)
+{
+	/*
+	 * The handler we receive takes a void * as its first argument. We
+	 * cannot change this because it needs to be passed down to the rproc
+	 * subsystem when SCP is used. VPU takes a const argument, which is
+	 * more constrained, so the conversion below is safe.
+	 */
+	ipi_handler_t handler_const = (ipi_handler_t)handler;
+
+	return vpu_ipi_register(fw->pdev, id, handler_const, name, priv);
+}
+
+static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
+		unsigned int len, unsigned int wait)
+{
+	return vpu_ipi_send(fw->pdev, id, buf, len);
+}
+
+static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
+	.load_firmware = mtk_vcodec_vpu_load_firmware,
+	.get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
+	.get_venc_capa = mtk_vcodec_vpu_get_venc_capa,
+	.map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
+	.ipi_register = mtk_vcodec_vpu_set_ipi_register,
+	.ipi_send = mtk_vcodec_vpu_ipi_send,
+};
+
+static void mtk_vcodec_reset_handler(void *priv)
+{
+	struct mtk_vcodec_dev *dev = priv;
+	struct mtk_vcodec_ctx *ctx;
+
+	mtk_v4l2_err("Watchdog timeout!!");
+
+	mutex_lock(&dev->dev_mutex);
+	list_for_each_entry(ctx, &dev->ctx_list, list) {
+		ctx->state = MTK_STATE_ABORT;
+		mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
+				ctx->id);
+	}
+	mutex_unlock(&dev->dev_mutex);
+}
+
+struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
+					   enum mtk_vcodec_fw_type type,
+					   enum rst_id rst_id)
+{
+	const struct mtk_vcodec_fw_ops *ops;
+	struct mtk_vcodec_fw *fw;
+	struct platform_device *fw_pdev = NULL;
+
+	switch (type) {
+	case VPU:
+		ops = &mtk_vcodec_vpu_msg;
+		fw_pdev = vpu_get_plat_device(dev->plat_dev);
+		if (!fw_pdev) {
+			mtk_v4l2_err("firmware device is not ready");
+			return ERR_PTR(-EINVAL);
+		}
+		vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
+				    dev, rst_id);
+		break;
+	default:
+		mtk_v4l2_err("invalid vcodec fw type");
+		return ERR_PTR(-EINVAL);
+	}
+
+	fw = devm_kzalloc(&dev->plat_dev->dev, sizeof(*fw), GFP_KERNEL);
+	if (!fw)
+		return ERR_PTR(-EINVAL);
+
+	fw->type = type;
+	fw->ops = ops;
+	fw->pdev = fw_pdev;
+
+	return fw;
+}
+EXPORT_SYMBOL_GPL(mtk_vcodec_fw_select);
+
+void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
+{
+	switch (fw->type) {
+	case VPU:
+		put_device(&fw->pdev->dev);
+		break;
+	}
+}
+EXPORT_SYMBOL_GPL(mtk_vcodec_fw_release);
+
+int mtk_vcodec_fw_load_firmware(struct mtk_vcodec_fw *fw)
+{
+	return fw->ops->load_firmware(fw);
+}
+EXPORT_SYMBOL_GPL(mtk_vcodec_fw_load_firmware);
+
+unsigned int mtk_vcodec_fw_get_vdec_capa(struct mtk_vcodec_fw *fw)
+{
+	return fw->ops->get_vdec_capa(fw);
+}
+EXPORT_SYMBOL_GPL(mtk_vcodec_fw_get_vdec_capa);
+
+unsigned int mtk_vcodec_fw_get_venc_capa(struct mtk_vcodec_fw *fw)
+{
+	return fw->ops->get_venc_capa(fw);
+}
+EXPORT_SYMBOL_GPL(mtk_vcodec_fw_get_venc_capa);
+
+void *mtk_vcodec_fw_map_dm_addr(struct mtk_vcodec_fw *fw, u32 mem_addr)
+{
+	return fw->ops->map_dm_addr(fw, mem_addr);
+}
+EXPORT_SYMBOL_GPL(mtk_vcodec_fw_map_dm_addr);
+
+int mtk_vcodec_fw_ipi_register(struct mtk_vcodec_fw *fw, int id,
+	mtk_vcodec_ipi_handler handler, const char *name, void *priv)
+{
+	return fw->ops->ipi_register(fw, id, handler, name, priv);
+}
+EXPORT_SYMBOL_GPL(mtk_vcodec_fw_ipi_register);
+
+int mtk_vcodec_fw_ipi_send(struct mtk_vcodec_fw *fw,
+	int id, void *buf, unsigned int len, unsigned int wait)
+{
+	return fw->ops->ipi_send(fw, id, buf, len, wait);
+}
+EXPORT_SYMBOL_GPL(mtk_vcodec_fw_ipi_send);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
new file mode 100644
index 000000000000..ff25b0c19f74
--- /dev/null
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef _MTK_VCODEC_FW_H_
+#define _MTK_VCODEC_FW_H_
+
+#include <linux/remoteproc.h>
+
+#include "../mtk-vpu/mtk_vpu.h"
+
+struct mtk_vcodec_dev;
+
+enum mtk_vcodec_fw_type {
+	VPU,
+};
+
+struct mtk_vcodec_fw;
+
+typedef void (*mtk_vcodec_ipi_handler) (void *data,
+	unsigned int len, void *priv);
+
+struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
+					   enum mtk_vcodec_fw_type type,
+					   enum rst_id rst_id);
+void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw);
+
+int mtk_vcodec_fw_load_firmware(struct mtk_vcodec_fw *fw);
+unsigned int mtk_vcodec_fw_get_vdec_capa(struct mtk_vcodec_fw *fw);
+unsigned int mtk_vcodec_fw_get_venc_capa(struct mtk_vcodec_fw *fw);
+void *mtk_vcodec_fw_map_dm_addr(struct mtk_vcodec_fw *fw, u32 mem_addr);
+int mtk_vcodec_fw_ipi_register(struct mtk_vcodec_fw *fw, int id,
+	mtk_vcodec_ipi_handler handler, const char *name, void *priv);
+int mtk_vcodec_fw_ipi_send(struct mtk_vcodec_fw *fw,
+	int id, void *buf, unsigned int len, unsigned int wait);
+
+#endif /* _MTK_VCODEC_FW_H_ */
+
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
index d48f542db1a9..ac5973b6735f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
@@ -9,7 +9,6 @@
 
 #include "mtk_vcodec_drv.h"
 #include "mtk_vcodec_util.h"
-#include "mtk_vpu.h"
 
 /* For encoder, this will enable logs in venc/*/
 bool mtk_vcodec_dbg;
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
index 50048c170b99..40d6e6c5ac7a 100644
--- a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
@@ -281,7 +281,6 @@ static int vdec_h264_init(struct mtk_vcodec_ctx *ctx)
 	inst->ctx = ctx;
 
 	inst->vpu.id = IPI_VDEC_H264;
-	inst->vpu.dev = ctx->dev->vpu_plat_dev;
 	inst->vpu.ctx = ctx;
 
 	err = vpu_dec_init(&inst->vpu);
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c
index 6011fdd60a22..e5393f841080 100644
--- a/drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c
@@ -400,7 +400,6 @@ static int vdec_vp8_init(struct mtk_vcodec_ctx *ctx)
 	inst->ctx = ctx;
 
 	inst->vpu.id = IPI_VDEC_VP8;
-	inst->vpu.dev = ctx->dev->vpu_plat_dev;
 	inst->vpu.ctx = ctx;
 
 	err = vpu_dec_init(&inst->vpu);
diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c
index 257a5b5ad212..8e099b859f21 100644
--- a/drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c
@@ -795,7 +795,6 @@ static int vdec_vp9_init(struct mtk_vcodec_ctx *ctx)
 	inst->ctx = ctx;
 
 	inst->vpu.id = IPI_VDEC_VP9;
-	inst->vpu.dev = ctx->dev->vpu_plat_dev;
 	inst->vpu.ctx = ctx;
 
 	if (vpu_dec_init(&inst->vpu)) {
diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_base.h b/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
index ceb4db4cb3be..e913f963b7db 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
+++ b/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
@@ -7,8 +7,6 @@
 #ifndef _VDEC_DRV_BASE_
 #define _VDEC_DRV_BASE_
 
-#include "mtk_vcodec_drv.h"
-
 #include "vdec_drv_if.h"
 
 struct vdec_common_if {
diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
index 2e43dd4486e0..b18743b906ea 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
@@ -13,7 +13,6 @@
 #include "mtk_vcodec_dec.h"
 #include "vdec_drv_base.h"
 #include "mtk_vcodec_dec_pm.h"
-#include "mtk_vpu.h"
 
 int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
index 948a12fd9d46..58b0e6fa8fd2 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
@@ -8,6 +8,7 @@
 #include "mtk_vcodec_util.h"
 #include "vdec_ipi_msg.h"
 #include "vdec_vpu_if.h"
+#include "mtk_vcodec_fw.h"
 
 static void handle_init_ack_msg(const struct vdec_vpu_ipi_init_ack *msg)
 {
@@ -18,7 +19,8 @@ static void handle_init_ack_msg(const struct vdec_vpu_ipi_init_ack *msg)
 
 	/* mapping VPU address to kernel virtual address */
 	/* the content in vsi is initialized to 0 in VPU */
-	vpu->vsi = vpu_mapping_dm_addr(vpu->dev, msg->vpu_inst_addr);
+	vpu->vsi = mtk_vcodec_fw_map_dm_addr(vpu->ctx->dev->fw_handler,
+					     msg->vpu_inst_addr);
 	vpu->inst_addr = msg->vpu_inst_addr;
 
 	mtk_vcodec_debug(vpu, "- vpu_inst_addr = 0x%x", vpu->inst_addr);
@@ -34,7 +36,7 @@ static void handle_init_ack_msg(const struct vdec_vpu_ipi_init_ack *msg)
  * This function runs in interrupt context and it means there's an IPI MSG
  * from VPU.
  */
-static void vpu_dec_ipi_handler(const void *data, unsigned int len, void *priv)
+static void vpu_dec_ipi_handler(void *data, unsigned int len, void *priv)
 {
 	const struct vdec_vpu_ipi_ack *msg = data;
 	struct vdec_vpu_inst *vpu = (struct vdec_vpu_inst *)
@@ -74,7 +76,8 @@ static int vcodec_vpu_send_msg(struct vdec_vpu_inst *vpu, void *msg, int len)
 	vpu->failure = 0;
 	vpu->signaled = 0;
 
-	err = vpu_ipi_send(vpu->dev, vpu->id, msg, len);
+	err = mtk_vcodec_fw_ipi_send(vpu->ctx->dev->fw_handler, vpu->id, msg,
+				     len, 2000);
 	if (err) {
 		mtk_vcodec_err(vpu, "send fail vpu_id=%d msg_id=%X status=%d",
 			       vpu->id, *(uint32_t *)msg, err);
@@ -110,7 +113,8 @@ int vpu_dec_init(struct vdec_vpu_inst *vpu)
 	init_waitqueue_head(&vpu->wq);
 	vpu->handler = vpu_dec_ipi_handler;
 
-	err = vpu_ipi_register(vpu->dev, vpu->id, vpu->handler, "vdec", NULL);
+	err = mtk_vcodec_fw_ipi_register(vpu->ctx->dev->fw_handler, vpu->id,
+					 vpu->handler, "vdec", NULL);
 	if (err != 0) {
 		mtk_vcodec_err(vpu, "vpu_ipi_register fail status=%d", err);
 		return err;
diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
index f779b0676fbd..85224eb7e34b 100644
--- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
+++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
@@ -7,11 +7,13 @@
 #ifndef _VDEC_VPU_IF_H_
 #define _VDEC_VPU_IF_H_
 
-#include "mtk_vpu.h"
+#include "mtk_vcodec_fw.h"
+
+struct mtk_vcodec_ctx;
 
 /**
  * struct vdec_vpu_inst - VPU instance for video codec
- * @ipi_id      : ipi id for each decoder
+ * @id          : ipi msg id for each decoder
  * @vsi         : driver structure allocated by VPU side and shared to AP side
  *                for control and info share
  * @failure     : VPU execution result status, 0: success, others: fail
@@ -23,15 +25,14 @@
  * @handler     : ipi handler for each decoder
  */
 struct vdec_vpu_inst {
-	enum ipi_id id;
+	int id;
 	void *vsi;
 	int32_t failure;
 	uint32_t inst_addr;
 	unsigned int signaled;
 	struct mtk_vcodec_ctx *ctx;
-	struct platform_device *dev;
 	wait_queue_head_t wq;
-	ipi_handler_t handler;
+	mtk_vcodec_ipi_handler handler;
 };
 
 /**
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
index b9624f8df0e9..7a00f050ec36 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
@@ -18,7 +18,6 @@
 #include "../venc_drv_base.h"
 #include "../venc_ipi_msg.h"
 #include "../venc_vpu_if.h"
-#include "mtk_vpu.h"
 
 static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
 
@@ -257,8 +256,11 @@ static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst)
 		 */
 		inst->work_bufs[i].size = wb[i].size;
 		if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
-			inst->work_bufs[i].va = vpu_mapping_dm_addr(
-				inst->vpu_inst.dev, wb[i].vpua);
+			struct mtk_vcodec_fw *handler;
+
+			handler = inst->vpu_inst.ctx->dev->fw_handler;
+			inst->work_bufs[i].va =
+				mtk_vcodec_fw_map_dm_addr(handler, wb[i].vpua);
 			inst->work_bufs[i].dma_addr = 0;
 		} else {
 			ret = mtk_vcodec_mem_alloc(inst->ctx,
@@ -275,10 +277,12 @@ static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst)
 			 * setting in VPU side.
 			 */
 			if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
+				struct mtk_vcodec_fw *handler;
 				void *tmp_va;
 
-				tmp_va = vpu_mapping_dm_addr(inst->vpu_inst.dev,
-							     wb[i].vpua);
+				handler = inst->vpu_inst.ctx->dev->fw_handler;
+				tmp_va = mtk_vcodec_fw_map_dm_addr(handler,
+								   wb[i].vpua);
 				memcpy(inst->work_bufs[i].va, tmp_va,
 				       wb[i].size);
 			}
@@ -469,7 +473,6 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
 
 	inst->ctx = ctx;
 	inst->vpu_inst.ctx = ctx;
-	inst->vpu_inst.dev = ctx->dev->vpu_plat_dev;
 	inst->vpu_inst.id = IPI_VENC_H264;
 	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
 
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
index 8d36f0362efe..6426af514526 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
@@ -17,7 +17,6 @@
 #include "../venc_drv_base.h"
 #include "../venc_ipi_msg.h"
 #include "../venc_vpu_if.h"
-#include "mtk_vpu.h"
 
 #define VENC_BITSTREAM_FRAME_SIZE 0x0098
 #define VENC_BITSTREAM_HEADER_LEN 0x00e8
@@ -190,10 +189,12 @@ static int vp8_enc_alloc_work_buf(struct venc_vp8_inst *inst)
 		if (i == VENC_VP8_VPU_WORK_BUF_RC_CODE ||
 		    i == VENC_VP8_VPU_WORK_BUF_RC_CODE2 ||
 		    i == VENC_VP8_VPU_WORK_BUF_RC_CODE3) {
+			struct mtk_vcodec_fw *handler;
 			void *tmp_va;
 
-			tmp_va = vpu_mapping_dm_addr(inst->vpu_inst.dev,
-						     wb[i].vpua);
+			handler = inst->vpu_inst.ctx->dev->fw_handler;
+			tmp_va = mtk_vcodec_fw_map_dm_addr(handler,
+							   wb[i].vpua);
 			memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
 		}
 		wb[i].iova = inst->work_bufs[i].dma_addr;
@@ -334,7 +335,6 @@ static int vp8_enc_init(struct mtk_vcodec_ctx *ctx)
 
 	inst->ctx = ctx;
 	inst->vpu_inst.ctx = ctx;
-	inst->vpu_inst.dev = ctx->dev->vpu_plat_dev;
 	inst->vpu_inst.id = IPI_VENC_VP8;
 	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_LT_SYS);
 
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
index c6bb82ac2dcd..ce0bce811615 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
@@ -15,7 +15,6 @@
 
 #include "mtk_vcodec_enc.h"
 #include "mtk_vcodec_enc_pm.h"
-#include "mtk_vpu.h"
 
 int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 {
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
index 9540709c1905..53854127814b 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
@@ -4,7 +4,7 @@
  * Author: PoChun Lin <pochun.lin@mediatek.com>
  */
 
-#include "mtk_vpu.h"
+#include "mtk_vcodec_fw.h"
 #include "venc_ipi_msg.h"
 #include "venc_vpu_if.h"
 
@@ -13,7 +13,8 @@ static void handle_enc_init_msg(struct venc_vpu_inst *vpu, const void *data)
 	const struct venc_vpu_ipi_msg_init *msg = data;
 
 	vpu->inst_addr = msg->vpu_inst_addr;
-	vpu->vsi = vpu_mapping_dm_addr(vpu->dev, msg->vpu_inst_addr);
+	vpu->vsi = mtk_vcodec_fw_map_dm_addr(vpu->ctx->dev->fw_handler,
+					     msg->vpu_inst_addr);
 }
 
 static void handle_enc_encode_msg(struct venc_vpu_inst *vpu, const void *data)
@@ -25,7 +26,7 @@ static void handle_enc_encode_msg(struct venc_vpu_inst *vpu, const void *data)
 	vpu->is_key_frm = msg->is_key_frm;
 }
 
-static void vpu_enc_ipi_handler(const void *data, unsigned int len, void *priv)
+static void vpu_enc_ipi_handler(void *data, unsigned int len, void *priv)
 {
 	const struct venc_vpu_ipi_msg_common *msg = data;
 	struct venc_vpu_inst *vpu =
@@ -63,12 +64,13 @@ static int vpu_enc_send_msg(struct venc_vpu_inst *vpu, void *msg,
 
 	mtk_vcodec_debug_enter(vpu);
 
-	if (!vpu->dev) {
+	if (!vpu->ctx->dev->fw_handler) {
 		mtk_vcodec_err(vpu, "inst dev is NULL");
 		return -EINVAL;
 	}
 
-	status = vpu_ipi_send(vpu->dev, vpu->id, msg, len);
+	status = mtk_vcodec_fw_ipi_send(vpu->ctx->dev->fw_handler, vpu->id, msg,
+					len, 2000);
 	if (status) {
 		mtk_vcodec_err(vpu, "vpu_ipi_send msg_id %x len %d fail %d",
 			       *(uint32_t *)msg, len, status);
@@ -93,8 +95,9 @@ int vpu_enc_init(struct venc_vpu_inst *vpu)
 	vpu->signaled = 0;
 	vpu->failure = 0;
 
-	status = vpu_ipi_register(vpu->dev, vpu->id, vpu_enc_ipi_handler,
-				  NULL, NULL);
+	status = mtk_vcodec_fw_ipi_register(vpu->ctx->dev->fw_handler, vpu->id,
+		vpu_enc_ipi_handler, "venc", NULL);
+
 	if (status) {
 		mtk_vcodec_err(vpu, "vpu_ipi_register fail %d", status);
 		return -EINVAL;
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
index ba301a138a5a..edd411621b68 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
@@ -7,7 +7,7 @@
 #ifndef _VENC_VPU_IF_H_
 #define _VENC_VPU_IF_H_
 
-#include "mtk_vpu.h"
+#include "mtk_vcodec_fw.h"
 #include "venc_drv_if.h"
 
 /*
@@ -34,9 +34,8 @@ struct venc_vpu_inst {
 	int is_key_frm;
 	unsigned int inst_addr;
 	void *vsi;
-	enum ipi_id id;
+	int id;
 	struct mtk_vcodec_ctx *ctx;
-	struct platform_device *dev;
 };
 
 int vpu_enc_init(struct venc_vpu_inst *vpu);
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 02/16] dt-bindings: media: mtk-vcodec: document SCP node

[Alexandre Courbot]

The mediatek codecs can use either the VPU or the SCP as their interface
to firmware. Reflect this in the DT bindings.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
index b6b5dde6abd8..7aef0a4fe207 100644
--- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -19,7 +19,9 @@ Required properties:
 - iommus : should point to the respective IOMMU block with master port as
   argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
   for details.
-- mediatek,vpu : the node of video processor unit
+One of the two following nodes:
+- mediatek,vpu : the node of the video processor unit, if using VPU.
+- mediatek,scp : the noode of the SCP unit, if using SCP.
 
 
 Example:
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 03/16] media: mtk-vcodec: add SCP firmware ops

[Alexandre Courbot]

From: Yunfei Dong <yunfei.dong@mediatek.com>

Add support for communicating with the SCP firmware, which will be used
by MT8183.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/Kconfig                |  1 +
 .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  3 +
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  3 +
 .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 56 +++++++++++++++++++
 .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  2 +
 5 files changed, 65 insertions(+)

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index c57ee78fa99d..f0dbe048efea 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -256,6 +256,7 @@ config VIDEO_MEDIATEK_VCODEC
 	select VIDEOBUF2_DMA_CONTIG
 	select V4L2_MEM2MEM_DEV
 	select VIDEO_MEDIATEK_VPU
+	select MTK_SCP
 	help
 	    Mediatek video codec driver provides HW capability to
 	    encode and decode in a range of video formats
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
index 4f07a5fcce7f..5b5765b98e57 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
@@ -225,6 +225,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
 				  &rproc_phandle)) {
 		fw_type = VPU;
+	} else if (!of_property_read_u32(pdev->dev.of_node, "mediatek,scp",
+					 &rproc_phandle)) {
+		fw_type = SCP;
 	} else {
 		mtk_v4l2_err("Could not get vdec IPI device");
 		return -ENODEV;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 4340ea10afd0..42530cd01a30 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -233,6 +233,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
 				  &rproc_phandle)) {
 		fw_type = VPU;
+	} else if (!of_property_read_u32(pdev->dev.of_node, "mediatek,scp",
+					 &rproc_phandle)) {
+		fw_type = SCP;
 	} else {
 		mtk_v4l2_err("Could not get venc IPI device");
 		return -ENODEV;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
index 967bb100a990..f2a62ea62fc6 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
@@ -19,6 +19,7 @@ struct mtk_vcodec_fw {
 	enum mtk_vcodec_fw_type type;
 	const struct mtk_vcodec_fw_ops *ops;
 	struct platform_device *pdev;
+	struct mtk_scp *scp;
 };
 
 static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
@@ -71,6 +72,48 @@ static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
 	.ipi_send = mtk_vcodec_vpu_ipi_send,
 };
 
+static int mtk_vcodec_scp_load_firmware(struct mtk_vcodec_fw *fw)
+{
+	return rproc_boot(scp_get_rproc(fw->scp));
+}
+
+static unsigned int mtk_vcodec_scp_get_vdec_capa(struct mtk_vcodec_fw *fw)
+{
+	return scp_get_vdec_hw_capa(fw->scp);
+}
+
+static unsigned int mtk_vcodec_scp_get_venc_capa(struct mtk_vcodec_fw *fw)
+{
+	return scp_get_venc_hw_capa(fw->scp);
+}
+
+static void *mtk_vcodec_vpu_scp_dm_addr(struct mtk_vcodec_fw *fw,
+					u32 dtcm_dmem_addr)
+{
+	return scp_mapping_dm_addr(fw->scp, dtcm_dmem_addr);
+}
+
+static int mtk_vcodec_scp_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
+		mtk_vcodec_ipi_handler handler, const char *name, void *priv)
+{
+	return scp_ipi_register(fw->scp, id, handler, priv);
+}
+
+static int mtk_vcodec_scp_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
+		unsigned int len, unsigned int wait)
+{
+	return scp_ipi_send(fw->scp, id, buf, len, wait);
+}
+
+static const struct mtk_vcodec_fw_ops mtk_vcodec_rproc_msg = {
+	.load_firmware = mtk_vcodec_scp_load_firmware,
+	.get_vdec_capa = mtk_vcodec_scp_get_vdec_capa,
+	.get_venc_capa = mtk_vcodec_scp_get_venc_capa,
+	.map_dm_addr = mtk_vcodec_vpu_scp_dm_addr,
+	.ipi_register = mtk_vcodec_scp_set_ipi_register,
+	.ipi_send = mtk_vcodec_scp_ipi_send,
+};
+
 static void mtk_vcodec_reset_handler(void *priv)
 {
 	struct mtk_vcodec_dev *dev = priv;
@@ -94,6 +137,7 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
 	const struct mtk_vcodec_fw_ops *ops;
 	struct mtk_vcodec_fw *fw;
 	struct platform_device *fw_pdev = NULL;
+	struct mtk_scp *scp = NULL;
 
 	switch (type) {
 	case VPU:
@@ -106,6 +150,14 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
 		vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
 				    dev, rst_id);
 		break;
+	case SCP:
+		ops = &mtk_vcodec_rproc_msg;
+		scp = scp_get(dev->plat_dev);
+		if (!scp) {
+			mtk_v4l2_err("could not get vdec scp handle");
+			return ERR_PTR(-EPROBE_DEFER);
+		}
+		break;
 	default:
 		mtk_v4l2_err("invalid vcodec fw type");
 		return ERR_PTR(-EINVAL);
@@ -118,6 +170,7 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
 	fw->type = type;
 	fw->ops = ops;
 	fw->pdev = fw_pdev;
+	fw->scp = scp;
 
 	return fw;
 }
@@ -129,6 +182,9 @@ void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
 	case VPU:
 		put_device(&fw->pdev->dev);
 		break;
+	case SCP:
+		scp_put(fw->scp);
+		break;
 	}
 }
 EXPORT_SYMBOL_GPL(mtk_vcodec_fw_release);
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
index ff25b0c19f74..ced1b6a10e07 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
@@ -4,6 +4,7 @@
 #define _MTK_VCODEC_FW_H_
 
 #include <linux/remoteproc.h>
+#include <linux/remoteproc/mtk_scp.h>
 
 #include "../mtk-vpu/mtk_vpu.h"
 
@@ -11,6 +12,7 @@ struct mtk_vcodec_dev;
 
 enum mtk_vcodec_fw_type {
 	VPU,
+	SCP,
 };
 
 struct mtk_vcodec_fw;
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 04/16] media: mtk-vcodec: venc: support SCP firmware

[Alexandre Courbot]

From: Yunfei Dong <yunfei.dong@mediatek.com>

Support the new extended firmware used by MT8183's encoder.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      | 12 +++
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 34 ++++---
 .../platform/mtk-vcodec/venc/venc_h264_if.c   | 65 +++++++++++--
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |  3 +-
 .../media/platform/mtk-vcodec/venc_drv_if.h   |  6 ++
 .../media/platform/mtk-vcodec/venc_ipi_msg.h  | 15 ++-
 .../media/platform/mtk-vcodec/venc_vpu_if.c   | 97 +++++++++++++------
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |  3 +-
 8 files changed, 181 insertions(+), 54 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index e132c4ec463a..45c8adfc6a0c 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -300,6 +300,17 @@ struct mtk_vcodec_ctx {
 
 };
 
+/**
+ * struct mtk_vcodec_enc_pdata - compatible data for each IC
+ *
+ * @uses_ext: whether the encoder uses the extended firmware messaging format
+ * @has_lt_irq: whether the encoder uses the LT irq
+ */
+struct mtk_vcodec_enc_pdata {
+	bool uses_ext;
+	bool has_lt_irq;
+};
+
 /**
  * struct mtk_vcodec_dev - driver data
  * @v4l2_dev: V4L2 device to register video devices for.
@@ -348,6 +359,7 @@ struct mtk_vcodec_dev {
 	spinlock_t irqlock;
 	struct mtk_vcodec_ctx *curr_ctx;
 	void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
+	const struct mtk_vcodec_enc_pdata *venc_pdata;
 
 	struct mtk_vcodec_fw *fw_handler;
 
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 42530cd01a30..922bc8883811 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -244,6 +244,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	if (IS_ERR(dev->fw_handler))
 		return PTR_ERR(dev->fw_handler);
 
+	dev->venc_pdata = of_device_get_match_data(&pdev->dev);
 	ret = mtk_vcodec_init_enc_pm(dev);
 	if (ret < 0) {
 		dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
@@ -278,21 +279,24 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		ret = -EINVAL;
 		goto err_res;
 	}
+	disable_irq(dev->enc_irq);
 
-	dev->enc_lt_irq = platform_get_irq(pdev, 1);
-	ret = devm_request_irq(&pdev->dev,
-			       dev->enc_lt_irq, mtk_vcodec_enc_lt_irq_handler,
-			       0, pdev->name, dev);
-	if (ret) {
-		dev_err(&pdev->dev,
-			"Failed to install dev->enc_lt_irq %d (%d)",
-			dev->enc_lt_irq, ret);
-		ret = -EINVAL;
-		goto err_res;
+	if (dev->venc_pdata->has_lt_irq) {
+		dev->enc_lt_irq = platform_get_irq(pdev, 1);
+		ret = devm_request_irq(&pdev->dev,
+				       dev->enc_lt_irq,
+				       mtk_vcodec_enc_lt_irq_handler,
+				       0, pdev->name, dev);
+		if (ret) {
+			dev_err(&pdev->dev,
+				"Failed to install dev->enc_lt_irq %d (%d)",
+				dev->enc_lt_irq, ret);
+			ret = -EINVAL;
+			goto err_res;
+		}
+		disable_irq(dev->enc_lt_irq); /* VENC_LT */
 	}
 
-	disable_irq(dev->enc_irq);
-	disable_irq(dev->enc_lt_irq); /* VENC_LT */
 	mutex_init(&dev->enc_mutex);
 	mutex_init(&dev->dev_mutex);
 	spin_lock_init(&dev->irqlock);
@@ -373,8 +377,12 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	return ret;
 }
 
+static const struct mtk_vcodec_enc_pdata mt8173_pdata = {
+	.has_lt_irq = true,
+};
+
 static const struct of_device_id mtk_vcodec_enc_match[] = {
-	{.compatible = "mediatek,mt8173-vcodec-enc",},
+	{.compatible = "mediatek,mt8173-vcodec-enc", .data = &mt8173_pdata},
 	{},
 };
 MODULE_DEVICE_TABLE(of, mtk_vcodec_enc_match);
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
index 7a00f050ec36..050787b2896c 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
@@ -24,6 +24,16 @@ static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
 #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
 #define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
 
+/*
+ * enum venc_h264_frame_type - h264 encoder output bitstream frame type
+ */
+enum venc_h264_frame_type {
+	VENC_H264_IDR_FRM,
+	VENC_H264_I_FRM,
+	VENC_H264_P_FRM,
+	VENC_H264_B_FRM,
+};
+
 /*
  * enum venc_h264_vpu_work_buf - h264 encoder buffer index
  */
@@ -137,7 +147,8 @@ struct venc_h264_inst {
 	struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
 	struct mtk_vcodec_mem pps_buf;
 	bool work_buf_allocated;
-	unsigned int frm_cnt;
+	u32 frm_cnt;	/* declared as u32 to properly overflow */
+	unsigned int skip_frm_cnt;
 	unsigned int prepend_hdr;
 	struct venc_vpu_inst vpu_inst;
 	struct venc_h264_vsi *vsi;
@@ -327,6 +338,22 @@ static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
 	return irq_status;
 }
 
+static int h264_frame_type(struct venc_h264_inst *inst)
+{
+	if ((inst->vsi->config.gop_size != 0 &&
+		(inst->frm_cnt % inst->vsi->config.gop_size) == 0) ||
+		(inst->frm_cnt == 0 && inst->vsi->config.gop_size == 0)) {
+		/* IDR frame */
+		return VENC_H264_IDR_FRM;
+	} else if ((inst->vsi->config.intra_period != 0 &&
+		(inst->frm_cnt % inst->vsi->config.intra_period) == 0) ||
+		(inst->frm_cnt == 0 && inst->vsi->config.intra_period == 0)) {
+		/* I frame */
+		return VENC_H264_I_FRM;
+	} else {
+		return VENC_H264_P_FRM;  /* Note: B frames are not supported */
+	}
+}
 static int h264_encode_sps(struct venc_h264_inst *inst,
 			   struct mtk_vcodec_mem *bs_buf,
 			   unsigned int *bs_size)
@@ -337,7 +364,7 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
-			     bs_buf, bs_size);
+			     bs_buf, bs_size, NULL);
 	if (ret)
 		return ret;
 
@@ -364,7 +391,7 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
 	mtk_vcodec_debug_enter(inst);
 
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
-			     bs_buf, bs_size);
+			     bs_buf, bs_size, NULL);
 	if (ret)
 		return ret;
 
@@ -410,13 +437,24 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 {
 	int ret = 0;
 	unsigned int irq_status;
+	struct venc_frame_info frame_info;
 
 	mtk_vcodec_debug_enter(inst);
-
+	/* Overflowing back to 0 is ok and expected here */
+	inst->frm_cnt++;
+	mtk_vcodec_debug(inst, "frm_cnt++ = %d\n ", inst->frm_cnt);
+	frame_info.frm_cnt = inst->frm_cnt;
+	frame_info.skip_frm_cnt = inst->skip_frm_cnt;
+	frame_info.frm_type = h264_frame_type(inst);
+	mtk_vcodec_debug(inst, "frm_cnt++ = %d,skip_frm_cnt =%d,frm_type=%d.\n",
+		frame_info.frm_cnt, frame_info.skip_frm_cnt,
+		frame_info.frm_type);
 	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
-			     bs_buf, bs_size);
-	if (ret)
+			     bs_buf, bs_size, &frame_info);
+	if (ret) {
+		inst->frm_cnt--;
 		return ret;
+	}
 
 	/*
 	 * skip frame case: The skip frame buffer is composed by vpu side only,
@@ -427,19 +465,19 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 		memcpy(bs_buf->va,
 		       inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
 		       *bs_size);
-		++inst->frm_cnt;
+		++inst->skip_frm_cnt;
 		return ret;
 	}
 
 	irq_status = h264_enc_wait_venc_done(inst);
 	if (irq_status != MTK_VENC_IRQ_STATUS_FRM) {
 		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
+		inst->frm_cnt--;
 		return -EIO;
 	}
 
 	*bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
 
-	++inst->frm_cnt;
 	mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-",
 			 inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm);
 
@@ -464,6 +502,7 @@ static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
 
 static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
 {
+	const bool is_ext = ctx->dev->venc_pdata->uses_ext;
 	int ret = 0;
 	struct venc_h264_inst *inst;
 
@@ -473,8 +512,9 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
 
 	inst->ctx = ctx;
 	inst->vpu_inst.ctx = ctx;
-	inst->vpu_inst.id = IPI_VENC_H264;
+	inst->vpu_inst.id = is_ext ? SCP_IPI_VENC_H264 : IPI_VENC_H264;
 	inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
+	inst->frm_cnt = 0xffffffff;
 
 	mtk_vcodec_debug_enter(inst);
 
@@ -629,7 +669,12 @@ static int h264_enc_set_param(void *handle,
 		inst->prepend_hdr = 1;
 		mtk_vcodec_debug(inst, "set prepend header mode");
 		break;
-
+	case VENC_SET_PARAM_FORCE_INTRA:
+	case VENC_SET_PARAM_GOP_SIZE:
+	case VENC_SET_PARAM_INTRA_PERIOD:
+		inst->frm_cnt = 0xffffffff;
+		inst->skip_frm_cnt = 0;
+		fallthrough;
 	default:
 		ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
 		break;
diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
index 6426af514526..11abb191ada5 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
@@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
 
 	mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
 
-	ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size);
+	ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size,
+			     NULL);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
index 52fc9cc812fc..51b52625ca22 100644
--- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
@@ -92,6 +92,12 @@ struct venc_enc_param {
 	unsigned int gop_size;
 };
 
+struct venc_frame_info {
+	unsigned int frm_cnt;		/* per frame update */
+	unsigned int skip_frm_cnt;	/* per frame update */
+	unsigned int frm_type;		/* per frame update */
+};
+
 /*
  * struct venc_frm_buf - frame buffer information used in venc_if_encode()
  * @fb_addr: plane frame buffer addresses
diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
index 28ee04ca6241..4cafbf92d9cd 100644
--- a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
+++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
@@ -51,17 +51,22 @@ struct venc_ap_ipi_msg_init {
  * @vpu_inst_addr:	VPU encoder instance addr
  *			(struct venc_vp8_vsi/venc_h264_vsi *)
  * @param_id:	parameter id (venc_set_param_type)
- * @data_item:	number of items in the data array
+ * @num_data:	number of items in the data array
  * @data[8]:	data array to store the set parameters
  */
 struct venc_ap_ipi_msg_set_param {
 	uint32_t msg_id;
 	uint32_t vpu_inst_addr;
 	uint32_t param_id;
-	uint32_t data_item;
+	uint32_t num_data;
 	uint32_t data[8];
 };
 
+struct venc_ap_ipi_msg_set_param_ext {
+	struct venc_ap_ipi_msg_set_param base;
+	uint32_t data_ext[24];
+};
+
 /**
  * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
  * @msg_id:	message id (AP_IPIMSG_XXX_ENC_ENCODE)
@@ -82,6 +87,12 @@ struct venc_ap_ipi_msg_enc {
 	uint32_t bs_size;
 };
 
+struct venc_ap_ipi_msg_enc_ext {
+	struct venc_ap_ipi_msg_enc base;
+	uint32_t data_item;
+	uint32_t data[32];
+};
+
 /**
  * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
  * @msg_id:	message id (AP_IPIMSG_XXX_ENC_DEINIT)
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
index 53854127814b..6c77bf025172 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
@@ -116,49 +116,81 @@ int vpu_enc_init(struct venc_vpu_inst *vpu)
 	return 0;
 }
 
+static unsigned int venc_enc_param_crop_right(struct venc_vpu_inst *vpu,
+					      struct venc_enc_param *enc_prm)
+{
+	unsigned int img_crop_right = enc_prm->buf_width - enc_prm->width;
+
+	return img_crop_right % 16;
+}
+
+static unsigned int venc_enc_param_crop_bottom(struct venc_enc_param *enc_prm)
+{
+	return round_up(enc_prm->height, 16) - enc_prm->height;
+}
+
+static unsigned int venc_enc_param_num_mb(struct venc_enc_param *enc_prm)
+{
+	return DIV_ROUND_UP(enc_prm->width, 16) *
+	       DIV_ROUND_UP(enc_prm->height, 16);
+}
+
 int vpu_enc_set_param(struct venc_vpu_inst *vpu,
 		      enum venc_set_param_type id,
 		      struct venc_enc_param *enc_param)
 {
-	struct venc_ap_ipi_msg_set_param out;
+	const bool is_ext = vpu->ctx->dev->venc_pdata->uses_ext;
+	size_t msg_size = is_ext ?
+		sizeof(struct venc_ap_ipi_msg_set_param_ext) :
+		sizeof(struct venc_ap_ipi_msg_set_param);
+	struct venc_ap_ipi_msg_set_param_ext out;
 
 	mtk_vcodec_debug(vpu, "id %d ->", id);
 
 	memset(&out, 0, sizeof(out));
-	out.msg_id = AP_IPIMSG_ENC_SET_PARAM;
-	out.vpu_inst_addr = vpu->inst_addr;
-	out.param_id = id;
+	out.base.msg_id = AP_IPIMSG_ENC_SET_PARAM;
+	out.base.vpu_inst_addr = vpu->inst_addr;
+	out.base.param_id = id;
 	switch (id) {
 	case VENC_SET_PARAM_ENC:
-		out.data_item = 0;
+		if (is_ext) {
+			out.base.num_data = 3;
+			out.base.data[0] =
+				venc_enc_param_crop_right(vpu, enc_param);
+			out.base.data[1] =
+				venc_enc_param_crop_bottom(enc_param);
+			out.base.data[2] = venc_enc_param_num_mb(enc_param);
+		} else {
+			out.base.num_data = 0;
+		}
 		break;
 	case VENC_SET_PARAM_FORCE_INTRA:
-		out.data_item = 0;
+		out.base.num_data = 0;
 		break;
 	case VENC_SET_PARAM_ADJUST_BITRATE:
-		out.data_item = 1;
-		out.data[0] = enc_param->bitrate;
+		out.base.num_data = 1;
+		out.base.data[0] = enc_param->bitrate;
 		break;
 	case VENC_SET_PARAM_ADJUST_FRAMERATE:
-		out.data_item = 1;
-		out.data[0] = enc_param->frm_rate;
+		out.base.num_data = 1;
+		out.base.data[0] = enc_param->frm_rate;
 		break;
 	case VENC_SET_PARAM_GOP_SIZE:
-		out.data_item = 1;
-		out.data[0] = enc_param->gop_size;
+		out.base.num_data = 1;
+		out.base.data[0] = enc_param->gop_size;
 		break;
 	case VENC_SET_PARAM_INTRA_PERIOD:
-		out.data_item = 1;
-		out.data[0] = enc_param->intra_period;
+		out.base.num_data = 1;
+		out.base.data[0] = enc_param->intra_period;
 		break;
 	case VENC_SET_PARAM_SKIP_FRAME:
-		out.data_item = 0;
+		out.base.num_data = 0;
 		break;
 	default:
 		mtk_vcodec_err(vpu, "id %d not supported", id);
 		return -EINVAL;
 	}
-	if (vpu_enc_send_msg(vpu, &out, sizeof(out))) {
+	if (vpu_enc_send_msg(vpu, &out, msg_size)) {
 		mtk_vcodec_err(vpu,
 			       "AP_IPIMSG_ENC_SET_PARAM %d fail", id);
 		return -EINVAL;
@@ -172,33 +204,44 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
 int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
-		   unsigned int *bs_size)
+		   unsigned int *bs_size,
+		   struct venc_frame_info *frame_info)
 {
-	struct venc_ap_ipi_msg_enc out;
+	const bool is_ext = vpu->ctx->dev->venc_pdata->uses_ext;
+	size_t msg_size = is_ext ?
+		sizeof(struct venc_ap_ipi_msg_enc_ext) :
+		sizeof(struct venc_ap_ipi_msg_enc);
+	struct venc_ap_ipi_msg_enc_ext out;
 
 	mtk_vcodec_debug(vpu, "bs_mode %d ->", bs_mode);
 
 	memset(&out, 0, sizeof(out));
-	out.msg_id = AP_IPIMSG_ENC_ENCODE;
-	out.vpu_inst_addr = vpu->inst_addr;
-	out.bs_mode = bs_mode;
+	out.base.msg_id = AP_IPIMSG_ENC_ENCODE;
+	out.base.vpu_inst_addr = vpu->inst_addr;
+	out.base.bs_mode = bs_mode;
 	if (frm_buf) {
 		if ((frm_buf->fb_addr[0].dma_addr % 16 == 0) &&
 		    (frm_buf->fb_addr[1].dma_addr % 16 == 0) &&
 		    (frm_buf->fb_addr[2].dma_addr % 16 == 0)) {
-			out.input_addr[0] = frm_buf->fb_addr[0].dma_addr;
-			out.input_addr[1] = frm_buf->fb_addr[1].dma_addr;
-			out.input_addr[2] = frm_buf->fb_addr[2].dma_addr;
+			out.base.input_addr[0] = frm_buf->fb_addr[0].dma_addr;
+			out.base.input_addr[1] = frm_buf->fb_addr[1].dma_addr;
+			out.base.input_addr[2] = frm_buf->fb_addr[2].dma_addr;
 		} else {
 			mtk_vcodec_err(vpu, "dma_addr not align to 16");
 			return -EINVAL;
 		}
 	}
 	if (bs_buf) {
-		out.bs_addr = bs_buf->dma_addr;
-		out.bs_size = bs_buf->size;
+		out.base.bs_addr = bs_buf->dma_addr;
+		out.base.bs_size = bs_buf->size;
 	}
-	if (vpu_enc_send_msg(vpu, &out, sizeof(out))) {
+	if (is_ext && frame_info) {
+		out.data_item = 3;
+		out.data[0] = frame_info->frm_cnt;
+		out.data[1] = frame_info->skip_frm_cnt;
+		out.data[2] = frame_info->frm_type;
+	}
+	if (vpu_enc_send_msg(vpu, &out, msg_size)) {
 		mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail",
 			       bs_mode);
 		return -EINVAL;
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
index edd411621b68..f9be9cab7ff7 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
@@ -45,7 +45,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
 int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
-		   unsigned int *bs_size);
+		   unsigned int *bs_size,
+		   struct venc_frame_info *frame_info);
 int vpu_enc_deinit(struct venc_vpu_inst *vpu);
 
 #endif
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 05/16] media: mtk-vcodec: venc: handle firmware version field

[Alexandre Courbot]

Firmwares for encoders newer than MT8173 will include an ABI version
number in their initialization ack message. Add the capacity to manage
it and make initialization fail if the firmware ABI is of a version that
we don't support.

For MT8173, this ABI version field is reserved and thus undefined ; thus
ignore it on this chip. There should only be one firmware version available
for it anyway.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  8 ++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  1 +
 .../media/platform/mtk-vcodec/venc_ipi_msg.h  |  9 ++++---
 .../media/platform/mtk-vcodec/venc_vpu_if.c   | 27 ++++++++++++++++---
 4 files changed, 38 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 45c8adfc6a0c..e7b155e7432e 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -300,13 +300,21 @@ struct mtk_vcodec_ctx {
 
 };
 
+enum mtk_chip {
+	MTK_MT8173,
+};
+
 /**
  * struct mtk_vcodec_enc_pdata - compatible data for each IC
  *
+ * @chip: chip this encoder is compatible with
+ *
  * @uses_ext: whether the encoder uses the extended firmware messaging format
  * @has_lt_irq: whether the encoder uses the LT irq
  */
 struct mtk_vcodec_enc_pdata {
+	enum mtk_chip chip;
+
 	bool uses_ext;
 	bool has_lt_irq;
 };
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 922bc8883811..c1365209263e 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -378,6 +378,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 }
 
 static const struct mtk_vcodec_enc_pdata mt8173_pdata = {
+	.chip = MTK_MT8173,
 	.has_lt_irq = true,
 };
 
diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
index 4cafbf92d9cd..31a3c76f7d0d 100644
--- a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
+++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
@@ -131,16 +131,17 @@ struct venc_vpu_ipi_msg_common {
  * @venc_inst:	AP encoder instance (struct venc_vp8_inst/venc_h264_inst *)
  * @vpu_inst_addr:	VPU encoder instance addr
  *			(struct venc_vp8_vsi/venc_h264_vsi *)
- * @reserved:	reserved for future use. vpu is running in 32bit. Without
- *		this reserved field, if kernel run in 64bit. this struct size
- *		will be different between kernel and vpu
+ * @venc_abi_version:	ABI version of the firmware. Kernel can use it to
+ *			ensure that it is compatible with the firmware.
+ *			For MT8173 the value of this field is undefined and
+ *			should not be used.
  */
 struct venc_vpu_ipi_msg_init {
 	uint32_t msg_id;
 	uint32_t status;
 	uint64_t venc_inst;
 	uint32_t vpu_inst_addr;
-	uint32_t reserved;
+	uint32_t venc_abi_version;
 };
 
 /**
diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
index 6c77bf025172..472503701003 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
@@ -4,6 +4,7 @@
  * Author: PoChun Lin <pochun.lin@mediatek.com>
  */
 
+#include "mtk_vcodec_drv.h"
 #include "mtk_vcodec_fw.h"
 #include "venc_ipi_msg.h"
 #include "venc_vpu_if.h"
@@ -15,6 +16,23 @@ static void handle_enc_init_msg(struct venc_vpu_inst *vpu, const void *data)
 	vpu->inst_addr = msg->vpu_inst_addr;
 	vpu->vsi = mtk_vcodec_fw_map_dm_addr(vpu->ctx->dev->fw_handler,
 					     msg->vpu_inst_addr);
+
+	/* Firmware version field value is unspecified on MT8173. */
+	if (vpu->ctx->dev->venc_pdata->chip == MTK_MT8173)
+		return;
+
+	/* Check firmware version. */
+	mtk_vcodec_debug(vpu, "firmware version: 0x%x\n",
+			 msg->venc_abi_version);
+	switch (msg->venc_abi_version) {
+	case 1:
+		break;
+	default:
+		mtk_vcodec_err(vpu, "unhandled firmware version 0x%x\n",
+			       msg->venc_abi_version);
+		vpu->failure = 1;
+		break;
+	}
 }
 
 static void handle_enc_encode_msg(struct venc_vpu_inst *vpu, const void *data)
@@ -35,6 +53,11 @@ static void vpu_enc_ipi_handler(void *data, unsigned int len, void *priv)
 	mtk_vcodec_debug(vpu, "msg_id %x inst %p status %d",
 			 msg->msg_id, vpu, msg->status);
 
+	vpu->signaled = 1;
+	vpu->failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
+	if (vpu->failure)
+		goto failure;
+
 	switch (msg->msg_id) {
 	case VPU_IPIMSG_ENC_INIT_DONE:
 		handle_enc_init_msg(vpu, data);
@@ -51,9 +74,7 @@ static void vpu_enc_ipi_handler(void *data, unsigned int len, void *priv)
 		break;
 	}
 
-	vpu->signaled = 1;
-	vpu->failure = (msg->status != VENC_IPI_MSG_STATUS_OK);
-
+failure:
 	mtk_vcodec_debug_leave(vpu);
 }
 
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 06/16] media: mtk-vcodec: venc: specify bitrate range per-chip

[Alexandre Courbot]

Different chips have different supported bitrate ranges. Move the min
and max supported bitrates to the platform data.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h     | 4 ++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c     | 3 ++-
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 2 ++
 3 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index e7b155e7432e..b8f913de8d80 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -311,12 +311,16 @@ enum mtk_chip {
  *
  * @uses_ext: whether the encoder uses the extended firmware messaging format
  * @has_lt_irq: whether the encoder uses the LT irq
+ * @min_birate: minimum supported encoding bitrate
+ * @max_bitrate: maximum supported encoding bitrate
  */
 struct mtk_vcodec_enc_pdata {
 	enum mtk_chip chip;
 
 	bool uses_ext;
 	bool has_lt_irq;
+	unsigned long min_bitrate;
+	unsigned long max_bitrate;
 };
 
 /**
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index d469ff6464b2..50ba9da59153 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1232,7 +1232,8 @@ int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
 	v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS_HINT);
 
 	v4l2_ctrl_new_std(handler, ops, V4L2_CID_MPEG_VIDEO_BITRATE,
-			1, 4000000, 1, 4000000);
+			  ctx->dev->venc_pdata->min_bitrate,
+			  ctx->dev->venc_pdata->max_bitrate, 1, 4000000);
 	v4l2_ctrl_new_std(handler, ops, V4L2_CID_MPEG_VIDEO_B_FRAMES,
 			0, 2, 1, 0);
 	v4l2_ctrl_new_std(handler, ops, V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE,
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index c1365209263e..221fa0303075 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -380,6 +380,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 static const struct mtk_vcodec_enc_pdata mt8173_pdata = {
 	.chip = MTK_MT8173,
 	.has_lt_irq = true,
+	.min_bitrate = 1,
+	.max_bitrate = 4000000,
 };
 
 static const struct of_device_id mtk_vcodec_enc_match[] = {
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 07/16] media: mtk-vcodec: venc: specify supported formats per-chip

[Alexandre Courbot]

Different chips have different supported bitrate ranges. Move the list
of supported formats to the platform data, and split the output and
capture formats into two lists to make it easier to find the default
format for each queue.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   8 ++
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 122 +++++++-----------
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  40 ++++++
 3 files changed, 95 insertions(+), 75 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index b8f913de8d80..59b4b750666b 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -313,6 +313,10 @@ enum mtk_chip {
  * @has_lt_irq: whether the encoder uses the LT irq
  * @min_birate: minimum supported encoding bitrate
  * @max_bitrate: maximum supported encoding bitrate
+ * @capture_formats: array of supported capture formats
+ * @num_capture_formats: number of entries in capture_formats
+ * @output_formats: array of supported output formats
+ * @num_output_formats: number of entries in output_formats
  */
 struct mtk_vcodec_enc_pdata {
 	enum mtk_chip chip;
@@ -321,6 +325,10 @@ struct mtk_vcodec_enc_pdata {
 	bool has_lt_irq;
 	unsigned long min_bitrate;
 	unsigned long max_bitrate;
+	const struct mtk_video_fmt *capture_formats;
+	size_t num_capture_formats;
+	const struct mtk_video_fmt *output_formats;
+	size_t num_output_formats;
 };
 
 /**
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 50ba9da59153..05743a745a11 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -23,47 +23,9 @@
 #define DFT_CFG_WIDTH	MTK_VENC_MIN_W
 #define DFT_CFG_HEIGHT	MTK_VENC_MIN_H
 #define MTK_MAX_CTRLS_HINT	20
-#define OUT_FMT_IDX		0
-#define CAP_FMT_IDX		4
-
 
 static void mtk_venc_worker(struct work_struct *work);
 
-static const struct mtk_video_fmt mtk_video_formats[] = {
-	{
-		.fourcc = V4L2_PIX_FMT_NV12M,
-		.type = MTK_FMT_FRAME,
-		.num_planes = 2,
-	},
-	{
-		.fourcc = V4L2_PIX_FMT_NV21M,
-		.type = MTK_FMT_FRAME,
-		.num_planes = 2,
-	},
-	{
-		.fourcc = V4L2_PIX_FMT_YUV420M,
-		.type = MTK_FMT_FRAME,
-		.num_planes = 3,
-	},
-	{
-		.fourcc = V4L2_PIX_FMT_YVU420M,
-		.type = MTK_FMT_FRAME,
-		.num_planes = 3,
-	},
-	{
-		.fourcc = V4L2_PIX_FMT_H264,
-		.type = MTK_FMT_ENC,
-		.num_planes = 1,
-	},
-	{
-		.fourcc = V4L2_PIX_FMT_VP8,
-		.type = MTK_FMT_ENC,
-		.num_planes = 1,
-	},
-};
-
-#define NUM_FORMATS ARRAY_SIZE(mtk_video_formats)
-
 static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
 	{
 		.fourcc	= V4L2_PIX_FMT_H264,
@@ -156,27 +118,17 @@ static const struct v4l2_ctrl_ops mtk_vcodec_enc_ctrl_ops = {
 	.s_ctrl = vidioc_venc_s_ctrl,
 };
 
-static int vidioc_enum_fmt(struct v4l2_fmtdesc *f, bool output_queue)
+static int vidioc_enum_fmt(struct v4l2_fmtdesc *f,
+			   const struct mtk_video_fmt *formats,
+			   size_t num_formats)
 {
-	const struct mtk_video_fmt *fmt;
-	int i, j = 0;
+	if (f->index >= num_formats)
+		return -EINVAL;
 
-	for (i = 0; i < NUM_FORMATS; ++i) {
-		if (output_queue && mtk_video_formats[i].type != MTK_FMT_FRAME)
-			continue;
-		if (!output_queue && mtk_video_formats[i].type != MTK_FMT_ENC)
-			continue;
+	f->pixelformat = formats[f->index].fourcc;
+	memset(f->reserved, 0, sizeof(f->reserved));
 
-		if (j == f->index) {
-			fmt = &mtk_video_formats[i];
-			f->pixelformat = fmt->fourcc;
-			memset(f->reserved, 0, sizeof(f->reserved));
-			return 0;
-		}
-		++j;
-	}
-
-	return -EINVAL;
+	return 0;
 }
 
 static int vidioc_enum_framesizes(struct file *file, void *fh,
@@ -202,13 +154,21 @@ static int vidioc_enum_framesizes(struct file *file, void *fh,
 static int vidioc_enum_fmt_vid_cap(struct file *file, void *priv,
 				   struct v4l2_fmtdesc *f)
 {
-	return vidioc_enum_fmt(f, false);
+	const struct mtk_vcodec_enc_pdata *pdata =
+		fh_to_ctx(priv)->dev->venc_pdata;
+
+	return vidioc_enum_fmt(f, pdata->capture_formats,
+			       pdata->num_capture_formats);
 }
 
 static int vidioc_enum_fmt_vid_out(struct file *file, void *priv,
 				   struct v4l2_fmtdesc *f)
 {
-	return vidioc_enum_fmt(f, true);
+	const struct mtk_vcodec_enc_pdata *pdata =
+		fh_to_ctx(priv)->dev->venc_pdata;
+
+	return vidioc_enum_fmt(f, pdata->output_formats,
+			       pdata->num_output_formats);
 }
 
 static int vidioc_venc_querycap(struct file *file, void *priv,
@@ -266,13 +226,20 @@ static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
 	return &ctx->q_data[MTK_Q_DATA_DST];
 }
 
-static const struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f)
+static const struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f,
+	const struct mtk_vcodec_enc_pdata *pdata)
 {
 	const struct mtk_video_fmt *fmt;
 	unsigned int k;
 
-	for (k = 0; k < NUM_FORMATS; k++) {
-		fmt = &mtk_video_formats[k];
+	for (k = 0; k < pdata->num_capture_formats; k++) {
+		fmt = &pdata->capture_formats[k];
+		if (fmt->fourcc == f->fmt.pix.pixelformat)
+			return fmt;
+	}
+
+	for (k = 0; k < pdata->num_output_formats; k++) {
+		fmt = &pdata->output_formats[k];
 		if (fmt->fourcc == f->fmt.pix.pixelformat)
 			return fmt;
 	}
@@ -414,6 +381,7 @@ static int vidioc_venc_s_fmt_cap(struct file *file, void *priv,
 			     struct v4l2_format *f)
 {
 	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	const struct mtk_vcodec_enc_pdata *pdata = ctx->dev->venc_pdata;
 	struct vb2_queue *vq;
 	struct mtk_q_data *q_data;
 	int i, ret;
@@ -436,10 +404,10 @@ static int vidioc_venc_s_fmt_cap(struct file *file, void *priv,
 		return -EINVAL;
 	}
 
-	fmt = mtk_venc_find_format(f);
+	fmt = mtk_venc_find_format(f, pdata);
 	if (!fmt) {
-		f->fmt.pix.pixelformat = mtk_video_formats[CAP_FMT_IDX].fourcc;
-		fmt = mtk_venc_find_format(f);
+		fmt = &ctx->dev->venc_pdata->capture_formats[0];
+		f->fmt.pix.pixelformat = fmt->fourcc;
 	}
 
 	q_data->fmt = fmt;
@@ -476,6 +444,7 @@ static int vidioc_venc_s_fmt_out(struct file *file, void *priv,
 			     struct v4l2_format *f)
 {
 	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	const struct mtk_vcodec_enc_pdata *pdata = ctx->dev->venc_pdata;
 	struct vb2_queue *vq;
 	struct mtk_q_data *q_data;
 	int ret, i;
@@ -499,10 +468,10 @@ static int vidioc_venc_s_fmt_out(struct file *file, void *priv,
 		return -EINVAL;
 	}
 
-	fmt = mtk_venc_find_format(f);
+	fmt = mtk_venc_find_format(f, pdata);
 	if (!fmt) {
-		f->fmt.pix.pixelformat = mtk_video_formats[OUT_FMT_IDX].fourcc;
-		fmt = mtk_venc_find_format(f);
+		fmt = &ctx->dev->venc_pdata->output_formats[0];
+		f->fmt.pix.pixelformat = fmt->fourcc;
 	}
 
 	pix_fmt_mp->height = clamp(pix_fmt_mp->height,
@@ -580,11 +549,12 @@ static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 {
 	const struct mtk_video_fmt *fmt;
 	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	const struct mtk_vcodec_enc_pdata *pdata = ctx->dev->venc_pdata;
 
-	fmt = mtk_venc_find_format(f);
+	fmt = mtk_venc_find_format(f, pdata);
 	if (!fmt) {
-		f->fmt.pix.pixelformat = mtk_video_formats[CAP_FMT_IDX].fourcc;
-		fmt = mtk_venc_find_format(f);
+		fmt = &ctx->dev->venc_pdata->capture_formats[0];
+		f->fmt.pix.pixelformat = fmt->fourcc;
 	}
 	f->fmt.pix_mp.colorspace = ctx->colorspace;
 	f->fmt.pix_mp.ycbcr_enc = ctx->ycbcr_enc;
@@ -598,11 +568,13 @@ static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
 					 struct v4l2_format *f)
 {
 	const struct mtk_video_fmt *fmt;
+	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	const struct mtk_vcodec_enc_pdata *pdata = ctx->dev->venc_pdata;
 
-	fmt = mtk_venc_find_format(f);
+	fmt = mtk_venc_find_format(f, pdata);
 	if (!fmt) {
-		f->fmt.pix.pixelformat = mtk_video_formats[OUT_FMT_IDX].fourcc;
-		fmt = mtk_venc_find_format(f);
+		fmt = &ctx->dev->venc_pdata->output_formats[0];
+		f->fmt.pix.pixelformat = fmt->fourcc;
 	}
 	if (!f->fmt.pix_mp.colorspace) {
 		f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
@@ -1187,7 +1159,7 @@ void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx)
 	q_data->coded_height = DFT_CFG_HEIGHT;
 	q_data->field = V4L2_FIELD_NONE;
 
-	q_data->fmt = &mtk_video_formats[OUT_FMT_IDX];
+	q_data->fmt = &ctx->dev->venc_pdata->output_formats[0];
 
 	v4l_bound_align_image(&q_data->coded_width,
 				MTK_VENC_MIN_W,
@@ -1216,7 +1188,7 @@ void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx)
 	memset(q_data, 0, sizeof(struct mtk_q_data));
 	q_data->coded_width = DFT_CFG_WIDTH;
 	q_data->coded_height = DFT_CFG_HEIGHT;
-	q_data->fmt = &mtk_video_formats[CAP_FMT_IDX];
+	q_data->fmt = &ctx->dev->venc_pdata->capture_formats[0];
 	q_data->field = V4L2_FIELD_NONE;
 	ctx->q_data[MTK_Q_DATA_DST].sizeimage[0] =
 		DFT_CFG_WIDTH * DFT_CFG_HEIGHT;
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index 221fa0303075..c8a4b85a81c4 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -26,6 +26,42 @@
 module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
 module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
 
+static const struct mtk_video_fmt mtk_video_formats_output_mt8173[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_NV12M,
+		.type = MTK_FMT_FRAME,
+		.num_planes = 2,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_NV21M,
+		.type = MTK_FMT_FRAME,
+		.num_planes = 2,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_YUV420M,
+		.type = MTK_FMT_FRAME,
+		.num_planes = 3,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_YVU420M,
+		.type = MTK_FMT_FRAME,
+		.num_planes = 3,
+	},
+};
+
+static const struct mtk_video_fmt mtk_video_formats_capture_mt8173[] =  {
+	{
+		.fourcc = V4L2_PIX_FMT_H264,
+		.type = MTK_FMT_ENC,
+		.num_planes = 1,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_VP8,
+		.type = MTK_FMT_ENC,
+		.num_planes = 1,
+	},
+};
+
 /* Wake up context wait_queue */
 static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
 {
@@ -380,6 +416,10 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 static const struct mtk_vcodec_enc_pdata mt8173_pdata = {
 	.chip = MTK_MT8173,
 	.has_lt_irq = true,
+	.capture_formats = mtk_video_formats_capture_mt8173,
+	.num_capture_formats = ARRAY_SIZE(mtk_video_formats_capture_mt8173),
+	.output_formats = mtk_video_formats_output_mt8173,
+	.num_output_formats = ARRAY_SIZE(mtk_video_formats_output_mt8173),
 	.min_bitrate = 1,
 	.max_bitrate = 4000000,
 };
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 08/16] dt-bindings: media: document mediatek,mt8183-vcodec-enc

[Alexandre Courbot]

MT8183's encoder is similar to MT8173's.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
index 7aef0a4fe207..8093335afecb 100644
--- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
+++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
@@ -4,8 +4,9 @@ Mediatek Video Codec is the video codec hw present in Mediatek SoCs which
 supports high resolution encoding and decoding functionalities.
 
 Required properties:
-- compatible : "mediatek,mt8173-vcodec-enc" for encoder
-  "mediatek,mt8173-vcodec-dec" for decoder.
+- compatible : "mediatek,mt8173-vcodec-enc" for MT8173 encoder
+  "mediatek,mt8183-vcodec-enc" for MT8183 encoder.
+  "mediatek,mt8173-vcodec-dec" for MT8173 decoder.
 - reg : Physical base address of the video codec registers and length of
   memory mapped region.
 - interrupts : interrupt number to the cpu.
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 09/16] media: mtk-vcodec: add support for MT8183 encoder

[Alexandre Courbot]

From: Yunfei Dong <yunfei.dong@mediatek.com>

Now that all the supporting blocks are present, enable encoder for
MT8183.

Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
[acourbot: refactor, cleanup and split]
Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  1 +
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 22 +++++++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
index 59b4b750666b..a271611750ad 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
@@ -302,6 +302,7 @@ struct mtk_vcodec_ctx {
 
 enum mtk_chip {
 	MTK_MT8173,
+	MTK_MT8183,
 };
 
 /**
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
index c8a4b85a81c4..b8fe408dd6c7 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
@@ -62,6 +62,14 @@ static const struct mtk_video_fmt mtk_video_formats_capture_mt8173[] =  {
 	},
 };
 
+static const struct mtk_video_fmt mtk_video_formats_capture_mt8183[] =  {
+	{
+		.fourcc = V4L2_PIX_FMT_H264,
+		.type = MTK_FMT_ENC,
+		.num_planes = 1,
+	},
+};
+
 /* Wake up context wait_queue */
 static void wake_up_ctx(struct mtk_vcodec_ctx *ctx, unsigned int reason)
 {
@@ -424,8 +432,22 @@ static const struct mtk_vcodec_enc_pdata mt8173_pdata = {
 	.max_bitrate = 4000000,
 };
 
+static const struct mtk_vcodec_enc_pdata mt8183_pdata = {
+	.chip = MTK_MT8183,
+	.has_lt_irq = false,
+	.uses_ext = true,
+	.capture_formats = mtk_video_formats_capture_mt8183,
+	.num_capture_formats = ARRAY_SIZE(mtk_video_formats_capture_mt8183),
+	/* MT8183 supports the same output formats as MT8173 */
+	.output_formats = mtk_video_formats_output_mt8173,
+	.num_output_formats = ARRAY_SIZE(mtk_video_formats_output_mt8173),
+	.min_bitrate = 64,
+	.max_bitrate = 40000000,
+};
+
 static const struct of_device_id mtk_vcodec_enc_match[] = {
 	{.compatible = "mediatek,mt8173-vcodec-enc", .data = &mt8173_pdata},
+	{.compatible = "mediatek,mt8183-vcodec-enc", .data = &mt8183_pdata},
 	{},
 };
 MODULE_DEVICE_TABLE(of, mtk_vcodec_enc_match);
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 10/16] Revert "media: mtk-vcodec: Remove extra area allocation in an input buffer on encoding"

[Alexandre Courbot]

This reverts commit 81735ecb62f882853a37a8c157407ec4aed44fd0.

The hardware needs data to follow the previous alignment, so this extra
space was not superfluous after all. Besides, this also made
v4l2-compliance's G_FMT and S_FMT tests regress.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 05743a745a11..f2ba19c32400 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -299,12 +299,14 @@ static int vidioc_try_fmt(struct v4l2_format *f,
 
 		pix_fmt_mp->num_planes = fmt->num_planes;
 		pix_fmt_mp->plane_fmt[0].sizeimage =
-			pix_fmt_mp->width * pix_fmt_mp->height;
+				pix_fmt_mp->width * pix_fmt_mp->height +
+				((ALIGN(pix_fmt_mp->width, 16) * 2) * 16);
 		pix_fmt_mp->plane_fmt[0].bytesperline = pix_fmt_mp->width;
 
 		if (pix_fmt_mp->num_planes == 2) {
 			pix_fmt_mp->plane_fmt[1].sizeimage =
-				(pix_fmt_mp->width * pix_fmt_mp->height) / 2;
+				(pix_fmt_mp->width * pix_fmt_mp->height) / 2 +
+				(ALIGN(pix_fmt_mp->width, 16) * 16);
 			pix_fmt_mp->plane_fmt[2].sizeimage = 0;
 			pix_fmt_mp->plane_fmt[1].bytesperline =
 							pix_fmt_mp->width;
@@ -312,7 +314,8 @@ static int vidioc_try_fmt(struct v4l2_format *f,
 		} else if (pix_fmt_mp->num_planes == 3) {
 			pix_fmt_mp->plane_fmt[1].sizeimage =
 			pix_fmt_mp->plane_fmt[2].sizeimage =
-				(pix_fmt_mp->width * pix_fmt_mp->height) / 4;
+				(pix_fmt_mp->width * pix_fmt_mp->height) / 4 +
+				((ALIGN(pix_fmt_mp->width, 16) / 2) * 16);
 			pix_fmt_mp->plane_fmt[1].bytesperline =
 				pix_fmt_mp->plane_fmt[2].bytesperline =
 				pix_fmt_mp->width / 2;
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 11/16] media: mtk-vcodec: venc support MIN_OUTPUT_BUFFERS control

[Alexandre Courbot]

This control is required by v4l2-compliance for encoders. A value of 1
should be suitable for all scenarios.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index f2ba19c32400..f833aee4a06f 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -1206,6 +1206,8 @@ int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
 
 	v4l2_ctrl_handler_init(handler, MTK_MAX_CTRLS_HINT);
 
+	v4l2_ctrl_new_std(handler, ops, V4L2_CID_MIN_BUFFERS_FOR_OUTPUT,
+			  1, 1, 1, 1);
 	v4l2_ctrl_new_std(handler, ops, V4L2_CID_MPEG_VIDEO_BITRATE,
 			  ctx->dev->venc_pdata->min_bitrate,
 			  ctx->dev->venc_pdata->max_bitrate, 1, 4000000);
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 12/16] media: mtk-vcodec: venc: set OUTPUT buffers field to V4L2_FIELD_NONE

[Alexandre Courbot]

A default value of 0 means V4L2_FIELD_ANY, which is not correct.
Reported by v4l2-compliance.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index f833aee4a06f..1a981d842c19 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -893,8 +893,17 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 	ctx->state = MTK_STATE_FREE;
 }
 
+static int vb2ops_venc_buf_out_validate(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+	vbuf->field = V4L2_FIELD_NONE;
+	return 0;
+}
+
 static const struct vb2_ops mtk_venc_vb2_ops = {
 	.queue_setup		= vb2ops_venc_queue_setup,
+	.buf_out_validate	= vb2ops_venc_buf_out_validate,
 	.buf_prepare		= vb2ops_venc_buf_prepare,
 	.buf_queue		= vb2ops_venc_buf_queue,
 	.wait_prepare		= vb2_ops_wait_prepare,
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 13/16] media: mtk-vcodec: venc: use platform data for ENUM_FRAMESIZES

[Alexandre Courbot]

vidioc_enum_framesizes() assumes that all encoders support H.264 and VP8,
which is not necessarily true and requires to duplicate information about
the supported codecs which is already stored in the platform data.

Fix this by referring to the platform data to find out whether a given
format is supported. Since the supported sizes are all the same
regardless of the format, we can then return a copy of a static value if
the format is supported.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 24 ++++++++-----------
 1 file changed, 10 insertions(+), 14 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 1a981d842c19..f8d4fbe927f9 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -26,17 +26,9 @@
 
 static void mtk_venc_worker(struct work_struct *work);
 
-static const struct mtk_codec_framesizes mtk_venc_framesizes[] = {
-	{
-		.fourcc	= V4L2_PIX_FMT_H264,
-		.stepwise = { MTK_VENC_MIN_W, MTK_VENC_MAX_W, 16,
-			      MTK_VENC_MIN_H, MTK_VENC_MAX_H, 16 },
-	},
-	{
-		.fourcc = V4L2_PIX_FMT_VP8,
-		.stepwise = { MTK_VENC_MIN_W, MTK_VENC_MAX_W, 16,
-			      MTK_VENC_MIN_H, MTK_VENC_MAX_H, 16 },
-	},
+static const struct v4l2_frmsize_stepwise mtk_venc_framesizes = {
+	MTK_VENC_MIN_W, MTK_VENC_MAX_W, 16,
+	MTK_VENC_MIN_H, MTK_VENC_MAX_H, 16,
 };
 
 #define NUM_SUPPORTED_FRAMESIZE ARRAY_SIZE(mtk_venc_framesizes)
@@ -134,17 +126,21 @@ static int vidioc_enum_fmt(struct v4l2_fmtdesc *f,
 static int vidioc_enum_framesizes(struct file *file, void *fh,
 				  struct v4l2_frmsizeenum *fsize)
 {
+	const struct mtk_vcodec_enc_pdata *pdata =
+		fh_to_ctx(fh)->dev->venc_pdata;
 	int i = 0;
 
 	if (fsize->index != 0)
 		return -EINVAL;
 
-	for (i = 0; i < NUM_SUPPORTED_FRAMESIZE; ++i) {
-		if (fsize->pixel_format != mtk_venc_framesizes[i].fourcc)
+	for (i = 0; i < pdata->num_capture_formats; ++i) {
+		const struct mtk_video_fmt *fmt = &pdata->capture_formats[i];
+
+		if (fsize->pixel_format != fmt->fourcc)
 			continue;
 
 		fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
-		fsize->stepwise = mtk_venc_framesizes[i].stepwise;
+		fsize->stepwise = mtk_venc_framesizes;
 		return 0;
 	}
 
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 14/16] media: mtk-vcodec: venc: support ENUM_FRAMESIZES on OUTPUT formats

[Alexandre Courbot]

v4l2-compliance requires ENUM_FRAMESIZES to support OUTPUT formats.
Reuse mtk_venc_find_format() to make sure both queues are considered
when serving an ENUM_FRAMESIZES.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 71 +++++++++----------
 1 file changed, 33 insertions(+), 38 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index f8d4fbe927f9..1b79185cf922 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -123,28 +123,44 @@ static int vidioc_enum_fmt(struct v4l2_fmtdesc *f,
 	return 0;
 }
 
+static const struct mtk_video_fmt *mtk_venc_find_format(u32 fourcc,
+	const struct mtk_vcodec_enc_pdata *pdata)
+{
+	const struct mtk_video_fmt *fmt;
+	unsigned int k;
+
+	for (k = 0; k < pdata->num_capture_formats; k++) {
+		fmt = &pdata->capture_formats[k];
+		if (fmt->fourcc == fourcc)
+			return fmt;
+	}
+
+	for (k = 0; k < pdata->num_output_formats; k++) {
+		fmt = &pdata->output_formats[k];
+		if (fmt->fourcc == fourcc)
+			return fmt;
+	}
+
+	return NULL;
+}
+
 static int vidioc_enum_framesizes(struct file *file, void *fh,
 				  struct v4l2_frmsizeenum *fsize)
 {
-	const struct mtk_vcodec_enc_pdata *pdata =
-		fh_to_ctx(fh)->dev->venc_pdata;
-	int i = 0;
+	const struct mtk_video_fmt *fmt;
 
 	if (fsize->index != 0)
 		return -EINVAL;
 
-	for (i = 0; i < pdata->num_capture_formats; ++i) {
-		const struct mtk_video_fmt *fmt = &pdata->capture_formats[i];
+	fmt = mtk_venc_find_format(fsize->pixel_format,
+				   fh_to_ctx(fh)->dev->venc_pdata);
+	if (!fmt)
+		return -EINVAL;
 
-		if (fsize->pixel_format != fmt->fourcc)
-			continue;
+	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+	fsize->stepwise = mtk_venc_framesizes;
 
-		fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
-		fsize->stepwise = mtk_venc_framesizes;
-		return 0;
-	}
-
-	return -EINVAL;
+	return 0;
 }
 
 static int vidioc_enum_fmt_vid_cap(struct file *file, void *priv,
@@ -222,27 +238,6 @@ static struct mtk_q_data *mtk_venc_get_q_data(struct mtk_vcodec_ctx *ctx,
 	return &ctx->q_data[MTK_Q_DATA_DST];
 }
 
-static const struct mtk_video_fmt *mtk_venc_find_format(struct v4l2_format *f,
-	const struct mtk_vcodec_enc_pdata *pdata)
-{
-	const struct mtk_video_fmt *fmt;
-	unsigned int k;
-
-	for (k = 0; k < pdata->num_capture_formats; k++) {
-		fmt = &pdata->capture_formats[k];
-		if (fmt->fourcc == f->fmt.pix.pixelformat)
-			return fmt;
-	}
-
-	for (k = 0; k < pdata->num_output_formats; k++) {
-		fmt = &pdata->output_formats[k];
-		if (fmt->fourcc == f->fmt.pix.pixelformat)
-			return fmt;
-	}
-
-	return NULL;
-}
-
 /* V4L2 specification suggests the driver corrects the format struct if any of
  * the dimensions is unsupported
  */
@@ -403,7 +398,7 @@ static int vidioc_venc_s_fmt_cap(struct file *file, void *priv,
 		return -EINVAL;
 	}
 
-	fmt = mtk_venc_find_format(f, pdata);
+	fmt = mtk_venc_find_format(f->fmt.pix.pixelformat, pdata);
 	if (!fmt) {
 		fmt = &ctx->dev->venc_pdata->capture_formats[0];
 		f->fmt.pix.pixelformat = fmt->fourcc;
@@ -467,7 +462,7 @@ static int vidioc_venc_s_fmt_out(struct file *file, void *priv,
 		return -EINVAL;
 	}
 
-	fmt = mtk_venc_find_format(f, pdata);
+	fmt = mtk_venc_find_format(f->fmt.pix.pixelformat, pdata);
 	if (!fmt) {
 		fmt = &ctx->dev->venc_pdata->output_formats[0];
 		f->fmt.pix.pixelformat = fmt->fourcc;
@@ -550,7 +545,7 @@ static int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
 	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
 	const struct mtk_vcodec_enc_pdata *pdata = ctx->dev->venc_pdata;
 
-	fmt = mtk_venc_find_format(f, pdata);
+	fmt = mtk_venc_find_format(f->fmt.pix.pixelformat, pdata);
 	if (!fmt) {
 		fmt = &ctx->dev->venc_pdata->capture_formats[0];
 		f->fmt.pix.pixelformat = fmt->fourcc;
@@ -570,7 +565,7 @@ static int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
 	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
 	const struct mtk_vcodec_enc_pdata *pdata = ctx->dev->venc_pdata;
 
-	fmt = mtk_venc_find_format(f, pdata);
+	fmt = mtk_venc_find_format(f->fmt.pix.pixelformat, pdata);
 	if (!fmt) {
 		fmt = &ctx->dev->venc_pdata->output_formats[0];
 		f->fmt.pix.pixelformat = fmt->fourcc;
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 15/16] media: mtk-vcodec: venc: set default time per frame

[Alexandre Courbot]

The time per frame was left initialized to 0/0, which make the driver
fail v4l2-compliance, and also leaves it potentially exposed to doing a
division by zero.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 1b79185cf922..09baac2dbc36 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -24,6 +24,9 @@
 #define DFT_CFG_HEIGHT	MTK_VENC_MIN_H
 #define MTK_MAX_CTRLS_HINT	20
 
+#define MTK_DEFAULT_FRAMERATE_NUM 1001
+#define MTK_DEFAULT_FRAMERATE_DENOM 30000
+
 static void mtk_venc_worker(struct work_struct *work);
 
 static const struct v4l2_frmsize_stepwise mtk_venc_framesizes = {
@@ -1197,6 +1200,8 @@ void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx)
 		DFT_CFG_WIDTH * DFT_CFG_HEIGHT;
 	ctx->q_data[MTK_Q_DATA_DST].bytesperline[0] = 0;
 
+	ctx->enc_params.framerate_num = MTK_DEFAULT_FRAMERATE_NUM;
+	ctx->enc_params.framerate_denom = MTK_DEFAULT_FRAMERATE_DENOM;
 }
 
 int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx)
-- 
2.27.0.383.g050319c2ae-goog

[PATCH v3 16/16] media: mtk-vcodec: venc: fix invalid time per frame in S_PARM

[Alexandre Courbot]

v4l2-compliance expects the driver to adjust the time per frame if it is
invalid (numerator or denominator set to 0). Adjust it to the default
value in these cases.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 09baac2dbc36..82b04714f750 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -200,14 +200,18 @@ static int vidioc_venc_s_parm(struct file *file, void *priv,
 			      struct v4l2_streamparm *a)
 {
 	struct mtk_vcodec_ctx *ctx = fh_to_ctx(priv);
+	struct v4l2_fract *timeperframe = &a->parm.output.timeperframe;
 
 	if (a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
 		return -EINVAL;
 
-	ctx->enc_params.framerate_num =
-			a->parm.output.timeperframe.denominator;
-	ctx->enc_params.framerate_denom =
-			a->parm.output.timeperframe.numerator;
+	if (timeperframe->numerator == 0 || timeperframe->denominator == 0) {
+		timeperframe->numerator = MTK_DEFAULT_FRAMERATE_NUM;
+		timeperframe->denominator = MTK_DEFAULT_FRAMERATE_DENOM;
+	}
+
+	ctx->enc_params.framerate_num = timeperframe->denominator;
+	ctx->enc_params.framerate_denom = timeperframe->numerator;
 	ctx->param_change |= MTK_ENCODE_PARAM_FRAMERATE;
 
 	a->parm.output.capability = V4L2_CAP_TIMEPERFRAME;
-- 
2.27.0.383.g050319c2ae-goog

Re: [PATCH v3 02/16] dt-bindings: media: mtk-vcodec: document SCP node

[Chen-Yu Tsai]

On Mon, Jul 13, 2020 at 2:09 PM Alexandre Courbot <acourbot@chromium.org> wrote:
>
> The mediatek codecs can use either the VPU or the SCP as their interface
> to firmware. Reflect this in the DT bindings.
>
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> index b6b5dde6abd8..7aef0a4fe207 100644
> --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -19,7 +19,9 @@ Required properties:
>  - iommus : should point to the respective IOMMU block with master port as
>    argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
>    for details.
> -- mediatek,vpu : the node of video processor unit
> +One of the two following nodes:
> +- mediatek,vpu : the node of the video processor unit, if using VPU.
> +- mediatek,scp : the noode of the SCP unit, if using SCP.

                         ^ typo / extra o

ChenYu

Re: [PATCH v3 08/16] dt-bindings: media: document mediatek,mt8183-vcodec-enc

[Rob Herring]

On Mon, 13 Jul 2020 15:08:34 +0900, Alexandre Courbot wrote:
> MT8183's encoder is similar to MT8173's.
> 
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> ---
>  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 

Acked-by: Rob Herring <robh@kernel.org>

Re: [PATCH v3 01/16] media: mtk-vcodec: abstract firmware interface

[Ezequiel Garcia]

On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
>
> From: Yunfei Dong <yunfei.dong@mediatek.com>
>
> MT8183's codec firwmare is run by a different remote processor from
> MT8173. While the firmware interface is basically the same, the way to
> invoke it differs. Abstract all firmware calls under a layer that will
> allow us to handle both firmware types transparently.
>
> Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> [acourbot: refactor, cleanup and split]
> Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> [pihsun: fix error path and add mtk_vcodec_fw_release]
> Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> Reviewed-by: Tiffany Lin <tiffany.lin@mediatek.com>
> Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/mtk-vcodec/Makefile    |   4 +-
>  .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  50 ++---
>  .../platform/mtk-vcodec/mtk_vcodec_dec_pm.c   |   1 -
>  .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   5 +-
>  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  47 ++---
>  .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   |   2 -
>  .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 172 ++++++++++++++++++
>  .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  36 ++++
>  .../platform/mtk-vcodec/mtk_vcodec_util.c     |   1 -
>  .../platform/mtk-vcodec/vdec/vdec_h264_if.c   |   1 -
>  .../platform/mtk-vcodec/vdec/vdec_vp8_if.c    |   1 -
>  .../platform/mtk-vcodec/vdec/vdec_vp9_if.c    |   1 -
>  .../media/platform/mtk-vcodec/vdec_drv_base.h |   2 -
>  .../media/platform/mtk-vcodec/vdec_drv_if.c   |   1 -
>  .../media/platform/mtk-vcodec/vdec_vpu_if.c   |  12 +-
>  .../media/platform/mtk-vcodec/vdec_vpu_if.h   |  11 +-
>  .../platform/mtk-vcodec/venc/venc_h264_if.c   |  15 +-
>  .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   8 +-
>  .../media/platform/mtk-vcodec/venc_drv_if.c   |   1 -
>  .../media/platform/mtk-vcodec/venc_vpu_if.c   |  17 +-
>  .../media/platform/mtk-vcodec/venc_vpu_if.h   |   5 +-
>  21 files changed, 290 insertions(+), 103 deletions(-)
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
>
> diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> index 37b94b555fa1..b8636119ed0a 100644
> --- a/drivers/media/platform/mtk-vcodec/Makefile
> +++ b/drivers/media/platform/mtk-vcodec/Makefile
> @@ -12,7 +12,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
>                 vdec_vpu_if.o \
>                 mtk_vcodec_dec.o \
>                 mtk_vcodec_dec_pm.o \
> -
> +               mtk_vcodec_fw.o
>
>  mtk-vcodec-enc-y := venc/venc_vp8_if.o \
>                 venc/venc_h264_if.o \
> @@ -25,5 +25,3 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
>
>  mtk-vcodec-common-y := mtk_vcodec_intr.o \
>                 mtk_vcodec_util.o\
> -
> -ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> index 97a1b6664c20..4f07a5fcce7f 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> @@ -20,7 +20,7 @@
>  #include "mtk_vcodec_dec_pm.h"
>  #include "mtk_vcodec_intr.h"
>  #include "mtk_vcodec_util.h"
> -#include "mtk_vpu.h"
> +#include "mtk_vcodec_fw.h"
>
>  #define VDEC_HW_ACTIVE 0x10
>  #define VDEC_IRQ_CFG   0x11
> @@ -77,22 +77,6 @@ static irqreturn_t mtk_vcodec_dec_irq_handler(int irq, void *priv)
>         return IRQ_HANDLED;
>  }
>
> -static void mtk_vcodec_dec_reset_handler(void *priv)
> -{
> -       struct mtk_vcodec_dev *dev = priv;
> -       struct mtk_vcodec_ctx *ctx;
> -
> -       mtk_v4l2_err("Watchdog timeout!!");
> -
> -       mutex_lock(&dev->dev_mutex);
> -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> -               ctx->state = MTK_STATE_ABORT;
> -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR",
> -                               ctx->id);
> -       }
> -       mutex_unlock(&dev->dev_mutex);
> -}
> -
>  static int fops_vcodec_open(struct file *file)
>  {
>         struct mtk_vcodec_dev *dev = video_drvdata(file);
> @@ -144,21 +128,20 @@ static int fops_vcodec_open(struct file *file)
>         if (v4l2_fh_is_singular(&ctx->fh)) {
>                 mtk_vcodec_dec_pw_on(&dev->pm);
>                 /*
> -                * vpu_load_firmware checks if it was loaded already and
> -                * does nothing in that case
> +                * Does nothing if firmware was already loaded.
>                  */
> -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
>                 if (ret < 0) {
>                         /*
>                          * Return 0 if downloading firmware successfully,
>                          * otherwise it is failed
>                          */
> -                       mtk_v4l2_err("vpu_load_firmware failed!");
> +                       mtk_v4l2_err("failed to load firmware!");
>                         goto err_load_fw;
>                 }
>
>                 dev->dec_capability =
> -                       vpu_get_vdec_hw_capa(dev->vpu_plat_dev);
> +                       mtk_vcodec_fw_get_vdec_capa(dev->fw_handler);
>                 mtk_v4l2_debug(0, "decoder capability %x", dev->dec_capability);
>         }
>
> @@ -228,6 +211,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>         struct mtk_vcodec_dev *dev;
>         struct video_device *vfd_dec;
>         struct resource *res;
> +       phandle rproc_phandle;
> +       enum mtk_vcodec_fw_type fw_type;
>         int i, ret;
>
>         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> @@ -237,19 +222,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>         INIT_LIST_HEAD(&dev->ctx_list);
>         dev->plat_dev = pdev;
>
> -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> -       if (dev->vpu_plat_dev == NULL) {
> -               mtk_v4l2_err("[VPU] vpu device in not ready");
> -               return -EPROBE_DEFER;
> +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> +                                 &rproc_phandle)) {
> +               fw_type = VPU;
> +       } else {
> +               mtk_v4l2_err("Could not get vdec IPI device");
> +               return -ENODEV;
>         }
> -
> -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_dec_reset_handler,
> -                       dev, VPU_RST_DEC);
> +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_DEC);
> +       if (IS_ERR(dev->fw_handler))
> +               return PTR_ERR(dev->fw_handler);
>
>         ret = mtk_vcodec_init_dec_pm(dev);
>         if (ret < 0) {
>                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source");
> -               return ret;
> +               goto err_dec_pm;
>         }
>
>         for (i = 0; i < NUM_MAX_VDEC_REG_BASE; i++) {
> @@ -352,6 +339,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>         v4l2_device_unregister(&dev->v4l2_dev);
>  err_res:
>         mtk_vcodec_release_dec_pm(dev);
> +err_dec_pm:
> +       mtk_vcodec_fw_release(dev->fw_handler);
>         return ret;
>  }
>
> @@ -376,6 +365,7 @@ static int mtk_vcodec_dec_remove(struct platform_device *pdev)
>
>         v4l2_device_unregister(&dev->v4l2_dev);
>         mtk_vcodec_release_dec_pm(dev);
> +       mtk_vcodec_fw_release(dev->fw_handler);
>         return 0;
>  }
>
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> index 5a6ec8fb52da..36dfe3fc056a 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> @@ -12,7 +12,6 @@
>
>  #include "mtk_vcodec_dec_pm.h"
>  #include "mtk_vcodec_util.h"
> -#include "mtk_vpu.h"
>
>  int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *mtkdev)
>  {
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> index 9fd56dee7fd1..e132c4ec463a 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -309,13 +309,13 @@ struct mtk_vcodec_ctx {
>   * @m2m_dev_dec: m2m device for decoder
>   * @m2m_dev_enc: m2m device for encoder.
>   * @plat_dev: platform device
> - * @vpu_plat_dev: mtk vpu platform device
>   * @ctx_list: list of struct mtk_vcodec_ctx
>   * @irqlock: protect data access by irq handler and work thread
>   * @curr_ctx: The context that is waiting for codec hardware
>   *
>   * @reg_base: Mapped address of MTK Vcodec registers.
>   *
> + * @fw_handler: used to communicate with the firmware.
>   * @id_counter: used to identify current opened instance
>   *
>   * @encode_workqueue: encode work queue
> @@ -344,12 +344,13 @@ struct mtk_vcodec_dev {
>         struct v4l2_m2m_dev *m2m_dev_dec;
>         struct v4l2_m2m_dev *m2m_dev_enc;
>         struct platform_device *plat_dev;
> -       struct platform_device *vpu_plat_dev;
>         struct list_head ctx_list;
>         spinlock_t irqlock;
>         struct mtk_vcodec_ctx *curr_ctx;
>         void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
>
> +       struct mtk_vcodec_fw *fw_handler;
> +
>         unsigned long id_counter;
>
>         struct workqueue_struct *decode_workqueue;
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> index 4d31f1ed113f..4340ea10afd0 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> @@ -21,7 +21,7 @@
>  #include "mtk_vcodec_enc_pm.h"
>  #include "mtk_vcodec_intr.h"
>  #include "mtk_vcodec_util.h"
> -#include "mtk_vpu.h"
> +#include "mtk_vcodec_fw.h"
>
>  module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
>  module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> @@ -101,22 +101,6 @@ static irqreturn_t mtk_vcodec_enc_lt_irq_handler(int irq, void *priv)
>         return IRQ_HANDLED;
>  }
>
> -static void mtk_vcodec_enc_reset_handler(void *priv)
> -{
> -       struct mtk_vcodec_dev *dev = priv;
> -       struct mtk_vcodec_ctx *ctx;
> -
> -       mtk_v4l2_debug(0, "Watchdog timeout!!");
> -
> -       mutex_lock(&dev->dev_mutex);
> -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> -               ctx->state = MTK_STATE_ABORT;
> -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> -                               ctx->id);
> -       }
> -       mutex_unlock(&dev->dev_mutex);
> -}
> -
>  static int fops_vcodec_open(struct file *file)
>  {
>         struct mtk_vcodec_dev *dev = video_drvdata(file);
> @@ -159,10 +143,10 @@ static int fops_vcodec_open(struct file *file)
>
>         if (v4l2_fh_is_singular(&ctx->fh)) {
>                 /*
> -                * vpu_load_firmware checks if it was loaded already and
> +                * load fireware to checks if it was loaded already and
>                  * does nothing in that case
>                  */
> -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
>                 if (ret < 0) {
>                         /*
>                          * Return 0 if downloading firmware successfully,
> @@ -173,7 +157,7 @@ static int fops_vcodec_open(struct file *file)
>                 }
>
>                 dev->enc_capability =
> -                       vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> +                       mtk_vcodec_fw_get_venc_capa(dev->fw_handler);
>                 mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
>         }
>
> @@ -235,6 +219,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>         struct mtk_vcodec_dev *dev;
>         struct video_device *vfd_enc;
>         struct resource *res;
> +       phandle rproc_phandle;
> +       enum mtk_vcodec_fw_type fw_type;
>         int i, j, ret;
>
>         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> @@ -244,19 +230,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>         INIT_LIST_HEAD(&dev->ctx_list);
>         dev->plat_dev = pdev;
>
> -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> -       if (dev->vpu_plat_dev == NULL) {
> -               mtk_v4l2_err("[VPU] vpu device in not ready");
> -               return -EPROBE_DEFER;
> +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> +                                 &rproc_phandle)) {
> +               fw_type = VPU;
> +       } else {
> +               mtk_v4l2_err("Could not get venc IPI device");
> +               return -ENODEV;
>         }
> -
> -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler,
> -                               dev, VPU_RST_ENC);
> +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_ENC);
> +       if (IS_ERR(dev->fw_handler))
> +               return PTR_ERR(dev->fw_handler);
>
>         ret = mtk_vcodec_init_enc_pm(dev);
>         if (ret < 0) {
>                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> -               return ret;
> +               goto err_enc_pm;
>         }
>
>         for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> @@ -377,6 +365,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>         v4l2_device_unregister(&dev->v4l2_dev);
>  err_res:
>         mtk_vcodec_release_enc_pm(dev);
> +err_enc_pm:
> +       mtk_vcodec_fw_release(dev->fw_handler);
>         return ret;
>  }
>
> @@ -401,6 +391,7 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
>
>         v4l2_device_unregister(&dev->v4l2_dev);
>         mtk_vcodec_release_enc_pm(dev);
> +       mtk_vcodec_fw_release(dev->fw_handler);
>         return 0;
>  }
>
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> index 3e2bfded79a6..ee22902aaa71 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> @@ -12,8 +12,6 @@
>
>  #include "mtk_vcodec_enc_pm.h"
>  #include "mtk_vcodec_util.h"
> -#include "mtk_vpu.h"
> -
>
>  int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
>  {
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> new file mode 100644
> index 000000000000..967bb100a990
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> @@ -0,0 +1,172 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#include "mtk_vcodec_fw.h"
> +#include "mtk_vcodec_util.h"
> +#include "mtk_vcodec_drv.h"
> +
> +struct mtk_vcodec_fw_ops {
> +       int (*load_firmware)(struct mtk_vcodec_fw *fw);
> +       unsigned int (*get_vdec_capa)(struct mtk_vcodec_fw *fw);
> +       unsigned int (*get_venc_capa)(struct mtk_vcodec_fw *fw);
> +       void * (*map_dm_addr)(struct mtk_vcodec_fw *fw, u32 dtcm_dmem_addr);
> +       int (*ipi_register)(struct mtk_vcodec_fw *fw, int id,
> +               mtk_vcodec_ipi_handler handler, const char *name, void *priv);
> +       int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
> +               unsigned int len, unsigned int wait);
> +};
> +
> +struct mtk_vcodec_fw {
> +       enum mtk_vcodec_fw_type type;
> +       const struct mtk_vcodec_fw_ops *ops;
> +       struct platform_device *pdev;
> +};
> +
> +static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
> +{
> +       return vpu_load_firmware(fw->pdev);
> +}
> +
> +static unsigned int mtk_vcodec_vpu_get_vdec_capa(struct mtk_vcodec_fw *fw)
> +{
> +       return vpu_get_vdec_hw_capa(fw->pdev);
> +}
> +
> +static unsigned int mtk_vcodec_vpu_get_venc_capa(struct mtk_vcodec_fw *fw)
> +{
> +       return vpu_get_venc_hw_capa(fw->pdev);
> +}
> +
> +static void *mtk_vcodec_vpu_map_dm_addr(struct mtk_vcodec_fw *fw,
> +                                       u32 dtcm_dmem_addr)
> +{
> +       return vpu_mapping_dm_addr(fw->pdev, dtcm_dmem_addr);
> +}
> +
> +static int mtk_vcodec_vpu_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
> +               mtk_vcodec_ipi_handler handler, const char *name, void *priv)
> +{
> +       /*
> +        * The handler we receive takes a void * as its first argument. We
> +        * cannot change this because it needs to be passed down to the rproc
> +        * subsystem when SCP is used. VPU takes a const argument, which is
> +        * more constrained, so the conversion below is safe.
> +        */
> +       ipi_handler_t handler_const = (ipi_handler_t)handler;
> +
> +       return vpu_ipi_register(fw->pdev, id, handler_const, name, priv);
> +}
> +
> +static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
> +               unsigned int len, unsigned int wait)
> +{
> +       return vpu_ipi_send(fw->pdev, id, buf, len);
> +}
> +
> +static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
> +       .load_firmware = mtk_vcodec_vpu_load_firmware,
> +       .get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
> +       .get_venc_capa = mtk_vcodec_vpu_get_venc_capa,
> +       .map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
> +       .ipi_register = mtk_vcodec_vpu_set_ipi_register,
> +       .ipi_send = mtk_vcodec_vpu_ipi_send,
> +};
> +
> +static void mtk_vcodec_reset_handler(void *priv)
> +{
> +       struct mtk_vcodec_dev *dev = priv;
> +       struct mtk_vcodec_ctx *ctx;
> +
> +       mtk_v4l2_err("Watchdog timeout!!");
> +
> +       mutex_lock(&dev->dev_mutex);
> +       list_for_each_entry(ctx, &dev->ctx_list, list) {
> +               ctx->state = MTK_STATE_ABORT;
> +               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> +                               ctx->id);
> +       }
> +       mutex_unlock(&dev->dev_mutex);
> +}
> +
> +struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> +                                          enum mtk_vcodec_fw_type type,
> +                                          enum rst_id rst_id)
> +{
> +       const struct mtk_vcodec_fw_ops *ops;
> +       struct mtk_vcodec_fw *fw;
> +       struct platform_device *fw_pdev = NULL;
> +
> +       switch (type) {
> +       case VPU:
> +               ops = &mtk_vcodec_vpu_msg;
> +               fw_pdev = vpu_get_plat_device(dev->plat_dev);
> +               if (!fw_pdev) {
> +                       mtk_v4l2_err("firmware device is not ready");
> +                       return ERR_PTR(-EINVAL);
> +               }
> +               vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
> +                                   dev, rst_id);
> +               break;
> +       default:
> +               mtk_v4l2_err("invalid vcodec fw type");
> +               return ERR_PTR(-EINVAL);
> +       }
> +
> +       fw = devm_kzalloc(&dev->plat_dev->dev, sizeof(*fw), GFP_KERNEL);
> +       if (!fw)
> +               return ERR_PTR(-EINVAL);
> +
> +       fw->type = type;
> +       fw->ops = ops;
> +       fw->pdev = fw_pdev;
> +
> +       return fw;
> +}
> +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_select);
> +
> +void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
> +{
> +       switch (fw->type) {
> +       case VPU:
> +               put_device(&fw->pdev->dev);
> +               break;
> +       }
> +}
> +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_release);
> +

What are these symbols exported for?

Thanks!
Ezequiel

> +int mtk_vcodec_fw_load_firmware(struct mtk_vcodec_fw *fw)
> +{
> +       return fw->ops->load_firmware(fw);
> +}
> +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_load_firmware);
> +
> +unsigned int mtk_vcodec_fw_get_vdec_capa(struct mtk_vcodec_fw *fw)
> +{
> +       return fw->ops->get_vdec_capa(fw);
> +}
> +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_get_vdec_capa);
> +
> +unsigned int mtk_vcodec_fw_get_venc_capa(struct mtk_vcodec_fw *fw)
> +{
> +       return fw->ops->get_venc_capa(fw);
> +}
> +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_get_venc_capa);
> +
> +void *mtk_vcodec_fw_map_dm_addr(struct mtk_vcodec_fw *fw, u32 mem_addr)
> +{
> +       return fw->ops->map_dm_addr(fw, mem_addr);
> +}
> +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_map_dm_addr);
> +
> +int mtk_vcodec_fw_ipi_register(struct mtk_vcodec_fw *fw, int id,
> +       mtk_vcodec_ipi_handler handler, const char *name, void *priv)
> +{
> +       return fw->ops->ipi_register(fw, id, handler, name, priv);
> +}
> +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_ipi_register);
> +
> +int mtk_vcodec_fw_ipi_send(struct mtk_vcodec_fw *fw,
> +       int id, void *buf, unsigned int len, unsigned int wait)
> +{
> +       return fw->ops->ipi_send(fw, id, buf, len, wait);
> +}
> +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_ipi_send);
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
> new file mode 100644
> index 000000000000..ff25b0c19f74
> --- /dev/null
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
> @@ -0,0 +1,36 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#ifndef _MTK_VCODEC_FW_H_
> +#define _MTK_VCODEC_FW_H_
> +
> +#include <linux/remoteproc.h>
> +
> +#include "../mtk-vpu/mtk_vpu.h"
> +
> +struct mtk_vcodec_dev;
> +
> +enum mtk_vcodec_fw_type {
> +       VPU,
> +};
> +
> +struct mtk_vcodec_fw;
> +
> +typedef void (*mtk_vcodec_ipi_handler) (void *data,
> +       unsigned int len, void *priv);
> +
> +struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> +                                          enum mtk_vcodec_fw_type type,
> +                                          enum rst_id rst_id);
> +void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw);
> +
> +int mtk_vcodec_fw_load_firmware(struct mtk_vcodec_fw *fw);
> +unsigned int mtk_vcodec_fw_get_vdec_capa(struct mtk_vcodec_fw *fw);
> +unsigned int mtk_vcodec_fw_get_venc_capa(struct mtk_vcodec_fw *fw);
> +void *mtk_vcodec_fw_map_dm_addr(struct mtk_vcodec_fw *fw, u32 mem_addr);
> +int mtk_vcodec_fw_ipi_register(struct mtk_vcodec_fw *fw, int id,
> +       mtk_vcodec_ipi_handler handler, const char *name, void *priv);
> +int mtk_vcodec_fw_ipi_send(struct mtk_vcodec_fw *fw,
> +       int id, void *buf, unsigned int len, unsigned int wait);
> +
> +#endif /* _MTK_VCODEC_FW_H_ */
> +
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> index d48f542db1a9..ac5973b6735f 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c
> @@ -9,7 +9,6 @@
>
>  #include "mtk_vcodec_drv.h"
>  #include "mtk_vcodec_util.h"
> -#include "mtk_vpu.h"
>
>  /* For encoder, this will enable logs in venc/*/
>  bool mtk_vcodec_dbg;
> diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
> index 50048c170b99..40d6e6c5ac7a 100644
> --- a/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
> +++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_h264_if.c
> @@ -281,7 +281,6 @@ static int vdec_h264_init(struct mtk_vcodec_ctx *ctx)
>         inst->ctx = ctx;
>
>         inst->vpu.id = IPI_VDEC_H264;
> -       inst->vpu.dev = ctx->dev->vpu_plat_dev;
>         inst->vpu.ctx = ctx;
>
>         err = vpu_dec_init(&inst->vpu);
> diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c
> index 6011fdd60a22..e5393f841080 100644
> --- a/drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c
> +++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp8_if.c
> @@ -400,7 +400,6 @@ static int vdec_vp8_init(struct mtk_vcodec_ctx *ctx)
>         inst->ctx = ctx;
>
>         inst->vpu.id = IPI_VDEC_VP8;
> -       inst->vpu.dev = ctx->dev->vpu_plat_dev;
>         inst->vpu.ctx = ctx;
>
>         err = vpu_dec_init(&inst->vpu);
> diff --git a/drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c
> index 257a5b5ad212..8e099b859f21 100644
> --- a/drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c
> +++ b/drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c
> @@ -795,7 +795,6 @@ static int vdec_vp9_init(struct mtk_vcodec_ctx *ctx)
>         inst->ctx = ctx;
>
>         inst->vpu.id = IPI_VDEC_VP9;
> -       inst->vpu.dev = ctx->dev->vpu_plat_dev;
>         inst->vpu.ctx = ctx;
>
>         if (vpu_dec_init(&inst->vpu)) {
> diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_base.h b/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
> index ceb4db4cb3be..e913f963b7db 100644
> --- a/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
> +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_base.h
> @@ -7,8 +7,6 @@
>  #ifndef _VDEC_DRV_BASE_
>  #define _VDEC_DRV_BASE_
>
> -#include "mtk_vcodec_drv.h"
> -
>  #include "vdec_drv_if.h"
>
>  struct vdec_common_if {
> diff --git a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> index 2e43dd4486e0..b18743b906ea 100644
> --- a/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> +++ b/drivers/media/platform/mtk-vcodec/vdec_drv_if.c
> @@ -13,7 +13,6 @@
>  #include "mtk_vcodec_dec.h"
>  #include "vdec_drv_base.h"
>  #include "mtk_vcodec_dec_pm.h"
> -#include "mtk_vpu.h"
>
>  int vdec_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
>  {
> diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
> index 948a12fd9d46..58b0e6fa8fd2 100644
> --- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
> +++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.c
> @@ -8,6 +8,7 @@
>  #include "mtk_vcodec_util.h"
>  #include "vdec_ipi_msg.h"
>  #include "vdec_vpu_if.h"
> +#include "mtk_vcodec_fw.h"
>
>  static void handle_init_ack_msg(const struct vdec_vpu_ipi_init_ack *msg)
>  {
> @@ -18,7 +19,8 @@ static void handle_init_ack_msg(const struct vdec_vpu_ipi_init_ack *msg)
>
>         /* mapping VPU address to kernel virtual address */
>         /* the content in vsi is initialized to 0 in VPU */
> -       vpu->vsi = vpu_mapping_dm_addr(vpu->dev, msg->vpu_inst_addr);
> +       vpu->vsi = mtk_vcodec_fw_map_dm_addr(vpu->ctx->dev->fw_handler,
> +                                            msg->vpu_inst_addr);
>         vpu->inst_addr = msg->vpu_inst_addr;
>
>         mtk_vcodec_debug(vpu, "- vpu_inst_addr = 0x%x", vpu->inst_addr);
> @@ -34,7 +36,7 @@ static void handle_init_ack_msg(const struct vdec_vpu_ipi_init_ack *msg)
>   * This function runs in interrupt context and it means there's an IPI MSG
>   * from VPU.
>   */
> -static void vpu_dec_ipi_handler(const void *data, unsigned int len, void *priv)
> +static void vpu_dec_ipi_handler(void *data, unsigned int len, void *priv)
>  {
>         const struct vdec_vpu_ipi_ack *msg = data;
>         struct vdec_vpu_inst *vpu = (struct vdec_vpu_inst *)
> @@ -74,7 +76,8 @@ static int vcodec_vpu_send_msg(struct vdec_vpu_inst *vpu, void *msg, int len)
>         vpu->failure = 0;
>         vpu->signaled = 0;
>
> -       err = vpu_ipi_send(vpu->dev, vpu->id, msg, len);
> +       err = mtk_vcodec_fw_ipi_send(vpu->ctx->dev->fw_handler, vpu->id, msg,
> +                                    len, 2000);
>         if (err) {
>                 mtk_vcodec_err(vpu, "send fail vpu_id=%d msg_id=%X status=%d",
>                                vpu->id, *(uint32_t *)msg, err);
> @@ -110,7 +113,8 @@ int vpu_dec_init(struct vdec_vpu_inst *vpu)
>         init_waitqueue_head(&vpu->wq);
>         vpu->handler = vpu_dec_ipi_handler;
>
> -       err = vpu_ipi_register(vpu->dev, vpu->id, vpu->handler, "vdec", NULL);
> +       err = mtk_vcodec_fw_ipi_register(vpu->ctx->dev->fw_handler, vpu->id,
> +                                        vpu->handler, "vdec", NULL);
>         if (err != 0) {
>                 mtk_vcodec_err(vpu, "vpu_ipi_register fail status=%d", err);
>                 return err;
> diff --git a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
> index f779b0676fbd..85224eb7e34b 100644
> --- a/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
> +++ b/drivers/media/platform/mtk-vcodec/vdec_vpu_if.h
> @@ -7,11 +7,13 @@
>  #ifndef _VDEC_VPU_IF_H_
>  #define _VDEC_VPU_IF_H_
>
> -#include "mtk_vpu.h"
> +#include "mtk_vcodec_fw.h"
> +
> +struct mtk_vcodec_ctx;
>
>  /**
>   * struct vdec_vpu_inst - VPU instance for video codec
> - * @ipi_id      : ipi id for each decoder
> + * @id          : ipi msg id for each decoder
>   * @vsi         : driver structure allocated by VPU side and shared to AP side
>   *                for control and info share
>   * @failure     : VPU execution result status, 0: success, others: fail
> @@ -23,15 +25,14 @@
>   * @handler     : ipi handler for each decoder
>   */
>  struct vdec_vpu_inst {
> -       enum ipi_id id;
> +       int id;
>         void *vsi;
>         int32_t failure;
>         uint32_t inst_addr;
>         unsigned int signaled;
>         struct mtk_vcodec_ctx *ctx;
> -       struct platform_device *dev;
>         wait_queue_head_t wq;
> -       ipi_handler_t handler;
> +       mtk_vcodec_ipi_handler handler;
>  };
>
>  /**
> diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> index b9624f8df0e9..7a00f050ec36 100644
> --- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> @@ -18,7 +18,6 @@
>  #include "../venc_drv_base.h"
>  #include "../venc_ipi_msg.h"
>  #include "../venc_vpu_if.h"
> -#include "mtk_vpu.h"
>
>  static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
>
> @@ -257,8 +256,11 @@ static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst)
>                  */
>                 inst->work_bufs[i].size = wb[i].size;
>                 if (i == VENC_H264_VPU_WORK_BUF_SKIP_FRAME) {
> -                       inst->work_bufs[i].va = vpu_mapping_dm_addr(
> -                               inst->vpu_inst.dev, wb[i].vpua);
> +                       struct mtk_vcodec_fw *handler;
> +
> +                       handler = inst->vpu_inst.ctx->dev->fw_handler;
> +                       inst->work_bufs[i].va =
> +                               mtk_vcodec_fw_map_dm_addr(handler, wb[i].vpua);
>                         inst->work_bufs[i].dma_addr = 0;
>                 } else {
>                         ret = mtk_vcodec_mem_alloc(inst->ctx,
> @@ -275,10 +277,12 @@ static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst)
>                          * setting in VPU side.
>                          */
>                         if (i == VENC_H264_VPU_WORK_BUF_RC_CODE) {
> +                               struct mtk_vcodec_fw *handler;
>                                 void *tmp_va;
>
> -                               tmp_va = vpu_mapping_dm_addr(inst->vpu_inst.dev,
> -                                                            wb[i].vpua);
> +                               handler = inst->vpu_inst.ctx->dev->fw_handler;
> +                               tmp_va = mtk_vcodec_fw_map_dm_addr(handler,
> +                                                                  wb[i].vpua);
>                                 memcpy(inst->work_bufs[i].va, tmp_va,
>                                        wb[i].size);
>                         }
> @@ -469,7 +473,6 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
>
>         inst->ctx = ctx;
>         inst->vpu_inst.ctx = ctx;
> -       inst->vpu_inst.dev = ctx->dev->vpu_plat_dev;
>         inst->vpu_inst.id = IPI_VENC_H264;
>         inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
>
> diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> index 8d36f0362efe..6426af514526 100644
> --- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> @@ -17,7 +17,6 @@
>  #include "../venc_drv_base.h"
>  #include "../venc_ipi_msg.h"
>  #include "../venc_vpu_if.h"
> -#include "mtk_vpu.h"
>
>  #define VENC_BITSTREAM_FRAME_SIZE 0x0098
>  #define VENC_BITSTREAM_HEADER_LEN 0x00e8
> @@ -190,10 +189,12 @@ static int vp8_enc_alloc_work_buf(struct venc_vp8_inst *inst)
>                 if (i == VENC_VP8_VPU_WORK_BUF_RC_CODE ||
>                     i == VENC_VP8_VPU_WORK_BUF_RC_CODE2 ||
>                     i == VENC_VP8_VPU_WORK_BUF_RC_CODE3) {
> +                       struct mtk_vcodec_fw *handler;
>                         void *tmp_va;
>
> -                       tmp_va = vpu_mapping_dm_addr(inst->vpu_inst.dev,
> -                                                    wb[i].vpua);
> +                       handler = inst->vpu_inst.ctx->dev->fw_handler;
> +                       tmp_va = mtk_vcodec_fw_map_dm_addr(handler,
> +                                                          wb[i].vpua);
>                         memcpy(inst->work_bufs[i].va, tmp_va, wb[i].size);
>                 }
>                 wb[i].iova = inst->work_bufs[i].dma_addr;
> @@ -334,7 +335,6 @@ static int vp8_enc_init(struct mtk_vcodec_ctx *ctx)
>
>         inst->ctx = ctx;
>         inst->vpu_inst.ctx = ctx;
> -       inst->vpu_inst.dev = ctx->dev->vpu_plat_dev;
>         inst->vpu_inst.id = IPI_VENC_VP8;
>         inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_LT_SYS);
>
> diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> index c6bb82ac2dcd..ce0bce811615 100644
> --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c
> @@ -15,7 +15,6 @@
>
>  #include "mtk_vcodec_enc.h"
>  #include "mtk_vcodec_enc_pm.h"
> -#include "mtk_vpu.h"
>
>  int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
>  {
> diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> index 9540709c1905..53854127814b 100644
> --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> @@ -4,7 +4,7 @@
>   * Author: PoChun Lin <pochun.lin@mediatek.com>
>   */
>
> -#include "mtk_vpu.h"
> +#include "mtk_vcodec_fw.h"
>  #include "venc_ipi_msg.h"
>  #include "venc_vpu_if.h"
>
> @@ -13,7 +13,8 @@ static void handle_enc_init_msg(struct venc_vpu_inst *vpu, const void *data)
>         const struct venc_vpu_ipi_msg_init *msg = data;
>
>         vpu->inst_addr = msg->vpu_inst_addr;
> -       vpu->vsi = vpu_mapping_dm_addr(vpu->dev, msg->vpu_inst_addr);
> +       vpu->vsi = mtk_vcodec_fw_map_dm_addr(vpu->ctx->dev->fw_handler,
> +                                            msg->vpu_inst_addr);
>  }
>
>  static void handle_enc_encode_msg(struct venc_vpu_inst *vpu, const void *data)
> @@ -25,7 +26,7 @@ static void handle_enc_encode_msg(struct venc_vpu_inst *vpu, const void *data)
>         vpu->is_key_frm = msg->is_key_frm;
>  }
>
> -static void vpu_enc_ipi_handler(const void *data, unsigned int len, void *priv)
> +static void vpu_enc_ipi_handler(void *data, unsigned int len, void *priv)
>  {
>         const struct venc_vpu_ipi_msg_common *msg = data;
>         struct venc_vpu_inst *vpu =
> @@ -63,12 +64,13 @@ static int vpu_enc_send_msg(struct venc_vpu_inst *vpu, void *msg,
>
>         mtk_vcodec_debug_enter(vpu);
>
> -       if (!vpu->dev) {
> +       if (!vpu->ctx->dev->fw_handler) {
>                 mtk_vcodec_err(vpu, "inst dev is NULL");
>                 return -EINVAL;
>         }
>
> -       status = vpu_ipi_send(vpu->dev, vpu->id, msg, len);
> +       status = mtk_vcodec_fw_ipi_send(vpu->ctx->dev->fw_handler, vpu->id, msg,
> +                                       len, 2000);
>         if (status) {
>                 mtk_vcodec_err(vpu, "vpu_ipi_send msg_id %x len %d fail %d",
>                                *(uint32_t *)msg, len, status);
> @@ -93,8 +95,9 @@ int vpu_enc_init(struct venc_vpu_inst *vpu)
>         vpu->signaled = 0;
>         vpu->failure = 0;
>
> -       status = vpu_ipi_register(vpu->dev, vpu->id, vpu_enc_ipi_handler,
> -                                 NULL, NULL);
> +       status = mtk_vcodec_fw_ipi_register(vpu->ctx->dev->fw_handler, vpu->id,
> +               vpu_enc_ipi_handler, "venc", NULL);
> +
>         if (status) {
>                 mtk_vcodec_err(vpu, "vpu_ipi_register fail %d", status);
>                 return -EINVAL;
> diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
> index ba301a138a5a..edd411621b68 100644
> --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
> +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
> @@ -7,7 +7,7 @@
>  #ifndef _VENC_VPU_IF_H_
>  #define _VENC_VPU_IF_H_
>
> -#include "mtk_vpu.h"
> +#include "mtk_vcodec_fw.h"
>  #include "venc_drv_if.h"
>
>  /*
> @@ -34,9 +34,8 @@ struct venc_vpu_inst {
>         int is_key_frm;
>         unsigned int inst_addr;
>         void *vsi;
> -       enum ipi_id id;
> +       int id;
>         struct mtk_vcodec_ctx *ctx;
> -       struct platform_device *dev;
>  };
>
>  int vpu_enc_init(struct venc_vpu_inst *vpu);
> --
> 2.27.0.383.g050319c2ae-goog
>

Re: [PATCH v3 02/16] dt-bindings: media: mtk-vcodec: document SCP node

[Ezequiel Garcia]

On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
>
> The mediatek codecs can use either the VPU or the SCP as their interface
> to firmware. Reflect this in the DT bindings.
>
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> index b6b5dde6abd8..7aef0a4fe207 100644
> --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> @@ -19,7 +19,9 @@ Required properties:
>  - iommus : should point to the respective IOMMU block with master port as
>    argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
>    for details.
> -- mediatek,vpu : the node of video processor unit
> +One of the two following nodes:
> +- mediatek,vpu : the node of the video processor unit, if using VPU.
> +- mediatek,scp : the noode of the SCP unit, if using SCP.
>

This interface doesn't enforce the fact only one of the two
should be present, but not both (which is the case, right?).

I hope I'm not bikeshedding here, but from an interface POV,
would it be cleaner to just have a single mediatek,coprocessor
property, and then use of_device_is_compatible
to distinguish VPU from SCP type?

Moreover, I'd argue you don't need a dt-binding change
and should just keep the current mediatek-vpu property,
and then rely on of_device_is_compatible.

Regards,
Ezequiel

>
>  Example:
> --
> 2.27.0.383.g050319c2ae-goog
>

Re: [PATCH v3 03/16] media: mtk-vcodec: add SCP firmware ops

[Ezequiel Garcia]

On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
>
> From: Yunfei Dong <yunfei.dong@mediatek.com>
>
> Add support for communicating with the SCP firmware, which will be used
> by MT8183.
>
> Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> [acourbot: refactor, cleanup and split]
> Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  drivers/media/platform/Kconfig                |  1 +
>  .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  3 +
>  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  3 +
>  .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 56 +++++++++++++++++++
>  .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  2 +
>  5 files changed, 65 insertions(+)
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index c57ee78fa99d..f0dbe048efea 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -256,6 +256,7 @@ config VIDEO_MEDIATEK_VCODEC
>         select VIDEOBUF2_DMA_CONTIG
>         select V4L2_MEM2MEM_DEV
>         select VIDEO_MEDIATEK_VPU
> +       select MTK_SCP
>         help
>             Mediatek video codec driver provides HW capability to
>             encode and decode in a range of video formats
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> index 4f07a5fcce7f..5b5765b98e57 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> @@ -225,6 +225,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>         if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
>                                   &rproc_phandle)) {
>                 fw_type = VPU;
> +       } else if (!of_property_read_u32(pdev->dev.of_node, "mediatek,scp",
> +                                        &rproc_phandle)) {
> +               fw_type = SCP;
>         } else {
>                 mtk_v4l2_err("Could not get vdec IPI device");
>                 return -ENODEV;
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> index 4340ea10afd0..42530cd01a30 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> @@ -233,6 +233,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>         if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
>                                   &rproc_phandle)) {
>                 fw_type = VPU;
> +       } else if (!of_property_read_u32(pdev->dev.of_node, "mediatek,scp",
> +                                        &rproc_phandle)) {
> +               fw_type = SCP;
>         } else {
>                 mtk_v4l2_err("Could not get venc IPI device");
>                 return -ENODEV;
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> index 967bb100a990..f2a62ea62fc6 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> @@ -19,6 +19,7 @@ struct mtk_vcodec_fw {
>         enum mtk_vcodec_fw_type type;
>         const struct mtk_vcodec_fw_ops *ops;
>         struct platform_device *pdev;
> +       struct mtk_scp *scp;
>  };
>
>  static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
> @@ -71,6 +72,48 @@ static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
>         .ipi_send = mtk_vcodec_vpu_ipi_send,
>  };
>
> +static int mtk_vcodec_scp_load_firmware(struct mtk_vcodec_fw *fw)
> +{
> +       return rproc_boot(scp_get_rproc(fw->scp));
> +}
> +
> +static unsigned int mtk_vcodec_scp_get_vdec_capa(struct mtk_vcodec_fw *fw)
> +{
> +       return scp_get_vdec_hw_capa(fw->scp);
> +}
> +
> +static unsigned int mtk_vcodec_scp_get_venc_capa(struct mtk_vcodec_fw *fw)
> +{
> +       return scp_get_venc_hw_capa(fw->scp);
> +}
> +
> +static void *mtk_vcodec_vpu_scp_dm_addr(struct mtk_vcodec_fw *fw,
> +                                       u32 dtcm_dmem_addr)
> +{
> +       return scp_mapping_dm_addr(fw->scp, dtcm_dmem_addr);
> +}
> +
> +static int mtk_vcodec_scp_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
> +               mtk_vcodec_ipi_handler handler, const char *name, void *priv)
> +{
> +       return scp_ipi_register(fw->scp, id, handler, priv);
> +}
> +
> +static int mtk_vcodec_scp_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
> +               unsigned int len, unsigned int wait)
> +{
> +       return scp_ipi_send(fw->scp, id, buf, len, wait);
> +}
> +
> +static const struct mtk_vcodec_fw_ops mtk_vcodec_rproc_msg = {
> +       .load_firmware = mtk_vcodec_scp_load_firmware,
> +       .get_vdec_capa = mtk_vcodec_scp_get_vdec_capa,
> +       .get_venc_capa = mtk_vcodec_scp_get_venc_capa,
> +       .map_dm_addr = mtk_vcodec_vpu_scp_dm_addr,
> +       .ipi_register = mtk_vcodec_scp_set_ipi_register,
> +       .ipi_send = mtk_vcodec_scp_ipi_send,
> +};
> +
>  static void mtk_vcodec_reset_handler(void *priv)
>  {
>         struct mtk_vcodec_dev *dev = priv;
> @@ -94,6 +137,7 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
>         const struct mtk_vcodec_fw_ops *ops;
>         struct mtk_vcodec_fw *fw;
>         struct platform_device *fw_pdev = NULL;
> +       struct mtk_scp *scp = NULL;
>
>         switch (type) {
>         case VPU:
> @@ -106,6 +150,14 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
>                 vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
>                                     dev, rst_id);
>                 break;
> +       case SCP:
> +               ops = &mtk_vcodec_rproc_msg;
> +               scp = scp_get(dev->plat_dev);
> +               if (!scp) {
> +                       mtk_v4l2_err("could not get vdec scp handle");
> +                       return ERR_PTR(-EPROBE_DEFER);

I suspect the EPROBE_DEFER should be returned by scp_get
itself instead.

> +               }
> +               break;
>         default:
>                 mtk_v4l2_err("invalid vcodec fw type");
>                 return ERR_PTR(-EINVAL);
> @@ -118,6 +170,7 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
>         fw->type = type;
>         fw->ops = ops;
>         fw->pdev = fw_pdev;
> +       fw->scp = scp;
>
>         return fw;
>  }
> @@ -129,6 +182,9 @@ void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
>         case VPU:
>                 put_device(&fw->pdev->dev);
>                 break;
> +       case SCP:
> +               scp_put(fw->scp);

Interestingly scp_put is a wrapper around put_device :-)
Perhaps not a reason to violate the layering.

Regards,
Ezequiel

Re: [PATCH v3 04/16] media: mtk-vcodec: venc: support SCP firmware

[Ezequiel Garcia]

Hi Alexandre,

I'm slowly making progress on the series. Here's some more comments.

On Mon, 13 Jul 2020 at 03:10, Alexandre Courbot <acourbot@chromium.org> wrote:
>
> From: Yunfei Dong <yunfei.dong@mediatek.com>
>
> Support the new extended firmware used by MT8183's encoder.
>

If you could add some more information about the MT8183 encoder
and the extended firmware interface, it would make the review
easier.

Some comments below, I have no idea how this hardware
works, so maybe I won't make sense :-)

> Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> [acourbot: refactor, cleanup and split]
> Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  .../platform/mtk-vcodec/mtk_vcodec_drv.h      | 12 +++
>  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 34 ++++---
>  .../platform/mtk-vcodec/venc/venc_h264_if.c   | 65 +++++++++++--
>  .../platform/mtk-vcodec/venc/venc_vp8_if.c    |  3 +-
>  .../media/platform/mtk-vcodec/venc_drv_if.h   |  6 ++
>  .../media/platform/mtk-vcodec/venc_ipi_msg.h  | 15 ++-
>  .../media/platform/mtk-vcodec/venc_vpu_if.c   | 97 +++++++++++++------
>  .../media/platform/mtk-vcodec/venc_vpu_if.h   |  3 +-
>  8 files changed, 181 insertions(+), 54 deletions(-)
>
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> index e132c4ec463a..45c8adfc6a0c 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -300,6 +300,17 @@ struct mtk_vcodec_ctx {
>
>  };
>
> +/**
> + * struct mtk_vcodec_enc_pdata - compatible data for each IC
> + *
> + * @uses_ext: whether the encoder uses the extended firmware messaging format
> + * @has_lt_irq: whether the encoder uses the LT irq
> + */
> +struct mtk_vcodec_enc_pdata {
> +       bool uses_ext;
> +       bool has_lt_irq;

Instead of this boolean here, I think it might be cleaner
to have a small array of strings, listing the interrupts
each variant requires.

See drivers/staging/media/hantro/rk3288_vpu_hw.c

It's future-proof and even if you don't plan to see
future works: a) we never know and b) cleaner code,
easier to maintain.

> +};
> +
>  /**
>   * struct mtk_vcodec_dev - driver data
>   * @v4l2_dev: V4L2 device to register video devices for.
> @@ -348,6 +359,7 @@ struct mtk_vcodec_dev {
>         spinlock_t irqlock;
>         struct mtk_vcodec_ctx *curr_ctx;
>         void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
> +       const struct mtk_vcodec_enc_pdata *venc_pdata;
>
>         struct mtk_vcodec_fw *fw_handler;
>
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> index 42530cd01a30..922bc8883811 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> @@ -244,6 +244,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>         if (IS_ERR(dev->fw_handler))
>                 return PTR_ERR(dev->fw_handler);
>
> +       dev->venc_pdata = of_device_get_match_data(&pdev->dev);
>         ret = mtk_vcodec_init_enc_pm(dev);
>         if (ret < 0) {
>                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> @@ -278,21 +279,24 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>                 ret = -EINVAL;
>                 goto err_res;
>         }
> +       disable_irq(dev->enc_irq);
>

I am aware this is not an issue with this patch:
requesting an interrupt and then immediately
disabling the interrupt looks racy.

Now, I haven't need this pattern myself,
but from a quick grep I think you want to do:

        irq_set_status_flags(irq, IRQ_NOAUTOEN);
        ret = devm_request_irq(dev, irq, ...

Perhaps something to fix (probably in another patch,
before this one).

> -       dev->enc_lt_irq = platform_get_irq(pdev, 1);
> -       ret = devm_request_irq(&pdev->dev,
> -                              dev->enc_lt_irq, mtk_vcodec_enc_lt_irq_handler,
> -                              0, pdev->name, dev);
> -       if (ret) {
> -               dev_err(&pdev->dev,
> -                       "Failed to install dev->enc_lt_irq %d (%d)",
> -                       dev->enc_lt_irq, ret);
> -               ret = -EINVAL;
> -               goto err_res;
> +       if (dev->venc_pdata->has_lt_irq) {
> +               dev->enc_lt_irq = platform_get_irq(pdev, 1);
> +               ret = devm_request_irq(&pdev->dev,
> +                                      dev->enc_lt_irq,
> +                                      mtk_vcodec_enc_lt_irq_handler,
> +                                      0, pdev->name, dev);
> +               if (ret) {
> +                       dev_err(&pdev->dev,
> +                               "Failed to install dev->enc_lt_irq %d (%d)",
> +                               dev->enc_lt_irq, ret);
> +                       ret = -EINVAL;
> +                       goto err_res;
> +               }
> +               disable_irq(dev->enc_lt_irq); /* VENC_LT */
>         }
>
> -       disable_irq(dev->enc_irq);
> -       disable_irq(dev->enc_lt_irq); /* VENC_LT */
>         mutex_init(&dev->enc_mutex);
>         mutex_init(&dev->dev_mutex);
>         spin_lock_init(&dev->irqlock);
> @@ -373,8 +377,12 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
>         return ret;
>  }
>
> +static const struct mtk_vcodec_enc_pdata mt8173_pdata = {
> +       .has_lt_irq = true,
> +};
> +
>  static const struct of_device_id mtk_vcodec_enc_match[] = {
> -       {.compatible = "mediatek,mt8173-vcodec-enc",},
> +       {.compatible = "mediatek,mt8173-vcodec-enc", .data = &mt8173_pdata},
>         {},
>  };
>  MODULE_DEVICE_TABLE(of, mtk_vcodec_enc_match);
> diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> index 7a00f050ec36..050787b2896c 100644
> --- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> @@ -24,6 +24,16 @@ static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
>  #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
>  #define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
>
> +/*
> + * enum venc_h264_frame_type - h264 encoder output bitstream frame type
> + */
> +enum venc_h264_frame_type {
> +       VENC_H264_IDR_FRM,
> +       VENC_H264_I_FRM,
> +       VENC_H264_P_FRM,
> +       VENC_H264_B_FRM,
> +};
> +
>  /*
>   * enum venc_h264_vpu_work_buf - h264 encoder buffer index
>   */
> @@ -137,7 +147,8 @@ struct venc_h264_inst {
>         struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
>         struct mtk_vcodec_mem pps_buf;
>         bool work_buf_allocated;
> -       unsigned int frm_cnt;
> +       u32 frm_cnt;    /* declared as u32 to properly overflow */

This looks like it's fixing some issue not related to the 8183
or the extended firmware. Perhaps you can add a
more details comment?

> +       unsigned int skip_frm_cnt;
>         unsigned int prepend_hdr;
>         struct venc_vpu_inst vpu_inst;
>         struct venc_h264_vsi *vsi;
> @@ -327,6 +338,22 @@ static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
>         return irq_status;
>  }
>
> +static int h264_frame_type(struct venc_h264_inst *inst)
> +{
> +       if ((inst->vsi->config.gop_size != 0 &&
> +               (inst->frm_cnt % inst->vsi->config.gop_size) == 0) ||
> +               (inst->frm_cnt == 0 && inst->vsi->config.gop_size == 0)) {
> +               /* IDR frame */
> +               return VENC_H264_IDR_FRM;
> +       } else if ((inst->vsi->config.intra_period != 0 &&
> +               (inst->frm_cnt % inst->vsi->config.intra_period) == 0) ||
> +               (inst->frm_cnt == 0 && inst->vsi->config.intra_period == 0)) {
> +               /* I frame */
> +               return VENC_H264_I_FRM;
> +       } else {
> +               return VENC_H264_P_FRM;  /* Note: B frames are not supported */
> +       }
> +}
>  static int h264_encode_sps(struct venc_h264_inst *inst,
>                            struct mtk_vcodec_mem *bs_buf,
>                            unsigned int *bs_size)
> @@ -337,7 +364,7 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
>         mtk_vcodec_debug_enter(inst);
>
>         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
> -                            bs_buf, bs_size);
> +                            bs_buf, bs_size, NULL);
>         if (ret)
>                 return ret;
>
> @@ -364,7 +391,7 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
>         mtk_vcodec_debug_enter(inst);
>
>         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
> -                            bs_buf, bs_size);
> +                            bs_buf, bs_size, NULL);
>         if (ret)
>                 return ret;
>
> @@ -410,13 +437,24 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
>  {
>         int ret = 0;
>         unsigned int irq_status;
> +       struct venc_frame_info frame_info;
>
>         mtk_vcodec_debug_enter(inst);
> -
> +       /* Overflowing back to 0 is ok and expected here */
> +       inst->frm_cnt++;
> +       mtk_vcodec_debug(inst, "frm_cnt++ = %d\n ", inst->frm_cnt);
> +       frame_info.frm_cnt = inst->frm_cnt;
> +       frame_info.skip_frm_cnt = inst->skip_frm_cnt;
> +       frame_info.frm_type = h264_frame_type(inst);

Ditto: Is this chunk related to the new support,
or is this addressing some current issue?

> +       mtk_vcodec_debug(inst, "frm_cnt++ = %d,skip_frm_cnt =%d,frm_type=%d.\n",
> +               frame_info.frm_cnt, frame_info.skip_frm_cnt,
> +               frame_info.frm_type);
>         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
> -                            bs_buf, bs_size);
> -       if (ret)
> +                            bs_buf, bs_size, &frame_info);
> +       if (ret) {
> +               inst->frm_cnt--;
>                 return ret;
> +       }
>
>         /*
>          * skip frame case: The skip frame buffer is composed by vpu side only,
> @@ -427,19 +465,19 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
>                 memcpy(bs_buf->va,
>                        inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
>                        *bs_size);
> -               ++inst->frm_cnt;
> +               ++inst->skip_frm_cnt;
>                 return ret;
>         }
>
>         irq_status = h264_enc_wait_venc_done(inst);
>         if (irq_status != MTK_VENC_IRQ_STATUS_FRM) {
>                 mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
> +               inst->frm_cnt--;
>                 return -EIO;
>         }
>
>         *bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
>
> -       ++inst->frm_cnt;
>         mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-",
>                          inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm);
>
> @@ -464,6 +502,7 @@ static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
>
>  static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
>  {
> +       const bool is_ext = ctx->dev->venc_pdata->uses_ext;
>         int ret = 0;
>         struct venc_h264_inst *inst;
>
> @@ -473,8 +512,9 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
>
>         inst->ctx = ctx;
>         inst->vpu_inst.ctx = ctx;
> -       inst->vpu_inst.id = IPI_VENC_H264;
> +       inst->vpu_inst.id = is_ext ? SCP_IPI_VENC_H264 : IPI_VENC_H264;
>         inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
> +       inst->frm_cnt = 0xffffffff;
>
>         mtk_vcodec_debug_enter(inst);
>
> @@ -629,7 +669,12 @@ static int h264_enc_set_param(void *handle,
>                 inst->prepend_hdr = 1;
>                 mtk_vcodec_debug(inst, "set prepend header mode");
>                 break;
> -
> +       case VENC_SET_PARAM_FORCE_INTRA:
> +       case VENC_SET_PARAM_GOP_SIZE:
> +       case VENC_SET_PARAM_INTRA_PERIOD:
> +               inst->frm_cnt = 0xffffffff;
> +               inst->skip_frm_cnt = 0;
> +               fallthrough;
>         default:
>                 ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
>                 break;
> diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> index 6426af514526..11abb191ada5 100644
> --- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> @@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
>
>         mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
>
> -       ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size);
> +       ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size,
> +                            NULL);
>         if (ret)
>                 return ret;
>
> diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> index 52fc9cc812fc..51b52625ca22 100644
> --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> @@ -92,6 +92,12 @@ struct venc_enc_param {
>         unsigned int gop_size;
>  };
>

How about a comment here documenting this struct?

> +struct venc_frame_info {
> +       unsigned int frm_cnt;           /* per frame update */
> +       unsigned int skip_frm_cnt;      /* per frame update */

I'd go for s/cnt/count, it's just 2 bytes
and in the long run it'll make our life easier.

> +       unsigned int frm_type;          /* per frame update */
> +};
> +
>  /*
>   * struct venc_frm_buf - frame buffer information used in venc_if_encode()
>   * @fb_addr: plane frame buffer addresses
> diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> index 28ee04ca6241..4cafbf92d9cd 100644
> --- a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> +++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> @@ -51,17 +51,22 @@ struct venc_ap_ipi_msg_init {
>   * @vpu_inst_addr:     VPU encoder instance addr
>   *                     (struct venc_vp8_vsi/venc_h264_vsi *)
>   * @param_id:  parameter id (venc_set_param_type)
> - * @data_item: number of items in the data array
> + * @num_data:  number of items in the data array
>   * @data[8]:   data array to store the set parameters
>   */
>  struct venc_ap_ipi_msg_set_param {
>         uint32_t msg_id;
>         uint32_t vpu_inst_addr;
>         uint32_t param_id;
> -       uint32_t data_item;
> +       uint32_t num_data;

This change looks either out of place, or unneeded.
Or I'm missing something.

>         uint32_t data[8];
>  };
>
> +struct venc_ap_ipi_msg_set_param_ext {
> +       struct venc_ap_ipi_msg_set_param base;
> +       uint32_t data_ext[24];
> +};
> +
>  /**
>   * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
>   * @msg_id:    message id (AP_IPIMSG_XXX_ENC_ENCODE)
> @@ -82,6 +87,12 @@ struct venc_ap_ipi_msg_enc {
>         uint32_t bs_size;
>  };
>

Ditto.

> +struct venc_ap_ipi_msg_enc_ext {
> +       struct venc_ap_ipi_msg_enc base;
> +       uint32_t data_item;
> +       uint32_t data[32];
> +};
> +
>  /**
>   * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
>   * @msg_id:    message id (AP_IPIMSG_XXX_ENC_DEINIT)
> diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> index 53854127814b..6c77bf025172 100644
> --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> @@ -116,49 +116,81 @@ int vpu_enc_init(struct venc_vpu_inst *vpu)
>         return 0;
>  }
>
> +static unsigned int venc_enc_param_crop_right(struct venc_vpu_inst *vpu,
> +                                             struct venc_enc_param *enc_prm)
> +{
> +       unsigned int img_crop_right = enc_prm->buf_width - enc_prm->width;
> +
> +       return img_crop_right % 16;
> +}
> +
> +static unsigned int venc_enc_param_crop_bottom(struct venc_enc_param *enc_prm)
> +{
> +       return round_up(enc_prm->height, 16) - enc_prm->height;
> +}
> +
> +static unsigned int venc_enc_param_num_mb(struct venc_enc_param *enc_prm)
> +{
> +       return DIV_ROUND_UP(enc_prm->width, 16) *
> +              DIV_ROUND_UP(enc_prm->height, 16);

^^^^
You could define the macroblock size in a macro and
use it in these various places.

> +}
> +
>  int vpu_enc_set_param(struct venc_vpu_inst *vpu,
>                       enum venc_set_param_type id,
>                       struct venc_enc_param *enc_param)
>  {
> -       struct venc_ap_ipi_msg_set_param out;
> +       const bool is_ext = vpu->ctx->dev->venc_pdata->uses_ext;
> +       size_t msg_size = is_ext ?

How about you use vpu->id to check for is_ext?

I feel like querying the "is_ext" property like this
is a layering violation.

Or maybe better some firmware interface type
should be set in struct venc_vpu_inst,
so the venc_vpu_if driver is aware of its own type.

Hopefully I'm making sense :-)

> +               sizeof(struct venc_ap_ipi_msg_set_param_ext) :
> +               sizeof(struct venc_ap_ipi_msg_set_param);
> +       struct venc_ap_ipi_msg_set_param_ext out;
>
>         mtk_vcodec_debug(vpu, "id %d ->", id);
>
>         memset(&out, 0, sizeof(out));
> -       out.msg_id = AP_IPIMSG_ENC_SET_PARAM;
> -       out.vpu_inst_addr = vpu->inst_addr;
> -       out.param_id = id;
> +       out.base.msg_id = AP_IPIMSG_ENC_SET_PARAM;
> +       out.base.vpu_inst_addr = vpu->inst_addr;
> +       out.base.param_id = id;
>         switch (id) {
>         case VENC_SET_PARAM_ENC:
> -               out.data_item = 0;
> +               if (is_ext) {
> +                       out.base.num_data = 3;
> +                       out.base.data[0] =
> +                               venc_enc_param_crop_right(vpu, enc_param);
> +                       out.base.data[1] =
> +                               venc_enc_param_crop_bottom(enc_param);
> +                       out.base.data[2] = venc_enc_param_num_mb(enc_param);
> +               } else {
> +                       out.base.num_data = 0;
> +               }
>                 break;
>         case VENC_SET_PARAM_FORCE_INTRA:
> -               out.data_item = 0;
> +               out.base.num_data = 0;
>                 break;
>         case VENC_SET_PARAM_ADJUST_BITRATE:
> -               out.data_item = 1;
> -               out.data[0] = enc_param->bitrate;
> +               out.base.num_data = 1;
> +               out.base.data[0] = enc_param->bitrate;
>                 break;
>         case VENC_SET_PARAM_ADJUST_FRAMERATE:
> -               out.data_item = 1;
> -               out.data[0] = enc_param->frm_rate;
> +               out.base.num_data = 1;
> +               out.base.data[0] = enc_param->frm_rate;
>                 break;
>         case VENC_SET_PARAM_GOP_SIZE:
> -               out.data_item = 1;
> -               out.data[0] = enc_param->gop_size;
> +               out.base.num_data = 1;
> +               out.base.data[0] = enc_param->gop_size;
>                 break;
>         case VENC_SET_PARAM_INTRA_PERIOD:
> -               out.data_item = 1;
> -               out.data[0] = enc_param->intra_period;
> +               out.base.num_data = 1;
> +               out.base.data[0] = enc_param->intra_period;
>                 break;
>         case VENC_SET_PARAM_SKIP_FRAME:
> -               out.data_item = 0;
> +               out.base.num_data = 0;
>                 break;
>         default:
>                 mtk_vcodec_err(vpu, "id %d not supported", id);
>                 return -EINVAL;
>         }
> -       if (vpu_enc_send_msg(vpu, &out, sizeof(out))) {
> +       if (vpu_enc_send_msg(vpu, &out, msg_size)) {
>                 mtk_vcodec_err(vpu,
>                                "AP_IPIMSG_ENC_SET_PARAM %d fail", id);
>                 return -EINVAL;
> @@ -172,33 +204,44 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
>  int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
>                    struct venc_frm_buf *frm_buf,
>                    struct mtk_vcodec_mem *bs_buf,
> -                  unsigned int *bs_size)
> +                  unsigned int *bs_size,
> +                  struct venc_frame_info *frame_info)
>  {
> -       struct venc_ap_ipi_msg_enc out;
> +       const bool is_ext = vpu->ctx->dev->venc_pdata->uses_ext;
> +       size_t msg_size = is_ext ?
> +               sizeof(struct venc_ap_ipi_msg_enc_ext) :
> +               sizeof(struct venc_ap_ipi_msg_enc);
> +       struct venc_ap_ipi_msg_enc_ext out;
>
>         mtk_vcodec_debug(vpu, "bs_mode %d ->", bs_mode);
>
>         memset(&out, 0, sizeof(out));
> -       out.msg_id = AP_IPIMSG_ENC_ENCODE;
> -       out.vpu_inst_addr = vpu->inst_addr;
> -       out.bs_mode = bs_mode;
> +       out.base.msg_id = AP_IPIMSG_ENC_ENCODE;
> +       out.base.vpu_inst_addr = vpu->inst_addr;
> +       out.base.bs_mode = bs_mode;
>         if (frm_buf) {
>                 if ((frm_buf->fb_addr[0].dma_addr % 16 == 0) &&
>                     (frm_buf->fb_addr[1].dma_addr % 16 == 0) &&
>                     (frm_buf->fb_addr[2].dma_addr % 16 == 0)) {
> -                       out.input_addr[0] = frm_buf->fb_addr[0].dma_addr;
> -                       out.input_addr[1] = frm_buf->fb_addr[1].dma_addr;
> -                       out.input_addr[2] = frm_buf->fb_addr[2].dma_addr;
> +                       out.base.input_addr[0] = frm_buf->fb_addr[0].dma_addr;
> +                       out.base.input_addr[1] = frm_buf->fb_addr[1].dma_addr;
> +                       out.base.input_addr[2] = frm_buf->fb_addr[2].dma_addr;
>                 } else {
>                         mtk_vcodec_err(vpu, "dma_addr not align to 16");
>                         return -EINVAL;
>                 }
>         }
>         if (bs_buf) {
> -               out.bs_addr = bs_buf->dma_addr;
> -               out.bs_size = bs_buf->size;
> +               out.base.bs_addr = bs_buf->dma_addr;
> +               out.base.bs_size = bs_buf->size;
>         }
> -       if (vpu_enc_send_msg(vpu, &out, sizeof(out))) {
> +       if (is_ext && frame_info) {
> +               out.data_item = 3;
> +               out.data[0] = frame_info->frm_cnt;
> +               out.data[1] = frame_info->skip_frm_cnt;
> +               out.data[2] = frame_info->frm_type;
> +       }
> +       if (vpu_enc_send_msg(vpu, &out, msg_size)) {
>                 mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail",
>                                bs_mode);
>                 return -EINVAL;
> diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
> index edd411621b68..f9be9cab7ff7 100644
> --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
> +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h
> @@ -45,7 +45,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
>  int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
>                    struct venc_frm_buf *frm_buf,
>                    struct mtk_vcodec_mem *bs_buf,
> -                  unsigned int *bs_size);
> +                  unsigned int *bs_size,
> +                  struct venc_frame_info *frame_info);
>  int vpu_enc_deinit(struct venc_vpu_inst *vpu);
>
>  #endif
> --
> 2.27.0.383.g050319c2ae-goog
>

Re: [PATCH v3 07/16] media: mtk-vcodec: venc: specify supported formats per-chip

[Ezequiel Garcia]

Hi Alexandre,

Last review on my side, this series looks mostly good.

On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
>
> Different chips have different supported bitrate ranges. Move the list

s/bitrate ranges/formats

> of supported formats to the platform data, and split the output and
> capture formats into two lists to make it easier to find the default
> format for each queue.
>
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> ---
>  .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   8 ++
>  .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 122 +++++++-----------
>  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  40 ++++++
>  3 files changed, 95 insertions(+), 75 deletions(-)
>
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> index b8f913de8d80..59b4b750666b 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> @@ -313,6 +313,10 @@ enum mtk_chip {
>   * @has_lt_irq: whether the encoder uses the LT irq
>   * @min_birate: minimum supported encoding bitrate
>   * @max_bitrate: maximum supported encoding bitrate
> + * @capture_formats: array of supported capture formats
> + * @num_capture_formats: number of entries in capture_formats
> + * @output_formats: array of supported output formats
> + * @num_output_formats: number of entries in output_formats
>   */
>  struct mtk_vcodec_enc_pdata {
>         enum mtk_chip chip;
> @@ -321,6 +325,10 @@ struct mtk_vcodec_enc_pdata {
>         bool has_lt_irq;
>         unsigned long min_bitrate;
>         unsigned long max_bitrate;
> +       const struct mtk_video_fmt *capture_formats;
> +       size_t num_capture_formats;
> +       const struct mtk_video_fmt *output_formats;
> +       size_t num_output_formats;
>  };
>
>  /**
> diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> index 50ba9da59153..05743a745a11 100644
> --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> @@ -23,47 +23,9 @@
>  #define DFT_CFG_WIDTH  MTK_VENC_MIN_W
>  #define DFT_CFG_HEIGHT MTK_VENC_MIN_H
>  #define MTK_MAX_CTRLS_HINT     20
> -#define OUT_FMT_IDX            0
> -#define CAP_FMT_IDX            4
> -
>
>  static void mtk_venc_worker(struct work_struct *work);
>
> -static const struct mtk_video_fmt mtk_video_formats[] = {
> -       {
> -               .fourcc = V4L2_PIX_FMT_NV12M,
> -               .type = MTK_FMT_FRAME,
> -               .num_planes = 2,

Again, not an issue with your patch, so feel free to ignore this.

You may avoid keeping track of num_planes (or any other
property of the pixel format) and use v4l2_fill_pixfmt_mp
to get all the information you need.

Thanks,
Ezequiel

Re: [PATCH v3 01/16] media: mtk-vcodec: abstract firmware interface

[Alexandre Courbot]

On Thu, Jul 23, 2020 at 6:23 AM Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
> >
> > From: Yunfei Dong <yunfei.dong@mediatek.com>
> >
> > MT8183's codec firwmare is run by a different remote processor from
> > MT8173. While the firmware interface is basically the same, the way to
> > invoke it differs. Abstract all firmware calls under a layer that will
> > allow us to handle both firmware types transparently.
> >
> > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > [acourbot: refactor, cleanup and split]
> > Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > [pihsun: fix error path and add mtk_vcodec_fw_release]
> > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > Reviewed-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/mtk-vcodec/Makefile    |   4 +-
> >  .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  50 ++---
> >  .../platform/mtk-vcodec/mtk_vcodec_dec_pm.c   |   1 -
> >  .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   5 +-
> >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  47 ++---
> >  .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   |   2 -
> >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 172 ++++++++++++++++++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  36 ++++
> >  .../platform/mtk-vcodec/mtk_vcodec_util.c     |   1 -
> >  .../platform/mtk-vcodec/vdec/vdec_h264_if.c   |   1 -
> >  .../platform/mtk-vcodec/vdec/vdec_vp8_if.c    |   1 -
> >  .../platform/mtk-vcodec/vdec/vdec_vp9_if.c    |   1 -
> >  .../media/platform/mtk-vcodec/vdec_drv_base.h |   2 -
> >  .../media/platform/mtk-vcodec/vdec_drv_if.c   |   1 -
> >  .../media/platform/mtk-vcodec/vdec_vpu_if.c   |  12 +-
> >  .../media/platform/mtk-vcodec/vdec_vpu_if.h   |  11 +-
> >  .../platform/mtk-vcodec/venc/venc_h264_if.c   |  15 +-
> >  .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   8 +-
> >  .../media/platform/mtk-vcodec/venc_drv_if.c   |   1 -
> >  .../media/platform/mtk-vcodec/venc_vpu_if.c   |  17 +-
> >  .../media/platform/mtk-vcodec/venc_vpu_if.h   |   5 +-
> >  21 files changed, 290 insertions(+), 103 deletions(-)
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
> >
> > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > index 37b94b555fa1..b8636119ed0a 100644
> > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > @@ -12,7 +12,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
> >                 vdec_vpu_if.o \
> >                 mtk_vcodec_dec.o \
> >                 mtk_vcodec_dec_pm.o \
> > -
> > +               mtk_vcodec_fw.o
> >
> >  mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> >                 venc/venc_h264_if.o \
> > @@ -25,5 +25,3 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> >
> >  mtk-vcodec-common-y := mtk_vcodec_intr.o \
> >                 mtk_vcodec_util.o\
> > -
> > -ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > index 97a1b6664c20..4f07a5fcce7f 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > @@ -20,7 +20,7 @@
> >  #include "mtk_vcodec_dec_pm.h"
> >  #include "mtk_vcodec_intr.h"
> >  #include "mtk_vcodec_util.h"
> > -#include "mtk_vpu.h"
> > +#include "mtk_vcodec_fw.h"
> >
> >  #define VDEC_HW_ACTIVE 0x10
> >  #define VDEC_IRQ_CFG   0x11
> > @@ -77,22 +77,6 @@ static irqreturn_t mtk_vcodec_dec_irq_handler(int irq, void *priv)
> >         return IRQ_HANDLED;
> >  }
> >
> > -static void mtk_vcodec_dec_reset_handler(void *priv)
> > -{
> > -       struct mtk_vcodec_dev *dev = priv;
> > -       struct mtk_vcodec_ctx *ctx;
> > -
> > -       mtk_v4l2_err("Watchdog timeout!!");
> > -
> > -       mutex_lock(&dev->dev_mutex);
> > -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > -               ctx->state = MTK_STATE_ABORT;
> > -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR",
> > -                               ctx->id);
> > -       }
> > -       mutex_unlock(&dev->dev_mutex);
> > -}
> > -
> >  static int fops_vcodec_open(struct file *file)
> >  {
> >         struct mtk_vcodec_dev *dev = video_drvdata(file);
> > @@ -144,21 +128,20 @@ static int fops_vcodec_open(struct file *file)
> >         if (v4l2_fh_is_singular(&ctx->fh)) {
> >                 mtk_vcodec_dec_pw_on(&dev->pm);
> >                 /*
> > -                * vpu_load_firmware checks if it was loaded already and
> > -                * does nothing in that case
> > +                * Does nothing if firmware was already loaded.
> >                  */
> > -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
> >                 if (ret < 0) {
> >                         /*
> >                          * Return 0 if downloading firmware successfully,
> >                          * otherwise it is failed
> >                          */
> > -                       mtk_v4l2_err("vpu_load_firmware failed!");
> > +                       mtk_v4l2_err("failed to load firmware!");
> >                         goto err_load_fw;
> >                 }
> >
> >                 dev->dec_capability =
> > -                       vpu_get_vdec_hw_capa(dev->vpu_plat_dev);
> > +                       mtk_vcodec_fw_get_vdec_capa(dev->fw_handler);
> >                 mtk_v4l2_debug(0, "decoder capability %x", dev->dec_capability);
> >         }
> >
> > @@ -228,6 +211,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> >         struct mtk_vcodec_dev *dev;
> >         struct video_device *vfd_dec;
> >         struct resource *res;
> > +       phandle rproc_phandle;
> > +       enum mtk_vcodec_fw_type fw_type;
> >         int i, ret;
> >
> >         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > @@ -237,19 +222,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> >         INIT_LIST_HEAD(&dev->ctx_list);
> >         dev->plat_dev = pdev;
> >
> > -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > -       if (dev->vpu_plat_dev == NULL) {
> > -               mtk_v4l2_err("[VPU] vpu device in not ready");
> > -               return -EPROBE_DEFER;
> > +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > +                                 &rproc_phandle)) {
> > +               fw_type = VPU;
> > +       } else {
> > +               mtk_v4l2_err("Could not get vdec IPI device");
> > +               return -ENODEV;
> >         }
> > -
> > -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_dec_reset_handler,
> > -                       dev, VPU_RST_DEC);
> > +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_DEC);
> > +       if (IS_ERR(dev->fw_handler))
> > +               return PTR_ERR(dev->fw_handler);
> >
> >         ret = mtk_vcodec_init_dec_pm(dev);
> >         if (ret < 0) {
> >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source");
> > -               return ret;
> > +               goto err_dec_pm;
> >         }
> >
> >         for (i = 0; i < NUM_MAX_VDEC_REG_BASE; i++) {
> > @@ -352,6 +339,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> >         v4l2_device_unregister(&dev->v4l2_dev);
> >  err_res:
> >         mtk_vcodec_release_dec_pm(dev);
> > +err_dec_pm:
> > +       mtk_vcodec_fw_release(dev->fw_handler);
> >         return ret;
> >  }
> >
> > @@ -376,6 +365,7 @@ static int mtk_vcodec_dec_remove(struct platform_device *pdev)
> >
> >         v4l2_device_unregister(&dev->v4l2_dev);
> >         mtk_vcodec_release_dec_pm(dev);
> > +       mtk_vcodec_fw_release(dev->fw_handler);
> >         return 0;
> >  }
> >
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > index 5a6ec8fb52da..36dfe3fc056a 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > @@ -12,7 +12,6 @@
> >
> >  #include "mtk_vcodec_dec_pm.h"
> >  #include "mtk_vcodec_util.h"
> > -#include "mtk_vpu.h"
> >
> >  int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *mtkdev)
> >  {
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > index 9fd56dee7fd1..e132c4ec463a 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > @@ -309,13 +309,13 @@ struct mtk_vcodec_ctx {
> >   * @m2m_dev_dec: m2m device for decoder
> >   * @m2m_dev_enc: m2m device for encoder.
> >   * @plat_dev: platform device
> > - * @vpu_plat_dev: mtk vpu platform device
> >   * @ctx_list: list of struct mtk_vcodec_ctx
> >   * @irqlock: protect data access by irq handler and work thread
> >   * @curr_ctx: The context that is waiting for codec hardware
> >   *
> >   * @reg_base: Mapped address of MTK Vcodec registers.
> >   *
> > + * @fw_handler: used to communicate with the firmware.
> >   * @id_counter: used to identify current opened instance
> >   *
> >   * @encode_workqueue: encode work queue
> > @@ -344,12 +344,13 @@ struct mtk_vcodec_dev {
> >         struct v4l2_m2m_dev *m2m_dev_dec;
> >         struct v4l2_m2m_dev *m2m_dev_enc;
> >         struct platform_device *plat_dev;
> > -       struct platform_device *vpu_plat_dev;
> >         struct list_head ctx_list;
> >         spinlock_t irqlock;
> >         struct mtk_vcodec_ctx *curr_ctx;
> >         void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
> >
> > +       struct mtk_vcodec_fw *fw_handler;
> > +
> >         unsigned long id_counter;
> >
> >         struct workqueue_struct *decode_workqueue;
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > index 4d31f1ed113f..4340ea10afd0 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > @@ -21,7 +21,7 @@
> >  #include "mtk_vcodec_enc_pm.h"
> >  #include "mtk_vcodec_intr.h"
> >  #include "mtk_vcodec_util.h"
> > -#include "mtk_vpu.h"
> > +#include "mtk_vcodec_fw.h"
> >
> >  module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> >  module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> > @@ -101,22 +101,6 @@ static irqreturn_t mtk_vcodec_enc_lt_irq_handler(int irq, void *priv)
> >         return IRQ_HANDLED;
> >  }
> >
> > -static void mtk_vcodec_enc_reset_handler(void *priv)
> > -{
> > -       struct mtk_vcodec_dev *dev = priv;
> > -       struct mtk_vcodec_ctx *ctx;
> > -
> > -       mtk_v4l2_debug(0, "Watchdog timeout!!");
> > -
> > -       mutex_lock(&dev->dev_mutex);
> > -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > -               ctx->state = MTK_STATE_ABORT;
> > -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> > -                               ctx->id);
> > -       }
> > -       mutex_unlock(&dev->dev_mutex);
> > -}
> > -
> >  static int fops_vcodec_open(struct file *file)
> >  {
> >         struct mtk_vcodec_dev *dev = video_drvdata(file);
> > @@ -159,10 +143,10 @@ static int fops_vcodec_open(struct file *file)
> >
> >         if (v4l2_fh_is_singular(&ctx->fh)) {
> >                 /*
> > -                * vpu_load_firmware checks if it was loaded already and
> > +                * load fireware to checks if it was loaded already and
> >                  * does nothing in that case
> >                  */
> > -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
> >                 if (ret < 0) {
> >                         /*
> >                          * Return 0 if downloading firmware successfully,
> > @@ -173,7 +157,7 @@ static int fops_vcodec_open(struct file *file)
> >                 }
> >
> >                 dev->enc_capability =
> > -                       vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > +                       mtk_vcodec_fw_get_venc_capa(dev->fw_handler);
> >                 mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> >         }
> >
> > @@ -235,6 +219,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> >         struct mtk_vcodec_dev *dev;
> >         struct video_device *vfd_enc;
> >         struct resource *res;
> > +       phandle rproc_phandle;
> > +       enum mtk_vcodec_fw_type fw_type;
> >         int i, j, ret;
> >
> >         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > @@ -244,19 +230,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> >         INIT_LIST_HEAD(&dev->ctx_list);
> >         dev->plat_dev = pdev;
> >
> > -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > -       if (dev->vpu_plat_dev == NULL) {
> > -               mtk_v4l2_err("[VPU] vpu device in not ready");
> > -               return -EPROBE_DEFER;
> > +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > +                                 &rproc_phandle)) {
> > +               fw_type = VPU;
> > +       } else {
> > +               mtk_v4l2_err("Could not get venc IPI device");
> > +               return -ENODEV;
> >         }
> > -
> > -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler,
> > -                               dev, VPU_RST_ENC);
> > +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_ENC);
> > +       if (IS_ERR(dev->fw_handler))
> > +               return PTR_ERR(dev->fw_handler);
> >
> >         ret = mtk_vcodec_init_enc_pm(dev);
> >         if (ret < 0) {
> >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> > -               return ret;
> > +               goto err_enc_pm;
> >         }
> >
> >         for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > @@ -377,6 +365,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> >         v4l2_device_unregister(&dev->v4l2_dev);
> >  err_res:
> >         mtk_vcodec_release_enc_pm(dev);
> > +err_enc_pm:
> > +       mtk_vcodec_fw_release(dev->fw_handler);
> >         return ret;
> >  }
> >
> > @@ -401,6 +391,7 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
> >
> >         v4l2_device_unregister(&dev->v4l2_dev);
> >         mtk_vcodec_release_enc_pm(dev);
> > +       mtk_vcodec_fw_release(dev->fw_handler);
> >         return 0;
> >  }
> >
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > index 3e2bfded79a6..ee22902aaa71 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > @@ -12,8 +12,6 @@
> >
> >  #include "mtk_vcodec_enc_pm.h"
> >  #include "mtk_vcodec_util.h"
> > -#include "mtk_vpu.h"
> > -
> >
> >  int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
> >  {
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > new file mode 100644
> > index 000000000000..967bb100a990
> > --- /dev/null
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > @@ -0,0 +1,172 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +#include "mtk_vcodec_fw.h"
> > +#include "mtk_vcodec_util.h"
> > +#include "mtk_vcodec_drv.h"
> > +
> > +struct mtk_vcodec_fw_ops {
> > +       int (*load_firmware)(struct mtk_vcodec_fw *fw);
> > +       unsigned int (*get_vdec_capa)(struct mtk_vcodec_fw *fw);
> > +       unsigned int (*get_venc_capa)(struct mtk_vcodec_fw *fw);
> > +       void * (*map_dm_addr)(struct mtk_vcodec_fw *fw, u32 dtcm_dmem_addr);
> > +       int (*ipi_register)(struct mtk_vcodec_fw *fw, int id,
> > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv);
> > +       int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
> > +               unsigned int len, unsigned int wait);
> > +};
> > +
> > +struct mtk_vcodec_fw {
> > +       enum mtk_vcodec_fw_type type;
> > +       const struct mtk_vcodec_fw_ops *ops;
> > +       struct platform_device *pdev;
> > +};
> > +
> > +static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
> > +{
> > +       return vpu_load_firmware(fw->pdev);
> > +}
> > +
> > +static unsigned int mtk_vcodec_vpu_get_vdec_capa(struct mtk_vcodec_fw *fw)
> > +{
> > +       return vpu_get_vdec_hw_capa(fw->pdev);
> > +}
> > +
> > +static unsigned int mtk_vcodec_vpu_get_venc_capa(struct mtk_vcodec_fw *fw)
> > +{
> > +       return vpu_get_venc_hw_capa(fw->pdev);
> > +}
> > +
> > +static void *mtk_vcodec_vpu_map_dm_addr(struct mtk_vcodec_fw *fw,
> > +                                       u32 dtcm_dmem_addr)
> > +{
> > +       return vpu_mapping_dm_addr(fw->pdev, dtcm_dmem_addr);
> > +}
> > +
> > +static int mtk_vcodec_vpu_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
> > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv)
> > +{
> > +       /*
> > +        * The handler we receive takes a void * as its first argument. We
> > +        * cannot change this because it needs to be passed down to the rproc
> > +        * subsystem when SCP is used. VPU takes a const argument, which is
> > +        * more constrained, so the conversion below is safe.
> > +        */
> > +       ipi_handler_t handler_const = (ipi_handler_t)handler;
> > +
> > +       return vpu_ipi_register(fw->pdev, id, handler_const, name, priv);
> > +}
> > +
> > +static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
> > +               unsigned int len, unsigned int wait)
> > +{
> > +       return vpu_ipi_send(fw->pdev, id, buf, len);
> > +}
> > +
> > +static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
> > +       .load_firmware = mtk_vcodec_vpu_load_firmware,
> > +       .get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
> > +       .get_venc_capa = mtk_vcodec_vpu_get_venc_capa,
> > +       .map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
> > +       .ipi_register = mtk_vcodec_vpu_set_ipi_register,
> > +       .ipi_send = mtk_vcodec_vpu_ipi_send,
> > +};
> > +
> > +static void mtk_vcodec_reset_handler(void *priv)
> > +{
> > +       struct mtk_vcodec_dev *dev = priv;
> > +       struct mtk_vcodec_ctx *ctx;
> > +
> > +       mtk_v4l2_err("Watchdog timeout!!");
> > +
> > +       mutex_lock(&dev->dev_mutex);
> > +       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > +               ctx->state = MTK_STATE_ABORT;
> > +               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> > +                               ctx->id);
> > +       }
> > +       mutex_unlock(&dev->dev_mutex);
> > +}
> > +
> > +struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> > +                                          enum mtk_vcodec_fw_type type,
> > +                                          enum rst_id rst_id)
> > +{
> > +       const struct mtk_vcodec_fw_ops *ops;
> > +       struct mtk_vcodec_fw *fw;
> > +       struct platform_device *fw_pdev = NULL;
> > +
> > +       switch (type) {
> > +       case VPU:
> > +               ops = &mtk_vcodec_vpu_msg;
> > +               fw_pdev = vpu_get_plat_device(dev->plat_dev);
> > +               if (!fw_pdev) {
> > +                       mtk_v4l2_err("firmware device is not ready");
> > +                       return ERR_PTR(-EINVAL);
> > +               }
> > +               vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
> > +                                   dev, rst_id);
> > +               break;
> > +       default:
> > +               mtk_v4l2_err("invalid vcodec fw type");
> > +               return ERR_PTR(-EINVAL);
> > +       }
> > +
> > +       fw = devm_kzalloc(&dev->plat_dev->dev, sizeof(*fw), GFP_KERNEL);
> > +       if (!fw)
> > +               return ERR_PTR(-EINVAL);
> > +
> > +       fw->type = type;
> > +       fw->ops = ops;
> > +       fw->pdev = fw_pdev;
> > +
> > +       return fw;
> > +}
> > +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_select);
> > +
> > +void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
> > +{
> > +       switch (fw->type) {
> > +       case VPU:
> > +               put_device(&fw->pdev->dev);
> > +               break;
> > +       }
> > +}
> > +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_release);
> > +
>
> What are these symbols exported for?

This driver is made of three modules: mtk_vcodec_common,
mtk_vcodec_dec and mtk_vcodec_enc. These functions are used by both
the encoder and decoder module, so they need to be exported.

There is mistake in the patch though that I noticed while
investigating your comment: the file defining these functions
(mtk_vcodec_fw.c) is linked against the mtk_vcodec_dec module while it
should be part of mtk_vcodec_common, so thanks for asking. :)

Re: [PATCH v3 02/16] dt-bindings: media: mtk-vcodec: document SCP node

[Alexandre Courbot]

On Thu, Jul 23, 2020 at 6:37 AM Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
> >
> > The mediatek codecs can use either the VPU or the SCP as their interface
> > to firmware. Reflect this in the DT bindings.
> >
> > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > index b6b5dde6abd8..7aef0a4fe207 100644
> > --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > @@ -19,7 +19,9 @@ Required properties:
> >  - iommus : should point to the respective IOMMU block with master port as
> >    argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> >    for details.
> > -- mediatek,vpu : the node of video processor unit
> > +One of the two following nodes:
> > +- mediatek,vpu : the node of the video processor unit, if using VPU.
> > +- mediatek,scp : the noode of the SCP unit, if using SCP.
> >
>
> This interface doesn't enforce the fact only one of the two
> should be present, but not both (which is the case, right?).

That's correct.

>
> I hope I'm not bikeshedding here, but from an interface POV,
> would it be cleaner to just have a single mediatek,coprocessor
> property, and then use of_device_is_compatible
> to distinguish VPU from SCP type?

From an interface point of view maybe, however doing so would
introduce a backward-incompatible change with the existing MT8173
bindings. I also feel like it is less error-prone to have the property
explicitly state what it is expecting at the other end of the phandle
(vpu or scp) instead of the more generic "coprocessor".

>
> Moreover, I'd argue you don't need a dt-binding change
> and should just keep the current mediatek-vpu property,
> and then rely on of_device_is_compatible.

VPU and SCP are different kinds of processors, so I'm not sure whether
it is desirable to use VPU interchangeably like this. Note that I'm
not strongly against it either, but for things like bindings I tend to
prefer precise language to avoid confusions.

Re: [PATCH v3 03/16] media: mtk-vcodec: add SCP firmware ops

[Alexandre Courbot]

On Thu, Jul 23, 2020 at 6:40 AM Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
> >
> > From: Yunfei Dong <yunfei.dong@mediatek.com>
> >
> > Add support for communicating with the SCP firmware, which will be used
> > by MT8183.
> >
> > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > [acourbot: refactor, cleanup and split]
> > Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  drivers/media/platform/Kconfig                |  1 +
> >  .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  3 +
> >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  3 +
> >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 56 +++++++++++++++++++
> >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  2 +
> >  5 files changed, 65 insertions(+)
> >
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index c57ee78fa99d..f0dbe048efea 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -256,6 +256,7 @@ config VIDEO_MEDIATEK_VCODEC
> >         select VIDEOBUF2_DMA_CONTIG
> >         select V4L2_MEM2MEM_DEV
> >         select VIDEO_MEDIATEK_VPU
> > +       select MTK_SCP
> >         help
> >             Mediatek video codec driver provides HW capability to
> >             encode and decode in a range of video formats
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > index 4f07a5fcce7f..5b5765b98e57 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > @@ -225,6 +225,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> >         if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> >                                   &rproc_phandle)) {
> >                 fw_type = VPU;
> > +       } else if (!of_property_read_u32(pdev->dev.of_node, "mediatek,scp",
> > +                                        &rproc_phandle)) {
> > +               fw_type = SCP;
> >         } else {
> >                 mtk_v4l2_err("Could not get vdec IPI device");
> >                 return -ENODEV;
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > index 4340ea10afd0..42530cd01a30 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > @@ -233,6 +233,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> >         if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> >                                   &rproc_phandle)) {
> >                 fw_type = VPU;
> > +       } else if (!of_property_read_u32(pdev->dev.of_node, "mediatek,scp",
> > +                                        &rproc_phandle)) {
> > +               fw_type = SCP;
> >         } else {
> >                 mtk_v4l2_err("Could not get venc IPI device");
> >                 return -ENODEV;
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > index 967bb100a990..f2a62ea62fc6 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > @@ -19,6 +19,7 @@ struct mtk_vcodec_fw {
> >         enum mtk_vcodec_fw_type type;
> >         const struct mtk_vcodec_fw_ops *ops;
> >         struct platform_device *pdev;
> > +       struct mtk_scp *scp;
> >  };
> >
> >  static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
> > @@ -71,6 +72,48 @@ static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
> >         .ipi_send = mtk_vcodec_vpu_ipi_send,
> >  };
> >
> > +static int mtk_vcodec_scp_load_firmware(struct mtk_vcodec_fw *fw)
> > +{
> > +       return rproc_boot(scp_get_rproc(fw->scp));
> > +}
> > +
> > +static unsigned int mtk_vcodec_scp_get_vdec_capa(struct mtk_vcodec_fw *fw)
> > +{
> > +       return scp_get_vdec_hw_capa(fw->scp);
> > +}
> > +
> > +static unsigned int mtk_vcodec_scp_get_venc_capa(struct mtk_vcodec_fw *fw)
> > +{
> > +       return scp_get_venc_hw_capa(fw->scp);
> > +}
> > +
> > +static void *mtk_vcodec_vpu_scp_dm_addr(struct mtk_vcodec_fw *fw,
> > +                                       u32 dtcm_dmem_addr)
> > +{
> > +       return scp_mapping_dm_addr(fw->scp, dtcm_dmem_addr);
> > +}
> > +
> > +static int mtk_vcodec_scp_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
> > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv)
> > +{
> > +       return scp_ipi_register(fw->scp, id, handler, priv);
> > +}
> > +
> > +static int mtk_vcodec_scp_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
> > +               unsigned int len, unsigned int wait)
> > +{
> > +       return scp_ipi_send(fw->scp, id, buf, len, wait);
> > +}
> > +
> > +static const struct mtk_vcodec_fw_ops mtk_vcodec_rproc_msg = {
> > +       .load_firmware = mtk_vcodec_scp_load_firmware,
> > +       .get_vdec_capa = mtk_vcodec_scp_get_vdec_capa,
> > +       .get_venc_capa = mtk_vcodec_scp_get_venc_capa,
> > +       .map_dm_addr = mtk_vcodec_vpu_scp_dm_addr,
> > +       .ipi_register = mtk_vcodec_scp_set_ipi_register,
> > +       .ipi_send = mtk_vcodec_scp_ipi_send,
> > +};
> > +
> >  static void mtk_vcodec_reset_handler(void *priv)
> >  {
> >         struct mtk_vcodec_dev *dev = priv;
> > @@ -94,6 +137,7 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> >         const struct mtk_vcodec_fw_ops *ops;
> >         struct mtk_vcodec_fw *fw;
> >         struct platform_device *fw_pdev = NULL;
> > +       struct mtk_scp *scp = NULL;
> >
> >         switch (type) {
> >         case VPU:
> > @@ -106,6 +150,14 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> >                 vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
> >                                     dev, rst_id);
> >                 break;
> > +       case SCP:
> > +               ops = &mtk_vcodec_rproc_msg;
> > +               scp = scp_get(dev->plat_dev);
> > +               if (!scp) {
> > +                       mtk_v4l2_err("could not get vdec scp handle");
> > +                       return ERR_PTR(-EPROBE_DEFER);
>
> I suspect the EPROBE_DEFER should be returned by scp_get
> itself instead.

scp_get() is a function of of mtk_scp remoteproc driver, so even if we
decide this is desirable (which I am not convinced, as the current
code leaves the freedom to decide how the absence of SCP should be
interpreted to the driver) this is beyond the scope of this series.

>
> > +               }
> > +               break;
> >         default:
> >                 mtk_v4l2_err("invalid vcodec fw type");
> >                 return ERR_PTR(-EINVAL);
> > @@ -118,6 +170,7 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> >         fw->type = type;
> >         fw->ops = ops;
> >         fw->pdev = fw_pdev;
> > +       fw->scp = scp;
> >
> >         return fw;
> >  }
> > @@ -129,6 +182,9 @@ void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
> >         case VPU:
> >                 put_device(&fw->pdev->dev);
> >                 break;
> > +       case SCP:
> > +               scp_put(fw->scp);
>
> Interestingly scp_put is a wrapper around put_device :-)
> Perhaps not a reason to violate the layering.

I don't see what is wrong with the current code? If SCP is in use, we
use SCP functions to manage it. If in the future SCP involves in such
a way that we need to do more than a put_device(), we are covered. Or
am I missing something?

Re: [PATCH v3 04/16] media: mtk-vcodec: venc: support SCP firmware

[Alexandre Courbot]

On Sat, Jul 25, 2020 at 6:13 AM Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> Hi Alexandre,
>
> I'm slowly making progress on the series. Here's some more comments.
>
> On Mon, 13 Jul 2020 at 03:10, Alexandre Courbot <acourbot@chromium.org> wrote:
> >
> > From: Yunfei Dong <yunfei.dong@mediatek.com>
> >
> > Support the new extended firmware used by MT8183's encoder.
> >
>
> If you could add some more information about the MT8183 encoder
> and the extended firmware interface, it would make the review
> easier.

Sure, I'll try to expand that description a bit.

>
> Some comments below, I have no idea how this hardware
> works, so maybe I won't make sense :-)
>
> > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > [acourbot: refactor, cleanup and split]
> > Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  .../platform/mtk-vcodec/mtk_vcodec_drv.h      | 12 +++
> >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 34 ++++---
> >  .../platform/mtk-vcodec/venc/venc_h264_if.c   | 65 +++++++++++--
> >  .../platform/mtk-vcodec/venc/venc_vp8_if.c    |  3 +-
> >  .../media/platform/mtk-vcodec/venc_drv_if.h   |  6 ++
> >  .../media/platform/mtk-vcodec/venc_ipi_msg.h  | 15 ++-
> >  .../media/platform/mtk-vcodec/venc_vpu_if.c   | 97 +++++++++++++------
> >  .../media/platform/mtk-vcodec/venc_vpu_if.h   |  3 +-
> >  8 files changed, 181 insertions(+), 54 deletions(-)
> >
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > index e132c4ec463a..45c8adfc6a0c 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > @@ -300,6 +300,17 @@ struct mtk_vcodec_ctx {
> >
> >  };
> >
> > +/**
> > + * struct mtk_vcodec_enc_pdata - compatible data for each IC
> > + *
> > + * @uses_ext: whether the encoder uses the extended firmware messaging format
> > + * @has_lt_irq: whether the encoder uses the LT irq
> > + */
> > +struct mtk_vcodec_enc_pdata {
> > +       bool uses_ext;
> > +       bool has_lt_irq;
>
> Instead of this boolean here, I think it might be cleaner
> to have a small array of strings, listing the interrupts
> each variant requires.
>
> See drivers/staging/media/hantro/rk3288_vpu_hw.c
>
> It's future-proof and even if you don't plan to see
> future works: a) we never know and b) cleaner code,
> easier to maintain.

I tend to agree. Note however that this boolean is supposed to
disappear soon as there is another patch (not part of this series)
that will split the decoder into two nodes. The reason why we have
this boolean is because MT8173 actually controls *2* instances of this
encoder (one for H.264, one for VP8) that are actually separate
hardware.

MT8183 only has one instance of the encoder (for H.264, that chip has
no VP8 support), so I have added this boolean to control this. Once
the DT node split happens it will go away. In the meantime, maybe we
can try to handle this better by looking at the number of interrupts
provided by the DT and enabling/disabling that extra interrupt based
on that. It will be 2 for MT8173 and 1 for MT8183, so that should work
fine.

>
> > +};
> > +
> >  /**
> >   * struct mtk_vcodec_dev - driver data
> >   * @v4l2_dev: V4L2 device to register video devices for.
> > @@ -348,6 +359,7 @@ struct mtk_vcodec_dev {
> >         spinlock_t irqlock;
> >         struct mtk_vcodec_ctx *curr_ctx;
> >         void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
> > +       const struct mtk_vcodec_enc_pdata *venc_pdata;
> >
> >         struct mtk_vcodec_fw *fw_handler;
> >
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > index 42530cd01a30..922bc8883811 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > @@ -244,6 +244,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> >         if (IS_ERR(dev->fw_handler))
> >                 return PTR_ERR(dev->fw_handler);
> >
> > +       dev->venc_pdata = of_device_get_match_data(&pdev->dev);
> >         ret = mtk_vcodec_init_enc_pm(dev);
> >         if (ret < 0) {
> >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> > @@ -278,21 +279,24 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> >                 ret = -EINVAL;
> >                 goto err_res;
> >         }
> > +       disable_irq(dev->enc_irq);
> >
>
> I am aware this is not an issue with this patch:
> requesting an interrupt and then immediately
> disabling the interrupt looks racy.
>
> Now, I haven't need this pattern myself,
> but from a quick grep I think you want to do:
>
>         irq_set_status_flags(irq, IRQ_NOAUTOEN);
>         ret = devm_request_irq(dev, irq, ...
>
> Perhaps something to fix (probably in another patch,
> before this one).

Nice suggestion, thanks. I will test and add a patch to this series.

>
> > -       dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > -       ret = devm_request_irq(&pdev->dev,
> > -                              dev->enc_lt_irq, mtk_vcodec_enc_lt_irq_handler,
> > -                              0, pdev->name, dev);
> > -       if (ret) {
> > -               dev_err(&pdev->dev,
> > -                       "Failed to install dev->enc_lt_irq %d (%d)",
> > -                       dev->enc_lt_irq, ret);
> > -               ret = -EINVAL;
> > -               goto err_res;
> > +       if (dev->venc_pdata->has_lt_irq) {
> > +               dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > +               ret = devm_request_irq(&pdev->dev,
> > +                                      dev->enc_lt_irq,
> > +                                      mtk_vcodec_enc_lt_irq_handler,
> > +                                      0, pdev->name, dev);
> > +               if (ret) {
> > +                       dev_err(&pdev->dev,
> > +                               "Failed to install dev->enc_lt_irq %d (%d)",
> > +                               dev->enc_lt_irq, ret);
> > +                       ret = -EINVAL;
> > +                       goto err_res;
> > +               }
> > +               disable_irq(dev->enc_lt_irq); /* VENC_LT */
> >         }
> >
> > -       disable_irq(dev->enc_irq);
> > -       disable_irq(dev->enc_lt_irq); /* VENC_LT */
> >         mutex_init(&dev->enc_mutex);
> >         mutex_init(&dev->dev_mutex);
> >         spin_lock_init(&dev->irqlock);
> > @@ -373,8 +377,12 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> >         return ret;
> >  }
> >
> > +static const struct mtk_vcodec_enc_pdata mt8173_pdata = {
> > +       .has_lt_irq = true,
> > +};
> > +
> >  static const struct of_device_id mtk_vcodec_enc_match[] = {
> > -       {.compatible = "mediatek,mt8173-vcodec-enc",},
> > +       {.compatible = "mediatek,mt8173-vcodec-enc", .data = &mt8173_pdata},
> >         {},
> >  };
> >  MODULE_DEVICE_TABLE(of, mtk_vcodec_enc_match);
> > diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> > index 7a00f050ec36..050787b2896c 100644
> > --- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> > +++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> > @@ -24,6 +24,16 @@ static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
> >  #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
> >  #define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
> >
> > +/*
> > + * enum venc_h264_frame_type - h264 encoder output bitstream frame type
> > + */
> > +enum venc_h264_frame_type {
> > +       VENC_H264_IDR_FRM,
> > +       VENC_H264_I_FRM,
> > +       VENC_H264_P_FRM,
> > +       VENC_H264_B_FRM,
> > +};
> > +
> >  /*
> >   * enum venc_h264_vpu_work_buf - h264 encoder buffer index
> >   */
> > @@ -137,7 +147,8 @@ struct venc_h264_inst {
> >         struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> >         struct mtk_vcodec_mem pps_buf;
> >         bool work_buf_allocated;
> > -       unsigned int frm_cnt;
> > +       u32 frm_cnt;    /* declared as u32 to properly overflow */
>
> This looks like it's fixing some issue not related to the 8183
> or the extended firmware. Perhaps you can add a
> more details comment?

Mmm, let me check that a bit closer because I know MT8183 communicates
with the firmware a bit differently, so this may require these fields.
But maybe we can split that part into another patch.

>
> > +       unsigned int skip_frm_cnt;
> >         unsigned int prepend_hdr;
> >         struct venc_vpu_inst vpu_inst;
> >         struct venc_h264_vsi *vsi;
> > @@ -327,6 +338,22 @@ static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
> >         return irq_status;
> >  }
> >
> > +static int h264_frame_type(struct venc_h264_inst *inst)
> > +{
> > +       if ((inst->vsi->config.gop_size != 0 &&
> > +               (inst->frm_cnt % inst->vsi->config.gop_size) == 0) ||
> > +               (inst->frm_cnt == 0 && inst->vsi->config.gop_size == 0)) {
> > +               /* IDR frame */
> > +               return VENC_H264_IDR_FRM;
> > +       } else if ((inst->vsi->config.intra_period != 0 &&
> > +               (inst->frm_cnt % inst->vsi->config.intra_period) == 0) ||
> > +               (inst->frm_cnt == 0 && inst->vsi->config.intra_period == 0)) {
> > +               /* I frame */
> > +               return VENC_H264_I_FRM;
> > +       } else {
> > +               return VENC_H264_P_FRM;  /* Note: B frames are not supported */
> > +       }
> > +}
> >  static int h264_encode_sps(struct venc_h264_inst *inst,
> >                            struct mtk_vcodec_mem *bs_buf,
> >                            unsigned int *bs_size)
> > @@ -337,7 +364,7 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
> >         mtk_vcodec_debug_enter(inst);
> >
> >         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
> > -                            bs_buf, bs_size);
> > +                            bs_buf, bs_size, NULL);
> >         if (ret)
> >                 return ret;
> >
> > @@ -364,7 +391,7 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
> >         mtk_vcodec_debug_enter(inst);
> >
> >         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
> > -                            bs_buf, bs_size);
> > +                            bs_buf, bs_size, NULL);
> >         if (ret)
> >                 return ret;
> >
> > @@ -410,13 +437,24 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
> >  {
> >         int ret = 0;
> >         unsigned int irq_status;
> > +       struct venc_frame_info frame_info;
> >
> >         mtk_vcodec_debug_enter(inst);
> > -
> > +       /* Overflowing back to 0 is ok and expected here */
> > +       inst->frm_cnt++;
> > +       mtk_vcodec_debug(inst, "frm_cnt++ = %d\n ", inst->frm_cnt);
> > +       frame_info.frm_cnt = inst->frm_cnt;
> > +       frame_info.skip_frm_cnt = inst->skip_frm_cnt;
> > +       frame_info.frm_type = h264_frame_type(inst);
>
> Ditto: Is this chunk related to the new support,
> or is this addressing some current issue?

I will need to re-read this part to understand which one it is, but
will split the patch if it turns out to make sense.

>
> > +       mtk_vcodec_debug(inst, "frm_cnt++ = %d,skip_frm_cnt =%d,frm_type=%d.\n",
> > +               frame_info.frm_cnt, frame_info.skip_frm_cnt,
> > +               frame_info.frm_type);
> >         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
> > -                            bs_buf, bs_size);
> > -       if (ret)
> > +                            bs_buf, bs_size, &frame_info);
> > +       if (ret) {
> > +               inst->frm_cnt--;
> >                 return ret;
> > +       }
> >
> >         /*
> >          * skip frame case: The skip frame buffer is composed by vpu side only,
> > @@ -427,19 +465,19 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
> >                 memcpy(bs_buf->va,
> >                        inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
> >                        *bs_size);
> > -               ++inst->frm_cnt;
> > +               ++inst->skip_frm_cnt;
> >                 return ret;
> >         }
> >
> >         irq_status = h264_enc_wait_venc_done(inst);
> >         if (irq_status != MTK_VENC_IRQ_STATUS_FRM) {
> >                 mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
> > +               inst->frm_cnt--;
> >                 return -EIO;
> >         }
> >
> >         *bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> >
> > -       ++inst->frm_cnt;
> >         mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-",
> >                          inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm);
> >
> > @@ -464,6 +502,7 @@ static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
> >
> >  static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
> >  {
> > +       const bool is_ext = ctx->dev->venc_pdata->uses_ext;
> >         int ret = 0;
> >         struct venc_h264_inst *inst;
> >
> > @@ -473,8 +512,9 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
> >
> >         inst->ctx = ctx;
> >         inst->vpu_inst.ctx = ctx;
> > -       inst->vpu_inst.id = IPI_VENC_H264;
> > +       inst->vpu_inst.id = is_ext ? SCP_IPI_VENC_H264 : IPI_VENC_H264;
> >         inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
> > +       inst->frm_cnt = 0xffffffff;
> >
> >         mtk_vcodec_debug_enter(inst);
> >
> > @@ -629,7 +669,12 @@ static int h264_enc_set_param(void *handle,
> >                 inst->prepend_hdr = 1;
> >                 mtk_vcodec_debug(inst, "set prepend header mode");
> >                 break;
> > -
> > +       case VENC_SET_PARAM_FORCE_INTRA:
> > +       case VENC_SET_PARAM_GOP_SIZE:
> > +       case VENC_SET_PARAM_INTRA_PERIOD:
> > +               inst->frm_cnt = 0xffffffff;
> > +               inst->skip_frm_cnt = 0;
> > +               fallthrough;
> >         default:
> >                 ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
> >                 break;
> > diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> > index 6426af514526..11abb191ada5 100644
> > --- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> > +++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> > @@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
> >
> >         mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
> >
> > -       ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size);
> > +       ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size,
> > +                            NULL);
> >         if (ret)
> >                 return ret;
> >
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > index 52fc9cc812fc..51b52625ca22 100644
> > --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > @@ -92,6 +92,12 @@ struct venc_enc_param {
> >         unsigned int gop_size;
> >  };
> >
>
> How about a comment here documenting this struct?

Will comment if I can figure out what it is useful for. :)

>
> > +struct venc_frame_info {
> > +       unsigned int frm_cnt;           /* per frame update */
> > +       unsigned int skip_frm_cnt;      /* per frame update */
>
> I'd go for s/cnt/count, it's just 2 bytes
> and in the long run it'll make our life easier.

Ack.

>
> > +       unsigned int frm_type;          /* per frame update */
> > +};
> > +
> >  /*
> >   * struct venc_frm_buf - frame buffer information used in venc_if_encode()
> >   * @fb_addr: plane frame buffer addresses
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > index 28ee04ca6241..4cafbf92d9cd 100644
> > --- a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > +++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > @@ -51,17 +51,22 @@ struct venc_ap_ipi_msg_init {
> >   * @vpu_inst_addr:     VPU encoder instance addr
> >   *                     (struct venc_vp8_vsi/venc_h264_vsi *)
> >   * @param_id:  parameter id (venc_set_param_type)
> > - * @data_item: number of items in the data array
> > + * @num_data:  number of items in the data array
> >   * @data[8]:   data array to store the set parameters
> >   */
> >  struct venc_ap_ipi_msg_set_param {
> >         uint32_t msg_id;
> >         uint32_t vpu_inst_addr;
> >         uint32_t param_id;
> > -       uint32_t data_item;
> > +       uint32_t num_data;
>
> This change looks either out of place, or unneeded.
> Or I'm missing something.

Yeah, it's just renaming this member to something that makes more
sense, "data_item" bringing absolutely no information about what this
does. Will make an extra patch for it.

>
> >         uint32_t data[8];
> >  };
> >
> > +struct venc_ap_ipi_msg_set_param_ext {
> > +       struct venc_ap_ipi_msg_set_param base;
> > +       uint32_t data_ext[24];
> > +};
> > +
> >  /**
> >   * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
> >   * @msg_id:    message id (AP_IPIMSG_XXX_ENC_ENCODE)
> > @@ -82,6 +87,12 @@ struct venc_ap_ipi_msg_enc {
> >         uint32_t bs_size;
> >  };
> >
>
> Ditto.

Ditto what? :) There is no code change right here.

>
> > +struct venc_ap_ipi_msg_enc_ext {
> > +       struct venc_ap_ipi_msg_enc base;
> > +       uint32_t data_item;
> > +       uint32_t data[32];
> > +};
> > +
> >  /**
> >   * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
> >   * @msg_id:    message id (AP_IPIMSG_XXX_ENC_DEINIT)
> > diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> > index 53854127814b..6c77bf025172 100644
> > --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> > +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> > @@ -116,49 +116,81 @@ int vpu_enc_init(struct venc_vpu_inst *vpu)
> >         return 0;
> >  }
> >
> > +static unsigned int venc_enc_param_crop_right(struct venc_vpu_inst *vpu,
> > +                                             struct venc_enc_param *enc_prm)
> > +{
> > +       unsigned int img_crop_right = enc_prm->buf_width - enc_prm->width;
> > +
> > +       return img_crop_right % 16;
> > +}
> > +
> > +static unsigned int venc_enc_param_crop_bottom(struct venc_enc_param *enc_prm)
> > +{
> > +       return round_up(enc_prm->height, 16) - enc_prm->height;
> > +}
> > +
> > +static unsigned int venc_enc_param_num_mb(struct venc_enc_param *enc_prm)
> > +{
> > +       return DIV_ROUND_UP(enc_prm->width, 16) *
> > +              DIV_ROUND_UP(enc_prm->height, 16);
>
> ^^^^
> You could define the macroblock size in a macro and
> use it in these various places.

Ack.

>
> > +}
> > +
> >  int vpu_enc_set_param(struct venc_vpu_inst *vpu,
> >                       enum venc_set_param_type id,
> >                       struct venc_enc_param *enc_param)
> >  {
> > -       struct venc_ap_ipi_msg_set_param out;
> > +       const bool is_ext = vpu->ctx->dev->venc_pdata->uses_ext;
> > +       size_t msg_size = is_ext ?
>
> How about you use vpu->id to check for is_ext?

Unfortunately vpu->id can be assigned an ipi_id or scp_ipi_id, which
are two different namespaces and which values can thus collide. So we
cannot use ID for this.

>
> I feel like querying the "is_ext" property like this
> is a layering violation.
>
> Or maybe better some firmware interface type
> should be set in struct venc_vpu_inst,
> so the venc_vpu_if driver is aware of its own type.

"is_ext" refers to whether the firmware uses an extended version of
the protocol to communicate, so using it here seems to be in place.

Re: [PATCH v3 07/16] media: mtk-vcodec: venc: specify supported formats per-chip

[Alexandre Courbot]

On Sun, Jul 26, 2020 at 11:29 PM Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> Hi Alexandre,
>
> Last review on my side, this series looks mostly good.

Thanks for taking the time to look at it! I know it's not the best
looking code out there. ^_^;

>
> On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
> >
> > Different chips have different supported bitrate ranges. Move the list
>
> s/bitrate ranges/formats

Ack.

>
> > of supported formats to the platform data, and split the output and
> > capture formats into two lists to make it easier to find the default
> > format for each queue.
> >
> > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   8 ++
> >  .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 122 +++++++-----------
> >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  40 ++++++
> >  3 files changed, 95 insertions(+), 75 deletions(-)
> >
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > index b8f913de8d80..59b4b750666b 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > @@ -313,6 +313,10 @@ enum mtk_chip {
> >   * @has_lt_irq: whether the encoder uses the LT irq
> >   * @min_birate: minimum supported encoding bitrate
> >   * @max_bitrate: maximum supported encoding bitrate
> > + * @capture_formats: array of supported capture formats
> > + * @num_capture_formats: number of entries in capture_formats
> > + * @output_formats: array of supported output formats
> > + * @num_output_formats: number of entries in output_formats
> >   */
> >  struct mtk_vcodec_enc_pdata {
> >         enum mtk_chip chip;
> > @@ -321,6 +325,10 @@ struct mtk_vcodec_enc_pdata {
> >         bool has_lt_irq;
> >         unsigned long min_bitrate;
> >         unsigned long max_bitrate;
> > +       const struct mtk_video_fmt *capture_formats;
> > +       size_t num_capture_formats;
> > +       const struct mtk_video_fmt *output_formats;
> > +       size_t num_output_formats;
> >  };
> >
> >  /**
> > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > index 50ba9da59153..05743a745a11 100644
> > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
> > @@ -23,47 +23,9 @@
> >  #define DFT_CFG_WIDTH  MTK_VENC_MIN_W
> >  #define DFT_CFG_HEIGHT MTK_VENC_MIN_H
> >  #define MTK_MAX_CTRLS_HINT     20
> > -#define OUT_FMT_IDX            0
> > -#define CAP_FMT_IDX            4
> > -
> >
> >  static void mtk_venc_worker(struct work_struct *work);
> >
> > -static const struct mtk_video_fmt mtk_video_formats[] = {
> > -       {
> > -               .fourcc = V4L2_PIX_FMT_NV12M,
> > -               .type = MTK_FMT_FRAME,
> > -               .num_planes = 2,
>
> Again, not an issue with your patch, so feel free to ignore this.
>
> You may avoid keeping track of num_planes (or any other
> property of the pixel format) and use v4l2_fill_pixfmt_mp
> to get all the information you need.

That's a good idea. Let's see if I can come with an extra patch in a
way that does not prevent the already acked patches from being merged.
:)

Re: [PATCH v3 04/16] media: mtk-vcodec: venc: support SCP firmware

[Ezequiel Garcia]

.

On Mon, 27 Jul 2020 at 06:06, Alexandre Courbot <acourbot@chromium.org> wrote:
>
> On Sat, Jul 25, 2020 at 6:13 AM Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > Hi Alexandre,
> >
> > I'm slowly making progress on the series. Here's some more comments.
> >
> > On Mon, 13 Jul 2020 at 03:10, Alexandre Courbot <acourbot@chromium.org> wrote:
> > >
> > > From: Yunfei Dong <yunfei.dong@mediatek.com>
> > >
> > > Support the new extended firmware used by MT8183's encoder.
> > >
> >
> > If you could add some more information about the MT8183 encoder
> > and the extended firmware interface, it would make the review
> > easier.
>
> Sure, I'll try to expand that description a bit.
>
> >
> > Some comments below, I have no idea how this hardware
> > works, so maybe I won't make sense :-)
> >
> > > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > > [acourbot: refactor, cleanup and split]
> > > Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > ---
> > >  .../platform/mtk-vcodec/mtk_vcodec_drv.h      | 12 +++
> > >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 34 ++++---
> > >  .../platform/mtk-vcodec/venc/venc_h264_if.c   | 65 +++++++++++--
> > >  .../platform/mtk-vcodec/venc/venc_vp8_if.c    |  3 +-
> > >  .../media/platform/mtk-vcodec/venc_drv_if.h   |  6 ++
> > >  .../media/platform/mtk-vcodec/venc_ipi_msg.h  | 15 ++-
> > >  .../media/platform/mtk-vcodec/venc_vpu_if.c   | 97 +++++++++++++------
> > >  .../media/platform/mtk-vcodec/venc_vpu_if.h   |  3 +-
> > >  8 files changed, 181 insertions(+), 54 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > index e132c4ec463a..45c8adfc6a0c 100644
> > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > @@ -300,6 +300,17 @@ struct mtk_vcodec_ctx {
> > >
> > >  };
> > >
> > > +/**
> > > + * struct mtk_vcodec_enc_pdata - compatible data for each IC
> > > + *
> > > + * @uses_ext: whether the encoder uses the extended firmware messaging format
> > > + * @has_lt_irq: whether the encoder uses the LT irq
> > > + */
> > > +struct mtk_vcodec_enc_pdata {
> > > +       bool uses_ext;
> > > +       bool has_lt_irq;
> >
> > Instead of this boolean here, I think it might be cleaner
> > to have a small array of strings, listing the interrupts
> > each variant requires.
> >
> > See drivers/staging/media/hantro/rk3288_vpu_hw.c
> >
> > It's future-proof and even if you don't plan to see
> > future works: a) we never know and b) cleaner code,
> > easier to maintain.
>
> I tend to agree. Note however that this boolean is supposed to
> disappear soon as there is another patch (not part of this series)
> that will split the decoder into two nodes. The reason why we have
> this boolean is because MT8173 actually controls *2* instances of this
> encoder (one for H.264, one for VP8) that are actually separate
> hardware.
>

Well, if you are planning to get rid of this boolean,
then ignore the comment.

> MT8183 only has one instance of the encoder (for H.264, that chip has
> no VP8 support), so I have added this boolean to control this. Once
> the DT node split happens it will go away.

Will this DT change break compatibility?

> In the meantime, maybe we
> can try to handle this better by looking at the number of interrupts
> provided by the DT and enabling/disabling that extra interrupt based
> on that. It will be 2 for MT8173 and 1 for MT8183, so that should work
> fine.
>
> >
> > > +};
> > > +
> > >  /**
> > >   * struct mtk_vcodec_dev - driver data
> > >   * @v4l2_dev: V4L2 device to register video devices for.
> > > @@ -348,6 +359,7 @@ struct mtk_vcodec_dev {
> > >         spinlock_t irqlock;
> > >         struct mtk_vcodec_ctx *curr_ctx;
> > >         void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
> > > +       const struct mtk_vcodec_enc_pdata *venc_pdata;
> > >
> > >         struct mtk_vcodec_fw *fw_handler;
> > >
> > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > index 42530cd01a30..922bc8883811 100644
> > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > @@ -244,6 +244,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > >         if (IS_ERR(dev->fw_handler))
> > >                 return PTR_ERR(dev->fw_handler);
> > >
> > > +       dev->venc_pdata = of_device_get_match_data(&pdev->dev);
> > >         ret = mtk_vcodec_init_enc_pm(dev);
> > >         if (ret < 0) {
> > >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> > > @@ -278,21 +279,24 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > >                 ret = -EINVAL;
> > >                 goto err_res;
> > >         }
> > > +       disable_irq(dev->enc_irq);
> > >
> >
> > I am aware this is not an issue with this patch:
> > requesting an interrupt and then immediately
> > disabling the interrupt looks racy.
> >
> > Now, I haven't need this pattern myself,
> > but from a quick grep I think you want to do:
> >
> >         irq_set_status_flags(irq, IRQ_NOAUTOEN);
> >         ret = devm_request_irq(dev, irq, ...
> >
> > Perhaps something to fix (probably in another patch,
> > before this one).
>
> Nice suggestion, thanks. I will test and add a patch to this series.
>
> >
> > > -       dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > > -       ret = devm_request_irq(&pdev->dev,
> > > -                              dev->enc_lt_irq, mtk_vcodec_enc_lt_irq_handler,
> > > -                              0, pdev->name, dev);
> > > -       if (ret) {
> > > -               dev_err(&pdev->dev,
> > > -                       "Failed to install dev->enc_lt_irq %d (%d)",
> > > -                       dev->enc_lt_irq, ret);
> > > -               ret = -EINVAL;
> > > -               goto err_res;
> > > +       if (dev->venc_pdata->has_lt_irq) {
> > > +               dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > > +               ret = devm_request_irq(&pdev->dev,
> > > +                                      dev->enc_lt_irq,
> > > +                                      mtk_vcodec_enc_lt_irq_handler,
> > > +                                      0, pdev->name, dev);
> > > +               if (ret) {
> > > +                       dev_err(&pdev->dev,
> > > +                               "Failed to install dev->enc_lt_irq %d (%d)",
> > > +                               dev->enc_lt_irq, ret);
> > > +                       ret = -EINVAL;
> > > +                       goto err_res;
> > > +               }
> > > +               disable_irq(dev->enc_lt_irq); /* VENC_LT */
> > >         }
> > >
> > > -       disable_irq(dev->enc_irq);
> > > -       disable_irq(dev->enc_lt_irq); /* VENC_LT */
> > >         mutex_init(&dev->enc_mutex);
> > >         mutex_init(&dev->dev_mutex);
> > >         spin_lock_init(&dev->irqlock);
> > > @@ -373,8 +377,12 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > >         return ret;
> > >  }
> > >
> > > +static const struct mtk_vcodec_enc_pdata mt8173_pdata = {
> > > +       .has_lt_irq = true,
> > > +};
> > > +
> > >  static const struct of_device_id mtk_vcodec_enc_match[] = {
> > > -       {.compatible = "mediatek,mt8173-vcodec-enc",},
> > > +       {.compatible = "mediatek,mt8173-vcodec-enc", .data = &mt8173_pdata},
> > >         {},
> > >  };
> > >  MODULE_DEVICE_TABLE(of, mtk_vcodec_enc_match);
> > > diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> > > index 7a00f050ec36..050787b2896c 100644
> > > --- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> > > +++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> > > @@ -24,6 +24,16 @@ static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
> > >  #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
> > >  #define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
> > >
> > > +/*
> > > + * enum venc_h264_frame_type - h264 encoder output bitstream frame type
> > > + */
> > > +enum venc_h264_frame_type {
> > > +       VENC_H264_IDR_FRM,
> > > +       VENC_H264_I_FRM,
> > > +       VENC_H264_P_FRM,
> > > +       VENC_H264_B_FRM,
> > > +};
> > > +
> > >  /*
> > >   * enum venc_h264_vpu_work_buf - h264 encoder buffer index
> > >   */
> > > @@ -137,7 +147,8 @@ struct venc_h264_inst {
> > >         struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> > >         struct mtk_vcodec_mem pps_buf;
> > >         bool work_buf_allocated;
> > > -       unsigned int frm_cnt;
> > > +       u32 frm_cnt;    /* declared as u32 to properly overflow */
> >
> > This looks like it's fixing some issue not related to the 8183
> > or the extended firmware. Perhaps you can add a
> > more details comment?
>
> Mmm, let me check that a bit closer because I know MT8183 communicates
> with the firmware a bit differently, so this may require these fields.
> But maybe we can split that part into another patch.
>
> >
> > > +       unsigned int skip_frm_cnt;
> > >         unsigned int prepend_hdr;
> > >         struct venc_vpu_inst vpu_inst;
> > >         struct venc_h264_vsi *vsi;
> > > @@ -327,6 +338,22 @@ static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
> > >         return irq_status;
> > >  }
> > >
> > > +static int h264_frame_type(struct venc_h264_inst *inst)
> > > +{
> > > +       if ((inst->vsi->config.gop_size != 0 &&
> > > +               (inst->frm_cnt % inst->vsi->config.gop_size) == 0) ||
> > > +               (inst->frm_cnt == 0 && inst->vsi->config.gop_size == 0)) {
> > > +               /* IDR frame */
> > > +               return VENC_H264_IDR_FRM;
> > > +       } else if ((inst->vsi->config.intra_period != 0 &&
> > > +               (inst->frm_cnt % inst->vsi->config.intra_period) == 0) ||
> > > +               (inst->frm_cnt == 0 && inst->vsi->config.intra_period == 0)) {
> > > +               /* I frame */
> > > +               return VENC_H264_I_FRM;
> > > +       } else {
> > > +               return VENC_H264_P_FRM;  /* Note: B frames are not supported */
> > > +       }
> > > +}
> > >  static int h264_encode_sps(struct venc_h264_inst *inst,
> > >                            struct mtk_vcodec_mem *bs_buf,
> > >                            unsigned int *bs_size)
> > > @@ -337,7 +364,7 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
> > >         mtk_vcodec_debug_enter(inst);
> > >
> > >         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
> > > -                            bs_buf, bs_size);
> > > +                            bs_buf, bs_size, NULL);
> > >         if (ret)
> > >                 return ret;
> > >
> > > @@ -364,7 +391,7 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
> > >         mtk_vcodec_debug_enter(inst);
> > >
> > >         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
> > > -                            bs_buf, bs_size);
> > > +                            bs_buf, bs_size, NULL);
> > >         if (ret)
> > >                 return ret;
> > >
> > > @@ -410,13 +437,24 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
> > >  {
> > >         int ret = 0;
> > >         unsigned int irq_status;
> > > +       struct venc_frame_info frame_info;
> > >
> > >         mtk_vcodec_debug_enter(inst);
> > > -
> > > +       /* Overflowing back to 0 is ok and expected here */
> > > +       inst->frm_cnt++;
> > > +       mtk_vcodec_debug(inst, "frm_cnt++ = %d\n ", inst->frm_cnt);
> > > +       frame_info.frm_cnt = inst->frm_cnt;
> > > +       frame_info.skip_frm_cnt = inst->skip_frm_cnt;
> > > +       frame_info.frm_type = h264_frame_type(inst);
> >
> > Ditto: Is this chunk related to the new support,
> > or is this addressing some current issue?
>
> I will need to re-read this part to understand which one it is, but
> will split the patch if it turns out to make sense.
>
> >
> > > +       mtk_vcodec_debug(inst, "frm_cnt++ = %d,skip_frm_cnt =%d,frm_type=%d.\n",
> > > +               frame_info.frm_cnt, frame_info.skip_frm_cnt,
> > > +               frame_info.frm_type);
> > >         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
> > > -                            bs_buf, bs_size);
> > > -       if (ret)
> > > +                            bs_buf, bs_size, &frame_info);
> > > +       if (ret) {
> > > +               inst->frm_cnt--;
> > >                 return ret;
> > > +       }
> > >
> > >         /*
> > >          * skip frame case: The skip frame buffer is composed by vpu side only,
> > > @@ -427,19 +465,19 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
> > >                 memcpy(bs_buf->va,
> > >                        inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
> > >                        *bs_size);
> > > -               ++inst->frm_cnt;
> > > +               ++inst->skip_frm_cnt;
> > >                 return ret;
> > >         }
> > >
> > >         irq_status = h264_enc_wait_venc_done(inst);
> > >         if (irq_status != MTK_VENC_IRQ_STATUS_FRM) {
> > >                 mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
> > > +               inst->frm_cnt--;
> > >                 return -EIO;
> > >         }
> > >
> > >         *bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> > >
> > > -       ++inst->frm_cnt;
> > >         mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-",
> > >                          inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm);
> > >
> > > @@ -464,6 +502,7 @@ static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
> > >
> > >  static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
> > >  {
> > > +       const bool is_ext = ctx->dev->venc_pdata->uses_ext;
> > >         int ret = 0;
> > >         struct venc_h264_inst *inst;
> > >
> > > @@ -473,8 +512,9 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
> > >
> > >         inst->ctx = ctx;
> > >         inst->vpu_inst.ctx = ctx;
> > > -       inst->vpu_inst.id = IPI_VENC_H264;
> > > +       inst->vpu_inst.id = is_ext ? SCP_IPI_VENC_H264 : IPI_VENC_H264;
> > >         inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
> > > +       inst->frm_cnt = 0xffffffff;
> > >
> > >         mtk_vcodec_debug_enter(inst);
> > >
> > > @@ -629,7 +669,12 @@ static int h264_enc_set_param(void *handle,
> > >                 inst->prepend_hdr = 1;
> > >                 mtk_vcodec_debug(inst, "set prepend header mode");
> > >                 break;
> > > -
> > > +       case VENC_SET_PARAM_FORCE_INTRA:
> > > +       case VENC_SET_PARAM_GOP_SIZE:
> > > +       case VENC_SET_PARAM_INTRA_PERIOD:
> > > +               inst->frm_cnt = 0xffffffff;
> > > +               inst->skip_frm_cnt = 0;
> > > +               fallthrough;
> > >         default:
> > >                 ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
> > >                 break;
> > > diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> > > index 6426af514526..11abb191ada5 100644
> > > --- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> > > +++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> > > @@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
> > >
> > >         mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
> > >
> > > -       ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size);
> > > +       ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size,
> > > +                            NULL);
> > >         if (ret)
> > >                 return ret;
> > >
> > > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > > index 52fc9cc812fc..51b52625ca22 100644
> > > --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > > @@ -92,6 +92,12 @@ struct venc_enc_param {
> > >         unsigned int gop_size;
> > >  };
> > >
> >
> > How about a comment here documenting this struct?
>
> Will comment if I can figure out what it is useful for. :)
>
> >
> > > +struct venc_frame_info {
> > > +       unsigned int frm_cnt;           /* per frame update */
> > > +       unsigned int skip_frm_cnt;      /* per frame update */
> >
> > I'd go for s/cnt/count, it's just 2 bytes
> > and in the long run it'll make our life easier.
>
> Ack.
>
> >
> > > +       unsigned int frm_type;          /* per frame update */
> > > +};
> > > +
> > >  /*
> > >   * struct venc_frm_buf - frame buffer information used in venc_if_encode()
> > >   * @fb_addr: plane frame buffer addresses
> > > diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > > index 28ee04ca6241..4cafbf92d9cd 100644
> > > --- a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > > +++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > > @@ -51,17 +51,22 @@ struct venc_ap_ipi_msg_init {
> > >   * @vpu_inst_addr:     VPU encoder instance addr
> > >   *                     (struct venc_vp8_vsi/venc_h264_vsi *)
> > >   * @param_id:  parameter id (venc_set_param_type)
> > > - * @data_item: number of items in the data array
> > > + * @num_data:  number of items in the data array
> > >   * @data[8]:   data array to store the set parameters
> > >   */
> > >  struct venc_ap_ipi_msg_set_param {
> > >         uint32_t msg_id;
> > >         uint32_t vpu_inst_addr;
> > >         uint32_t param_id;
> > > -       uint32_t data_item;
> > > +       uint32_t num_data;
> >
> > This change looks either out of place, or unneeded.
> > Or I'm missing something.
>
> Yeah, it's just renaming this member to something that makes more
> sense, "data_item" bringing absolutely no information about what this
> does. Will make an extra patch for it.
>
> >
> > >         uint32_t data[8];
> > >  };
> > >
> > > +struct venc_ap_ipi_msg_set_param_ext {
> > > +       struct venc_ap_ipi_msg_set_param base;
> > > +       uint32_t data_ext[24];
> > > +};
> > > +
> > >  /**
> > >   * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
> > >   * @msg_id:    message id (AP_IPIMSG_XXX_ENC_ENCODE)
> > > @@ -82,6 +87,12 @@ struct venc_ap_ipi_msg_enc {
> > >         uint32_t bs_size;
> > >  };
> > >
> >
> > Ditto.
>
> Ditto what? :) There is no code change right here.
>

Oh, sorry. I meant: same feedback about adding a comment
to the structure.

> >
> > > +struct venc_ap_ipi_msg_enc_ext {
> > > +       struct venc_ap_ipi_msg_enc base;
> > > +       uint32_t data_item;
> > > +       uint32_t data[32];
> > > +};
> > > +
> > >  /**
> > >   * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
> > >   * @msg_id:    message id (AP_IPIMSG_XXX_ENC_DEINIT)
> > > diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> > > index 53854127814b..6c77bf025172 100644
> > > --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> > > +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> > > @@ -116,49 +116,81 @@ int vpu_enc_init(struct venc_vpu_inst *vpu)
> > >         return 0;
> > >  }
> > >
> > > +static unsigned int venc_enc_param_crop_right(struct venc_vpu_inst *vpu,
> > > +                                             struct venc_enc_param *enc_prm)
> > > +{
> > > +       unsigned int img_crop_right = enc_prm->buf_width - enc_prm->width;
> > > +
> > > +       return img_crop_right % 16;
> > > +}
> > > +
> > > +static unsigned int venc_enc_param_crop_bottom(struct venc_enc_param *enc_prm)
> > > +{
> > > +       return round_up(enc_prm->height, 16) - enc_prm->height;
> > > +}
> > > +
> > > +static unsigned int venc_enc_param_num_mb(struct venc_enc_param *enc_prm)
> > > +{
> > > +       return DIV_ROUND_UP(enc_prm->width, 16) *
> > > +              DIV_ROUND_UP(enc_prm->height, 16);
> >
> > ^^^^
> > You could define the macroblock size in a macro and
> > use it in these various places.
>
> Ack.
>
> >
> > > +}
> > > +
> > >  int vpu_enc_set_param(struct venc_vpu_inst *vpu,
> > >                       enum venc_set_param_type id,
> > >                       struct venc_enc_param *enc_param)
> > >  {
> > > -       struct venc_ap_ipi_msg_set_param out;
> > > +       const bool is_ext = vpu->ctx->dev->venc_pdata->uses_ext;
> > > +       size_t msg_size = is_ext ?
> >
> > How about you use vpu->id to check for is_ext?
>
> Unfortunately vpu->id can be assigned an ipi_id or scp_ipi_id, which
> are two different namespaces and which values can thus collide. So we
> cannot use ID for this.
>
> >
> > I feel like querying the "is_ext" property like this
> > is a layering violation.
> >
> > Or maybe better some firmware interface type
> > should be set in struct venc_vpu_inst,
> > so the venc_vpu_if driver is aware of its own type.
>
> "is_ext" refers to whether the firmware uses an extended version of
> the protocol to communicate, so using it here seems to be in place.

Hm, maybe my comment wasn't really clear. I was trying
to point out that both venc_h264_if.c and venc_vpu_if.c
are using ctx->dev->venc_pdata->uses_ext to get
the "is_ext".

I.e. I would imagine it's cleaner to:

diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
index 050787b2896c..1ad25b59356a 100644
--- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
@@ -513,6 +513,7 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
        inst->ctx = ctx;
        inst->vpu_inst.ctx = ctx;
        inst->vpu_inst.id = is_ext ? SCP_IPI_VENC_H264 : IPI_VENC_H264;
+       inst->vpu_inst.is_ext = is_ext;
        inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
        inst->frm_cnt = 0xffffffff;

diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
index 472503701003..137453d66d8e 100644
--- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
+++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
@@ -160,7 +160,7 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
                      enum venc_set_param_type id,
                      struct venc_enc_param *enc_param)
 {
-       const bool is_ext = vpu->ctx->dev->venc_pdata->uses_ext;
+       const bool is_ext = vpu->is_ext;
        size_t msg_size = is_ext ?
                sizeof(struct venc_ap_ipi_msg_set_param_ext) :
                sizeof(struct venc_ap_ipi_msg_set_param);
@@ -228,7 +228,7 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu,
unsigned int bs_mode,
                   unsigned int *bs_size,
                   struct venc_frame_info *frame_info)
 {
-       const bool is_ext = vpu->ctx->dev->venc_pdata->uses_ext;
+       const bool is_ext = vpu->is_ext;
        size_t msg_size = is_ext ?
                sizeof(struct venc_ap_ipi_msg_enc_ext) :
                sizeof(struct venc_ap_ipi_msg_enc);

IMHO, this makes the structure cleaner, but
possibly this is just bikeshedding, so feel free to ignore.

Cheers,
Ezequiel

Re: [PATCH v3 03/16] media: mtk-vcodec: add SCP firmware ops

[Ezequiel Garcia]

On Mon, 27 Jul 2020 at 06:06, Alexandre Courbot <acourbot@chromium.org> wrote:
>
> On Thu, Jul 23, 2020 at 6:40 AM Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
> > >
> > > From: Yunfei Dong <yunfei.dong@mediatek.com>
> > >
> > > Add support for communicating with the SCP firmware, which will be used
> > > by MT8183.
> > >
> > > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > > [acourbot: refactor, cleanup and split]
> > > Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > ---
> > >  drivers/media/platform/Kconfig                |  1 +
> > >  .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  3 +
> > >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  3 +
> > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 56 +++++++++++++++++++
> > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  2 +
> > >  5 files changed, 65 insertions(+)
> > >
> > > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > > index c57ee78fa99d..f0dbe048efea 100644
> > > --- a/drivers/media/platform/Kconfig
> > > +++ b/drivers/media/platform/Kconfig
> > > @@ -256,6 +256,7 @@ config VIDEO_MEDIATEK_VCODEC
> > >         select VIDEOBUF2_DMA_CONTIG
> > >         select V4L2_MEM2MEM_DEV
> > >         select VIDEO_MEDIATEK_VPU
> > > +       select MTK_SCP
> > >         help
> > >             Mediatek video codec driver provides HW capability to
> > >             encode and decode in a range of video formats
> > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > index 4f07a5fcce7f..5b5765b98e57 100644
> > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > @@ -225,6 +225,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > >         if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > >                                   &rproc_phandle)) {
> > >                 fw_type = VPU;
> > > +       } else if (!of_property_read_u32(pdev->dev.of_node, "mediatek,scp",
> > > +                                        &rproc_phandle)) {
> > > +               fw_type = SCP;
> > >         } else {
> > >                 mtk_v4l2_err("Could not get vdec IPI device");
> > >                 return -ENODEV;
> > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > index 4340ea10afd0..42530cd01a30 100644
> > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > @@ -233,6 +233,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > >         if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > >                                   &rproc_phandle)) {
> > >                 fw_type = VPU;
> > > +       } else if (!of_property_read_u32(pdev->dev.of_node, "mediatek,scp",
> > > +                                        &rproc_phandle)) {
> > > +               fw_type = SCP;
> > >         } else {
> > >                 mtk_v4l2_err("Could not get venc IPI device");
> > >                 return -ENODEV;
> > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > index 967bb100a990..f2a62ea62fc6 100644
> > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > @@ -19,6 +19,7 @@ struct mtk_vcodec_fw {
> > >         enum mtk_vcodec_fw_type type;
> > >         const struct mtk_vcodec_fw_ops *ops;
> > >         struct platform_device *pdev;
> > > +       struct mtk_scp *scp;
> > >  };
> > >
> > >  static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
> > > @@ -71,6 +72,48 @@ static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
> > >         .ipi_send = mtk_vcodec_vpu_ipi_send,
> > >  };
> > >
> > > +static int mtk_vcodec_scp_load_firmware(struct mtk_vcodec_fw *fw)
> > > +{
> > > +       return rproc_boot(scp_get_rproc(fw->scp));
> > > +}
> > > +
> > > +static unsigned int mtk_vcodec_scp_get_vdec_capa(struct mtk_vcodec_fw *fw)
> > > +{
> > > +       return scp_get_vdec_hw_capa(fw->scp);
> > > +}
> > > +
> > > +static unsigned int mtk_vcodec_scp_get_venc_capa(struct mtk_vcodec_fw *fw)
> > > +{
> > > +       return scp_get_venc_hw_capa(fw->scp);
> > > +}
> > > +
> > > +static void *mtk_vcodec_vpu_scp_dm_addr(struct mtk_vcodec_fw *fw,
> > > +                                       u32 dtcm_dmem_addr)
> > > +{
> > > +       return scp_mapping_dm_addr(fw->scp, dtcm_dmem_addr);
> > > +}
> > > +
> > > +static int mtk_vcodec_scp_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
> > > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv)
> > > +{
> > > +       return scp_ipi_register(fw->scp, id, handler, priv);
> > > +}
> > > +
> > > +static int mtk_vcodec_scp_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
> > > +               unsigned int len, unsigned int wait)
> > > +{
> > > +       return scp_ipi_send(fw->scp, id, buf, len, wait);
> > > +}
> > > +
> > > +static const struct mtk_vcodec_fw_ops mtk_vcodec_rproc_msg = {
> > > +       .load_firmware = mtk_vcodec_scp_load_firmware,
> > > +       .get_vdec_capa = mtk_vcodec_scp_get_vdec_capa,
> > > +       .get_venc_capa = mtk_vcodec_scp_get_venc_capa,
> > > +       .map_dm_addr = mtk_vcodec_vpu_scp_dm_addr,
> > > +       .ipi_register = mtk_vcodec_scp_set_ipi_register,
> > > +       .ipi_send = mtk_vcodec_scp_ipi_send,
> > > +};
> > > +
> > >  static void mtk_vcodec_reset_handler(void *priv)
> > >  {
> > >         struct mtk_vcodec_dev *dev = priv;
> > > @@ -94,6 +137,7 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> > >         const struct mtk_vcodec_fw_ops *ops;
> > >         struct mtk_vcodec_fw *fw;
> > >         struct platform_device *fw_pdev = NULL;
> > > +       struct mtk_scp *scp = NULL;
> > >
> > >         switch (type) {
> > >         case VPU:
> > > @@ -106,6 +150,14 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> > >                 vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
> > >                                     dev, rst_id);
> > >                 break;
> > > +       case SCP:
> > > +               ops = &mtk_vcodec_rproc_msg;
> > > +               scp = scp_get(dev->plat_dev);
> > > +               if (!scp) {
> > > +                       mtk_v4l2_err("could not get vdec scp handle");
> > > +                       return ERR_PTR(-EPROBE_DEFER);
> >
> > I suspect the EPROBE_DEFER should be returned by scp_get
> > itself instead.
>
> scp_get() is a function of of mtk_scp remoteproc driver, so even if we
> decide this is desirable (which I am not convinced, as the current
> code leaves the freedom to decide how the absence of SCP should be
> interpreted to the driver) this is beyond the scope of this series.
>

How can the consumer decide if it should defer probing or not?

> >
> > > +               }
> > > +               break;
> > >         default:
> > >                 mtk_v4l2_err("invalid vcodec fw type");
> > >                 return ERR_PTR(-EINVAL);
> > > @@ -118,6 +170,7 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> > >         fw->type = type;
> > >         fw->ops = ops;
> > >         fw->pdev = fw_pdev;
> > > +       fw->scp = scp;
> > >
> > >         return fw;
> > >  }
> > > @@ -129,6 +182,9 @@ void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
> > >         case VPU:
> > >                 put_device(&fw->pdev->dev);
> > >                 break;
> > > +       case SCP:
> > > +               scp_put(fw->scp);
> >
> > Interestingly scp_put is a wrapper around put_device :-)
> > Perhaps not a reason to violate the layering.
>
> I don't see what is wrong with the current code? If SCP is in use, we
> use SCP functions to manage it. If in the future SCP involves in such
> a way that we need to do more than a put_device(), we are covered. Or
> am I missing something?

Oh, nothing wrong. I just found it interesting that scp_put was
just wrapping put_device. Nothing to fix really.

Thanks!
Ezequiel

Re: [PATCH v3 02/16] dt-bindings: media: mtk-vcodec: document SCP node

[Ezequiel Garcia]

On Mon, 27 Jul 2020 at 06:06, Alexandre Courbot <acourbot@chromium.org> wrote:
>
> On Thu, Jul 23, 2020 at 6:37 AM Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
> > >
> > > The mediatek codecs can use either the VPU or the SCP as their interface
> > > to firmware. Reflect this in the DT bindings.
> > >
> > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > ---
> > >  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 4 +++-
> > >  1 file changed, 3 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > > index b6b5dde6abd8..7aef0a4fe207 100644
> > > --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > > +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > > @@ -19,7 +19,9 @@ Required properties:
> > >  - iommus : should point to the respective IOMMU block with master port as
> > >    argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> > >    for details.
> > > -- mediatek,vpu : the node of video processor unit
> > > +One of the two following nodes:
> > > +- mediatek,vpu : the node of the video processor unit, if using VPU.
> > > +- mediatek,scp : the noode of the SCP unit, if using SCP.
> > >
> >
> > This interface doesn't enforce the fact only one of the two
> > should be present, but not both (which is the case, right?).
>
> That's correct.
>
> >
> > I hope I'm not bikeshedding here, but from an interface POV,
> > would it be cleaner to just have a single mediatek,coprocessor
> > property, and then use of_device_is_compatible
> > to distinguish VPU from SCP type?
>
> From an interface point of view maybe, however doing so would
> introduce a backward-incompatible change with the existing MT8173
> bindings. I also feel like it is less error-prone to have the property
> explicitly state what it is expecting at the other end of the phandle
> (vpu or scp) instead of the more generic "coprocessor".
>
> >
> > Moreover, I'd argue you don't need a dt-binding change
> > and should just keep the current mediatek-vpu property,
> > and then rely on of_device_is_compatible.
>
> VPU and SCP are different kinds of processors, so I'm not sure whether
> it is desirable to use VPU interchangeably like this. Note that I'm
> not strongly against it either, but for things like bindings I tend to
> prefer precise language to avoid confusions.

Yeah, I guess that makes sense.

Not only from a language precision point of view (after all
DT are not designed to be human readable).

But as you mention, given the processors will have different compatible
strings it would make sense to not allow overloading the property.

In any case, I don't have a strong opinion either.

Thanks,
Ezequiel

Re: [PATCH v3 01/16] media: mtk-vcodec: abstract firmware interface

[Ezequiel Garcia]

On Mon, 27 Jul 2020 at 06:06, Alexandre Courbot <acourbot@chromium.org> wrote:
>
> On Thu, Jul 23, 2020 at 6:23 AM Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
> > >
> > > From: Yunfei Dong <yunfei.dong@mediatek.com>
> > >
> > > MT8183's codec firwmare is run by a different remote processor from
> > > MT8173. While the firmware interface is basically the same, the way to
> > > invoke it differs. Abstract all firmware calls under a layer that will
> > > allow us to handle both firmware types transparently.
> > >
> > > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > > [acourbot: refactor, cleanup and split]
> > > Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > > [pihsun: fix error path and add mtk_vcodec_fw_release]
> > > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > > Reviewed-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > ---
> > >  drivers/media/platform/mtk-vcodec/Makefile    |   4 +-
> > >  .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  50 ++---
> > >  .../platform/mtk-vcodec/mtk_vcodec_dec_pm.c   |   1 -
> > >  .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   5 +-
> > >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  47 ++---
> > >  .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   |   2 -
> > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 172 ++++++++++++++++++
> > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  36 ++++
> > >  .../platform/mtk-vcodec/mtk_vcodec_util.c     |   1 -
> > >  .../platform/mtk-vcodec/vdec/vdec_h264_if.c   |   1 -
> > >  .../platform/mtk-vcodec/vdec/vdec_vp8_if.c    |   1 -
> > >  .../platform/mtk-vcodec/vdec/vdec_vp9_if.c    |   1 -
> > >  .../media/platform/mtk-vcodec/vdec_drv_base.h |   2 -
> > >  .../media/platform/mtk-vcodec/vdec_drv_if.c   |   1 -
> > >  .../media/platform/mtk-vcodec/vdec_vpu_if.c   |  12 +-
> > >  .../media/platform/mtk-vcodec/vdec_vpu_if.h   |  11 +-
> > >  .../platform/mtk-vcodec/venc/venc_h264_if.c   |  15 +-
> > >  .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   8 +-
> > >  .../media/platform/mtk-vcodec/venc_drv_if.c   |   1 -
> > >  .../media/platform/mtk-vcodec/venc_vpu_if.c   |  17 +-
> > >  .../media/platform/mtk-vcodec/venc_vpu_if.h   |   5 +-
> > >  21 files changed, 290 insertions(+), 103 deletions(-)
> > >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
> > >
> > > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > > index 37b94b555fa1..b8636119ed0a 100644
> > > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > > @@ -12,7 +12,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
> > >                 vdec_vpu_if.o \
> > >                 mtk_vcodec_dec.o \
> > >                 mtk_vcodec_dec_pm.o \
> > > -
> > > +               mtk_vcodec_fw.o
> > >
> > >  mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> > >                 venc/venc_h264_if.o \
> > > @@ -25,5 +25,3 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> > >
> > >  mtk-vcodec-common-y := mtk_vcodec_intr.o \
> > >                 mtk_vcodec_util.o\
> > > -
> > > -ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > index 97a1b6664c20..4f07a5fcce7f 100644
> > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > @@ -20,7 +20,7 @@
> > >  #include "mtk_vcodec_dec_pm.h"
> > >  #include "mtk_vcodec_intr.h"
> > >  #include "mtk_vcodec_util.h"
> > > -#include "mtk_vpu.h"
> > > +#include "mtk_vcodec_fw.h"
> > >
> > >  #define VDEC_HW_ACTIVE 0x10
> > >  #define VDEC_IRQ_CFG   0x11
> > > @@ -77,22 +77,6 @@ static irqreturn_t mtk_vcodec_dec_irq_handler(int irq, void *priv)
> > >         return IRQ_HANDLED;
> > >  }
> > >
> > > -static void mtk_vcodec_dec_reset_handler(void *priv)
> > > -{
> > > -       struct mtk_vcodec_dev *dev = priv;
> > > -       struct mtk_vcodec_ctx *ctx;
> > > -
> > > -       mtk_v4l2_err("Watchdog timeout!!");
> > > -
> > > -       mutex_lock(&dev->dev_mutex);
> > > -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > -               ctx->state = MTK_STATE_ABORT;
> > > -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR",
> > > -                               ctx->id);
> > > -       }
> > > -       mutex_unlock(&dev->dev_mutex);
> > > -}
> > > -
> > >  static int fops_vcodec_open(struct file *file)
> > >  {
> > >         struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > @@ -144,21 +128,20 @@ static int fops_vcodec_open(struct file *file)
> > >         if (v4l2_fh_is_singular(&ctx->fh)) {
> > >                 mtk_vcodec_dec_pw_on(&dev->pm);
> > >                 /*
> > > -                * vpu_load_firmware checks if it was loaded already and
> > > -                * does nothing in that case
> > > +                * Does nothing if firmware was already loaded.
> > >                  */
> > > -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > > +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
> > >                 if (ret < 0) {
> > >                         /*
> > >                          * Return 0 if downloading firmware successfully,
> > >                          * otherwise it is failed
> > >                          */
> > > -                       mtk_v4l2_err("vpu_load_firmware failed!");
> > > +                       mtk_v4l2_err("failed to load firmware!");
> > >                         goto err_load_fw;
> > >                 }
> > >
> > >                 dev->dec_capability =
> > > -                       vpu_get_vdec_hw_capa(dev->vpu_plat_dev);
> > > +                       mtk_vcodec_fw_get_vdec_capa(dev->fw_handler);
> > >                 mtk_v4l2_debug(0, "decoder capability %x", dev->dec_capability);
> > >         }
> > >
> > > @@ -228,6 +211,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > >         struct mtk_vcodec_dev *dev;
> > >         struct video_device *vfd_dec;
> > >         struct resource *res;
> > > +       phandle rproc_phandle;
> > > +       enum mtk_vcodec_fw_type fw_type;
> > >         int i, ret;
> > >
> > >         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > > @@ -237,19 +222,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > >         INIT_LIST_HEAD(&dev->ctx_list);
> > >         dev->plat_dev = pdev;
> > >
> > > -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > > -       if (dev->vpu_plat_dev == NULL) {
> > > -               mtk_v4l2_err("[VPU] vpu device in not ready");
> > > -               return -EPROBE_DEFER;
> > > +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > > +                                 &rproc_phandle)) {
> > > +               fw_type = VPU;
> > > +       } else {
> > > +               mtk_v4l2_err("Could not get vdec IPI device");
> > > +               return -ENODEV;
> > >         }
> > > -
> > > -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_dec_reset_handler,
> > > -                       dev, VPU_RST_DEC);
> > > +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_DEC);
> > > +       if (IS_ERR(dev->fw_handler))
> > > +               return PTR_ERR(dev->fw_handler);
> > >
> > >         ret = mtk_vcodec_init_dec_pm(dev);
> > >         if (ret < 0) {
> > >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source");
> > > -               return ret;
> > > +               goto err_dec_pm;
> > >         }
> > >
> > >         for (i = 0; i < NUM_MAX_VDEC_REG_BASE; i++) {
> > > @@ -352,6 +339,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > >         v4l2_device_unregister(&dev->v4l2_dev);
> > >  err_res:
> > >         mtk_vcodec_release_dec_pm(dev);
> > > +err_dec_pm:
> > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > >         return ret;
> > >  }
> > >
> > > @@ -376,6 +365,7 @@ static int mtk_vcodec_dec_remove(struct platform_device *pdev)
> > >
> > >         v4l2_device_unregister(&dev->v4l2_dev);
> > >         mtk_vcodec_release_dec_pm(dev);
> > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > >         return 0;
> > >  }
> > >
> > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > index 5a6ec8fb52da..36dfe3fc056a 100644
> > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > @@ -12,7 +12,6 @@
> > >
> > >  #include "mtk_vcodec_dec_pm.h"
> > >  #include "mtk_vcodec_util.h"
> > > -#include "mtk_vpu.h"
> > >
> > >  int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *mtkdev)
> > >  {
> > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > index 9fd56dee7fd1..e132c4ec463a 100644
> > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > @@ -309,13 +309,13 @@ struct mtk_vcodec_ctx {
> > >   * @m2m_dev_dec: m2m device for decoder
> > >   * @m2m_dev_enc: m2m device for encoder.
> > >   * @plat_dev: platform device
> > > - * @vpu_plat_dev: mtk vpu platform device
> > >   * @ctx_list: list of struct mtk_vcodec_ctx
> > >   * @irqlock: protect data access by irq handler and work thread
> > >   * @curr_ctx: The context that is waiting for codec hardware
> > >   *
> > >   * @reg_base: Mapped address of MTK Vcodec registers.
> > >   *
> > > + * @fw_handler: used to communicate with the firmware.
> > >   * @id_counter: used to identify current opened instance
> > >   *
> > >   * @encode_workqueue: encode work queue
> > > @@ -344,12 +344,13 @@ struct mtk_vcodec_dev {
> > >         struct v4l2_m2m_dev *m2m_dev_dec;
> > >         struct v4l2_m2m_dev *m2m_dev_enc;
> > >         struct platform_device *plat_dev;
> > > -       struct platform_device *vpu_plat_dev;
> > >         struct list_head ctx_list;
> > >         spinlock_t irqlock;
> > >         struct mtk_vcodec_ctx *curr_ctx;
> > >         void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
> > >
> > > +       struct mtk_vcodec_fw *fw_handler;
> > > +
> > >         unsigned long id_counter;
> > >
> > >         struct workqueue_struct *decode_workqueue;
> > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > index 4d31f1ed113f..4340ea10afd0 100644
> > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > @@ -21,7 +21,7 @@
> > >  #include "mtk_vcodec_enc_pm.h"
> > >  #include "mtk_vcodec_intr.h"
> > >  #include "mtk_vcodec_util.h"
> > > -#include "mtk_vpu.h"
> > > +#include "mtk_vcodec_fw.h"
> > >
> > >  module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> > >  module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> > > @@ -101,22 +101,6 @@ static irqreturn_t mtk_vcodec_enc_lt_irq_handler(int irq, void *priv)
> > >         return IRQ_HANDLED;
> > >  }
> > >
> > > -static void mtk_vcodec_enc_reset_handler(void *priv)
> > > -{
> > > -       struct mtk_vcodec_dev *dev = priv;
> > > -       struct mtk_vcodec_ctx *ctx;
> > > -
> > > -       mtk_v4l2_debug(0, "Watchdog timeout!!");
> > > -
> > > -       mutex_lock(&dev->dev_mutex);
> > > -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > -               ctx->state = MTK_STATE_ABORT;
> > > -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> > > -                               ctx->id);
> > > -       }
> > > -       mutex_unlock(&dev->dev_mutex);
> > > -}
> > > -
> > >  static int fops_vcodec_open(struct file *file)
> > >  {
> > >         struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > @@ -159,10 +143,10 @@ static int fops_vcodec_open(struct file *file)
> > >
> > >         if (v4l2_fh_is_singular(&ctx->fh)) {
> > >                 /*
> > > -                * vpu_load_firmware checks if it was loaded already and
> > > +                * load fireware to checks if it was loaded already and
> > >                  * does nothing in that case
> > >                  */
> > > -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > > +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
> > >                 if (ret < 0) {
> > >                         /*
> > >                          * Return 0 if downloading firmware successfully,
> > > @@ -173,7 +157,7 @@ static int fops_vcodec_open(struct file *file)
> > >                 }
> > >
> > >                 dev->enc_capability =
> > > -                       vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > > +                       mtk_vcodec_fw_get_venc_capa(dev->fw_handler);
> > >                 mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> > >         }
> > >
> > > @@ -235,6 +219,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > >         struct mtk_vcodec_dev *dev;
> > >         struct video_device *vfd_enc;
> > >         struct resource *res;
> > > +       phandle rproc_phandle;
> > > +       enum mtk_vcodec_fw_type fw_type;
> > >         int i, j, ret;
> > >
> > >         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > > @@ -244,19 +230,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > >         INIT_LIST_HEAD(&dev->ctx_list);
> > >         dev->plat_dev = pdev;
> > >
> > > -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > > -       if (dev->vpu_plat_dev == NULL) {
> > > -               mtk_v4l2_err("[VPU] vpu device in not ready");
> > > -               return -EPROBE_DEFER;
> > > +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > > +                                 &rproc_phandle)) {
> > > +               fw_type = VPU;
> > > +       } else {
> > > +               mtk_v4l2_err("Could not get venc IPI device");
> > > +               return -ENODEV;
> > >         }
> > > -
> > > -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler,
> > > -                               dev, VPU_RST_ENC);
> > > +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_ENC);
> > > +       if (IS_ERR(dev->fw_handler))
> > > +               return PTR_ERR(dev->fw_handler);
> > >
> > >         ret = mtk_vcodec_init_enc_pm(dev);
> > >         if (ret < 0) {
> > >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> > > -               return ret;
> > > +               goto err_enc_pm;
> > >         }
> > >
> > >         for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > > @@ -377,6 +365,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > >         v4l2_device_unregister(&dev->v4l2_dev);
> > >  err_res:
> > >         mtk_vcodec_release_enc_pm(dev);
> > > +err_enc_pm:
> > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > >         return ret;
> > >  }
> > >
> > > @@ -401,6 +391,7 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
> > >
> > >         v4l2_device_unregister(&dev->v4l2_dev);
> > >         mtk_vcodec_release_enc_pm(dev);
> > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > >         return 0;
> > >  }
> > >
> > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > index 3e2bfded79a6..ee22902aaa71 100644
> > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > @@ -12,8 +12,6 @@
> > >
> > >  #include "mtk_vcodec_enc_pm.h"
> > >  #include "mtk_vcodec_util.h"
> > > -#include "mtk_vpu.h"
> > > -
> > >
> > >  int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > >  {
> > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > new file mode 100644
> > > index 000000000000..967bb100a990
> > > --- /dev/null
> > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > @@ -0,0 +1,172 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +
> > > +#include "mtk_vcodec_fw.h"
> > > +#include "mtk_vcodec_util.h"
> > > +#include "mtk_vcodec_drv.h"
> > > +
> > > +struct mtk_vcodec_fw_ops {
> > > +       int (*load_firmware)(struct mtk_vcodec_fw *fw);
> > > +       unsigned int (*get_vdec_capa)(struct mtk_vcodec_fw *fw);
> > > +       unsigned int (*get_venc_capa)(struct mtk_vcodec_fw *fw);
> > > +       void * (*map_dm_addr)(struct mtk_vcodec_fw *fw, u32 dtcm_dmem_addr);
> > > +       int (*ipi_register)(struct mtk_vcodec_fw *fw, int id,
> > > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv);
> > > +       int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
> > > +               unsigned int len, unsigned int wait);
> > > +};
> > > +
> > > +struct mtk_vcodec_fw {
> > > +       enum mtk_vcodec_fw_type type;
> > > +       const struct mtk_vcodec_fw_ops *ops;
> > > +       struct platform_device *pdev;
> > > +};
> > > +
> > > +static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
> > > +{
> > > +       return vpu_load_firmware(fw->pdev);
> > > +}
> > > +
> > > +static unsigned int mtk_vcodec_vpu_get_vdec_capa(struct mtk_vcodec_fw *fw)
> > > +{
> > > +       return vpu_get_vdec_hw_capa(fw->pdev);
> > > +}
> > > +
> > > +static unsigned int mtk_vcodec_vpu_get_venc_capa(struct mtk_vcodec_fw *fw)
> > > +{
> > > +       return vpu_get_venc_hw_capa(fw->pdev);
> > > +}
> > > +
> > > +static void *mtk_vcodec_vpu_map_dm_addr(struct mtk_vcodec_fw *fw,
> > > +                                       u32 dtcm_dmem_addr)
> > > +{
> > > +       return vpu_mapping_dm_addr(fw->pdev, dtcm_dmem_addr);
> > > +}
> > > +
> > > +static int mtk_vcodec_vpu_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
> > > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv)
> > > +{
> > > +       /*
> > > +        * The handler we receive takes a void * as its first argument. We
> > > +        * cannot change this because it needs to be passed down to the rproc
> > > +        * subsystem when SCP is used. VPU takes a const argument, which is
> > > +        * more constrained, so the conversion below is safe.
> > > +        */
> > > +       ipi_handler_t handler_const = (ipi_handler_t)handler;
> > > +
> > > +       return vpu_ipi_register(fw->pdev, id, handler_const, name, priv);
> > > +}
> > > +
> > > +static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
> > > +               unsigned int len, unsigned int wait)
> > > +{
> > > +       return vpu_ipi_send(fw->pdev, id, buf, len);
> > > +}
> > > +
> > > +static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
> > > +       .load_firmware = mtk_vcodec_vpu_load_firmware,
> > > +       .get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
> > > +       .get_venc_capa = mtk_vcodec_vpu_get_venc_capa,
> > > +       .map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
> > > +       .ipi_register = mtk_vcodec_vpu_set_ipi_register,
> > > +       .ipi_send = mtk_vcodec_vpu_ipi_send,
> > > +};
> > > +
> > > +static void mtk_vcodec_reset_handler(void *priv)
> > > +{
> > > +       struct mtk_vcodec_dev *dev = priv;
> > > +       struct mtk_vcodec_ctx *ctx;
> > > +
> > > +       mtk_v4l2_err("Watchdog timeout!!");
> > > +
> > > +       mutex_lock(&dev->dev_mutex);
> > > +       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > +               ctx->state = MTK_STATE_ABORT;
> > > +               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> > > +                               ctx->id);
> > > +       }
> > > +       mutex_unlock(&dev->dev_mutex);
> > > +}
> > > +
> > > +struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> > > +                                          enum mtk_vcodec_fw_type type,
> > > +                                          enum rst_id rst_id)
> > > +{
> > > +       const struct mtk_vcodec_fw_ops *ops;
> > > +       struct mtk_vcodec_fw *fw;
> > > +       struct platform_device *fw_pdev = NULL;
> > > +
> > > +       switch (type) {
> > > +       case VPU:
> > > +               ops = &mtk_vcodec_vpu_msg;
> > > +               fw_pdev = vpu_get_plat_device(dev->plat_dev);
> > > +               if (!fw_pdev) {
> > > +                       mtk_v4l2_err("firmware device is not ready");
> > > +                       return ERR_PTR(-EINVAL);
> > > +               }
> > > +               vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
> > > +                                   dev, rst_id);
> > > +               break;
> > > +       default:
> > > +               mtk_v4l2_err("invalid vcodec fw type");
> > > +               return ERR_PTR(-EINVAL);
> > > +       }
> > > +
> > > +       fw = devm_kzalloc(&dev->plat_dev->dev, sizeof(*fw), GFP_KERNEL);
> > > +       if (!fw)
> > > +               return ERR_PTR(-EINVAL);
> > > +
> > > +       fw->type = type;
> > > +       fw->ops = ops;
> > > +       fw->pdev = fw_pdev;
> > > +
> > > +       return fw;
> > > +}
> > > +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_select);
> > > +
> > > +void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
> > > +{
> > > +       switch (fw->type) {
> > > +       case VPU:
> > > +               put_device(&fw->pdev->dev);
> > > +               break;
> > > +       }
> > > +}
> > > +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_release);
> > > +
> >
> > What are these symbols exported for?
>
> This driver is made of three modules: mtk_vcodec_common,
> mtk_vcodec_dec and mtk_vcodec_enc. These functions are used by both
> the encoder and decoder module, so they need to be exported.
>
> There is mistake in the patch though that I noticed while
> investigating your comment: the file defining these functions
> (mtk_vcodec_fw.c) is linked against the mtk_vcodec_dec module while it
> should be part of mtk_vcodec_common, so thanks for asking. :)

Oh, I missed that. Out of curiosity, what's the design decision motivating
this split?

FWIW, I've had a somewhat painful experience with splitted
modules, so I try to minimize that if possible.

If you decide to keep the split, how about you consider
also splitting the user option. Something around two visible
ones: VIDEO_MEDIATEK_VCODEC_ENC/DEC
and one hidden for the _COMMON one.

That would make the design more obvious, I think.

Minor nitpick, currently the Kconfig help states:

"""
            To compile this driver as a module, choose M here: the
            module will be called mtk-vcodec
"""

Thanks,
Ezequiel

Re: [PATCH v3 01/16] media: mtk-vcodec: abstract firmware interface

[Ezequiel Garcia]

On Mon, 27 Jul 2020 at 11:24, Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Mon, 27 Jul 2020 at 06:06, Alexandre Courbot <acourbot@chromium.org> wrote:
> >
> > On Thu, Jul 23, 2020 at 6:23 AM Ezequiel Garcia
> > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
> > > >
> > > > From: Yunfei Dong <yunfei.dong@mediatek.com>
> > > >
> > > > MT8183's codec firwmare is run by a different remote processor from
> > > > MT8173. While the firmware interface is basically the same, the way to
> > > > invoke it differs. Abstract all firmware calls under a layer that will
> > > > allow us to handle both firmware types transparently.
> > > >
> > > > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > > > [acourbot: refactor, cleanup and split]
> > > > Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> > > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > > > [pihsun: fix error path and add mtk_vcodec_fw_release]
> > > > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > > > Reviewed-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > > ---
> > > >  drivers/media/platform/mtk-vcodec/Makefile    |   4 +-
> > > >  .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  50 ++---
> > > >  .../platform/mtk-vcodec/mtk_vcodec_dec_pm.c   |   1 -
> > > >  .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   5 +-
> > > >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  47 ++---
> > > >  .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   |   2 -
> > > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 172 ++++++++++++++++++
> > > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  36 ++++
> > > >  .../platform/mtk-vcodec/mtk_vcodec_util.c     |   1 -
> > > >  .../platform/mtk-vcodec/vdec/vdec_h264_if.c   |   1 -
> > > >  .../platform/mtk-vcodec/vdec/vdec_vp8_if.c    |   1 -
> > > >  .../platform/mtk-vcodec/vdec/vdec_vp9_if.c    |   1 -
> > > >  .../media/platform/mtk-vcodec/vdec_drv_base.h |   2 -
> > > >  .../media/platform/mtk-vcodec/vdec_drv_if.c   |   1 -
> > > >  .../media/platform/mtk-vcodec/vdec_vpu_if.c   |  12 +-
> > > >  .../media/platform/mtk-vcodec/vdec_vpu_if.h   |  11 +-
> > > >  .../platform/mtk-vcodec/venc/venc_h264_if.c   |  15 +-
> > > >  .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   8 +-
> > > >  .../media/platform/mtk-vcodec/venc_drv_if.c   |   1 -
> > > >  .../media/platform/mtk-vcodec/venc_vpu_if.c   |  17 +-
> > > >  .../media/platform/mtk-vcodec/venc_vpu_if.h   |   5 +-
> > > >  21 files changed, 290 insertions(+), 103 deletions(-)
> > > >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
> > > >
> > > > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > > > index 37b94b555fa1..b8636119ed0a 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > > > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > > > @@ -12,7 +12,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
> > > >                 vdec_vpu_if.o \
> > > >                 mtk_vcodec_dec.o \
> > > >                 mtk_vcodec_dec_pm.o \
> > > > -
> > > > +               mtk_vcodec_fw.o
> > > >
> > > >  mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> > > >                 venc/venc_h264_if.o \
> > > > @@ -25,5 +25,3 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> > > >
> > > >  mtk-vcodec-common-y := mtk_vcodec_intr.o \
> > > >                 mtk_vcodec_util.o\
> > > > -
> > > > -ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > index 97a1b6664c20..4f07a5fcce7f 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > @@ -20,7 +20,7 @@
> > > >  #include "mtk_vcodec_dec_pm.h"
> > > >  #include "mtk_vcodec_intr.h"
> > > >  #include "mtk_vcodec_util.h"
> > > > -#include "mtk_vpu.h"
> > > > +#include "mtk_vcodec_fw.h"
> > > >
> > > >  #define VDEC_HW_ACTIVE 0x10
> > > >  #define VDEC_IRQ_CFG   0x11
> > > > @@ -77,22 +77,6 @@ static irqreturn_t mtk_vcodec_dec_irq_handler(int irq, void *priv)
> > > >         return IRQ_HANDLED;
> > > >  }
> > > >
> > > > -static void mtk_vcodec_dec_reset_handler(void *priv)
> > > > -{
> > > > -       struct mtk_vcodec_dev *dev = priv;
> > > > -       struct mtk_vcodec_ctx *ctx;
> > > > -
> > > > -       mtk_v4l2_err("Watchdog timeout!!");
> > > > -
> > > > -       mutex_lock(&dev->dev_mutex);
> > > > -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > > -               ctx->state = MTK_STATE_ABORT;
> > > > -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR",
> > > > -                               ctx->id);
> > > > -       }
> > > > -       mutex_unlock(&dev->dev_mutex);
> > > > -}
> > > > -
> > > >  static int fops_vcodec_open(struct file *file)
> > > >  {
> > > >         struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > > @@ -144,21 +128,20 @@ static int fops_vcodec_open(struct file *file)
> > > >         if (v4l2_fh_is_singular(&ctx->fh)) {
> > > >                 mtk_vcodec_dec_pw_on(&dev->pm);
> > > >                 /*
> > > > -                * vpu_load_firmware checks if it was loaded already and
> > > > -                * does nothing in that case
> > > > +                * Does nothing if firmware was already loaded.
> > > >                  */
> > > > -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > > > +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
> > > >                 if (ret < 0) {
> > > >                         /*
> > > >                          * Return 0 if downloading firmware successfully,
> > > >                          * otherwise it is failed
> > > >                          */
> > > > -                       mtk_v4l2_err("vpu_load_firmware failed!");
> > > > +                       mtk_v4l2_err("failed to load firmware!");
> > > >                         goto err_load_fw;
> > > >                 }
> > > >
> > > >                 dev->dec_capability =
> > > > -                       vpu_get_vdec_hw_capa(dev->vpu_plat_dev);
> > > > +                       mtk_vcodec_fw_get_vdec_capa(dev->fw_handler);
> > > >                 mtk_v4l2_debug(0, "decoder capability %x", dev->dec_capability);
> > > >         }
> > > >
> > > > @@ -228,6 +211,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         struct mtk_vcodec_dev *dev;
> > > >         struct video_device *vfd_dec;
> > > >         struct resource *res;
> > > > +       phandle rproc_phandle;
> > > > +       enum mtk_vcodec_fw_type fw_type;
> > > >         int i, ret;
> > > >
> > > >         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > > > @@ -237,19 +222,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         INIT_LIST_HEAD(&dev->ctx_list);
> > > >         dev->plat_dev = pdev;
> > > >
> > > > -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > > > -       if (dev->vpu_plat_dev == NULL) {
> > > > -               mtk_v4l2_err("[VPU] vpu device in not ready");
> > > > -               return -EPROBE_DEFER;
> > > > +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > > > +                                 &rproc_phandle)) {
> > > > +               fw_type = VPU;
> > > > +       } else {
> > > > +               mtk_v4l2_err("Could not get vdec IPI device");
> > > > +               return -ENODEV;
> > > >         }
> > > > -
> > > > -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_dec_reset_handler,
> > > > -                       dev, VPU_RST_DEC);
> > > > +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_DEC);
> > > > +       if (IS_ERR(dev->fw_handler))
> > > > +               return PTR_ERR(dev->fw_handler);
> > > >
> > > >         ret = mtk_vcodec_init_dec_pm(dev);
> > > >         if (ret < 0) {
> > > >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source");
> > > > -               return ret;
> > > > +               goto err_dec_pm;
> > > >         }
> > > >
> > > >         for (i = 0; i < NUM_MAX_VDEC_REG_BASE; i++) {
> > > > @@ -352,6 +339,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > >  err_res:
> > > >         mtk_vcodec_release_dec_pm(dev);
> > > > +err_dec_pm:
> > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > >         return ret;
> > > >  }
> > > >
> > > > @@ -376,6 +365,7 @@ static int mtk_vcodec_dec_remove(struct platform_device *pdev)
> > > >
> > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > >         mtk_vcodec_release_dec_pm(dev);
> > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > >         return 0;
> > > >  }
> > > >
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > > index 5a6ec8fb52da..36dfe3fc056a 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > > @@ -12,7 +12,6 @@
> > > >
> > > >  #include "mtk_vcodec_dec_pm.h"
> > > >  #include "mtk_vcodec_util.h"
> > > > -#include "mtk_vpu.h"
> > > >
> > > >  int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *mtkdev)
> > > >  {
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > index 9fd56dee7fd1..e132c4ec463a 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > @@ -309,13 +309,13 @@ struct mtk_vcodec_ctx {
> > > >   * @m2m_dev_dec: m2m device for decoder
> > > >   * @m2m_dev_enc: m2m device for encoder.
> > > >   * @plat_dev: platform device
> > > > - * @vpu_plat_dev: mtk vpu platform device
> > > >   * @ctx_list: list of struct mtk_vcodec_ctx
> > > >   * @irqlock: protect data access by irq handler and work thread
> > > >   * @curr_ctx: The context that is waiting for codec hardware
> > > >   *
> > > >   * @reg_base: Mapped address of MTK Vcodec registers.
> > > >   *
> > > > + * @fw_handler: used to communicate with the firmware.
> > > >   * @id_counter: used to identify current opened instance
> > > >   *
> > > >   * @encode_workqueue: encode work queue
> > > > @@ -344,12 +344,13 @@ struct mtk_vcodec_dev {
> > > >         struct v4l2_m2m_dev *m2m_dev_dec;
> > > >         struct v4l2_m2m_dev *m2m_dev_enc;
> > > >         struct platform_device *plat_dev;
> > > > -       struct platform_device *vpu_plat_dev;
> > > >         struct list_head ctx_list;
> > > >         spinlock_t irqlock;
> > > >         struct mtk_vcodec_ctx *curr_ctx;
> > > >         void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
> > > >
> > > > +       struct mtk_vcodec_fw *fw_handler;
> > > > +
> > > >         unsigned long id_counter;
> > > >
> > > >         struct workqueue_struct *decode_workqueue;
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > index 4d31f1ed113f..4340ea10afd0 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > @@ -21,7 +21,7 @@
> > > >  #include "mtk_vcodec_enc_pm.h"
> > > >  #include "mtk_vcodec_intr.h"
> > > >  #include "mtk_vcodec_util.h"
> > > > -#include "mtk_vpu.h"
> > > > +#include "mtk_vcodec_fw.h"
> > > >
> > > >  module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> > > >  module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> > > > @@ -101,22 +101,6 @@ static irqreturn_t mtk_vcodec_enc_lt_irq_handler(int irq, void *priv)
> > > >         return IRQ_HANDLED;
> > > >  }
> > > >
> > > > -static void mtk_vcodec_enc_reset_handler(void *priv)
> > > > -{
> > > > -       struct mtk_vcodec_dev *dev = priv;
> > > > -       struct mtk_vcodec_ctx *ctx;
> > > > -
> > > > -       mtk_v4l2_debug(0, "Watchdog timeout!!");
> > > > -
> > > > -       mutex_lock(&dev->dev_mutex);
> > > > -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > > -               ctx->state = MTK_STATE_ABORT;
> > > > -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> > > > -                               ctx->id);
> > > > -       }
> > > > -       mutex_unlock(&dev->dev_mutex);
> > > > -}
> > > > -
> > > >  static int fops_vcodec_open(struct file *file)
> > > >  {
> > > >         struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > > @@ -159,10 +143,10 @@ static int fops_vcodec_open(struct file *file)
> > > >
> > > >         if (v4l2_fh_is_singular(&ctx->fh)) {
> > > >                 /*
> > > > -                * vpu_load_firmware checks if it was loaded already and
> > > > +                * load fireware to checks if it was loaded already and
> > > >                  * does nothing in that case
> > > >                  */
> > > > -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > > > +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
> > > >                 if (ret < 0) {
> > > >                         /*
> > > >                          * Return 0 if downloading firmware successfully,
> > > > @@ -173,7 +157,7 @@ static int fops_vcodec_open(struct file *file)
> > > >                 }
> > > >
> > > >                 dev->enc_capability =
> > > > -                       vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > > > +                       mtk_vcodec_fw_get_venc_capa(dev->fw_handler);
> > > >                 mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> > > >         }
> > > >
> > > > @@ -235,6 +219,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         struct mtk_vcodec_dev *dev;
> > > >         struct video_device *vfd_enc;
> > > >         struct resource *res;
> > > > +       phandle rproc_phandle;
> > > > +       enum mtk_vcodec_fw_type fw_type;
> > > >         int i, j, ret;
> > > >
> > > >         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > > > @@ -244,19 +230,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         INIT_LIST_HEAD(&dev->ctx_list);
> > > >         dev->plat_dev = pdev;
> > > >
> > > > -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > > > -       if (dev->vpu_plat_dev == NULL) {
> > > > -               mtk_v4l2_err("[VPU] vpu device in not ready");
> > > > -               return -EPROBE_DEFER;
> > > > +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > > > +                                 &rproc_phandle)) {
> > > > +               fw_type = VPU;
> > > > +       } else {
> > > > +               mtk_v4l2_err("Could not get venc IPI device");
> > > > +               return -ENODEV;
> > > >         }
> > > > -
> > > > -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler,
> > > > -                               dev, VPU_RST_ENC);
> > > > +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_ENC);
> > > > +       if (IS_ERR(dev->fw_handler))
> > > > +               return PTR_ERR(dev->fw_handler);
> > > >
> > > >         ret = mtk_vcodec_init_enc_pm(dev);
> > > >         if (ret < 0) {
> > > >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> > > > -               return ret;
> > > > +               goto err_enc_pm;
> > > >         }
> > > >
> > > >         for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > > > @@ -377,6 +365,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > >  err_res:
> > > >         mtk_vcodec_release_enc_pm(dev);
> > > > +err_enc_pm:
> > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > >         return ret;
> > > >  }
> > > >
> > > > @@ -401,6 +391,7 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
> > > >
> > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > >         mtk_vcodec_release_enc_pm(dev);
> > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > >         return 0;
> > > >  }
> > > >
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > > index 3e2bfded79a6..ee22902aaa71 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > > @@ -12,8 +12,6 @@
> > > >
> > > >  #include "mtk_vcodec_enc_pm.h"
> > > >  #include "mtk_vcodec_util.h"
> > > > -#include "mtk_vpu.h"
> > > > -
> > > >
> > > >  int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > > >  {
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > > new file mode 100644
> > > > index 000000000000..967bb100a990
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > > @@ -0,0 +1,172 @@
> > > > +// SPDX-License-Identifier: GPL-2.0
> > > > +
> > > > +#include "mtk_vcodec_fw.h"
> > > > +#include "mtk_vcodec_util.h"
> > > > +#include "mtk_vcodec_drv.h"
> > > > +
> > > > +struct mtk_vcodec_fw_ops {
> > > > +       int (*load_firmware)(struct mtk_vcodec_fw *fw);
> > > > +       unsigned int (*get_vdec_capa)(struct mtk_vcodec_fw *fw);
> > > > +       unsigned int (*get_venc_capa)(struct mtk_vcodec_fw *fw);
> > > > +       void * (*map_dm_addr)(struct mtk_vcodec_fw *fw, u32 dtcm_dmem_addr);
> > > > +       int (*ipi_register)(struct mtk_vcodec_fw *fw, int id,
> > > > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv);
> > > > +       int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
> > > > +               unsigned int len, unsigned int wait);
> > > > +};
> > > > +
> > > > +struct mtk_vcodec_fw {
> > > > +       enum mtk_vcodec_fw_type type;
> > > > +       const struct mtk_vcodec_fw_ops *ops;
> > > > +       struct platform_device *pdev;
> > > > +};
> > > > +
> > > > +static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
> > > > +{
> > > > +       return vpu_load_firmware(fw->pdev);
> > > > +}
> > > > +
> > > > +static unsigned int mtk_vcodec_vpu_get_vdec_capa(struct mtk_vcodec_fw *fw)
> > > > +{
> > > > +       return vpu_get_vdec_hw_capa(fw->pdev);
> > > > +}
> > > > +
> > > > +static unsigned int mtk_vcodec_vpu_get_venc_capa(struct mtk_vcodec_fw *fw)
> > > > +{
> > > > +       return vpu_get_venc_hw_capa(fw->pdev);
> > > > +}
> > > > +
> > > > +static void *mtk_vcodec_vpu_map_dm_addr(struct mtk_vcodec_fw *fw,
> > > > +                                       u32 dtcm_dmem_addr)
> > > > +{
> > > > +       return vpu_mapping_dm_addr(fw->pdev, dtcm_dmem_addr);
> > > > +}
> > > > +
> > > > +static int mtk_vcodec_vpu_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
> > > > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv)
> > > > +{
> > > > +       /*
> > > > +        * The handler we receive takes a void * as its first argument. We
> > > > +        * cannot change this because it needs to be passed down to the rproc
> > > > +        * subsystem when SCP is used. VPU takes a const argument, which is
> > > > +        * more constrained, so the conversion below is safe.
> > > > +        */
> > > > +       ipi_handler_t handler_const = (ipi_handler_t)handler;
> > > > +
> > > > +       return vpu_ipi_register(fw->pdev, id, handler_const, name, priv);
> > > > +}
> > > > +
> > > > +static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
> > > > +               unsigned int len, unsigned int wait)
> > > > +{
> > > > +       return vpu_ipi_send(fw->pdev, id, buf, len);
> > > > +}
> > > > +
> > > > +static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
> > > > +       .load_firmware = mtk_vcodec_vpu_load_firmware,
> > > > +       .get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
> > > > +       .get_venc_capa = mtk_vcodec_vpu_get_venc_capa,
> > > > +       .map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
> > > > +       .ipi_register = mtk_vcodec_vpu_set_ipi_register,
> > > > +       .ipi_send = mtk_vcodec_vpu_ipi_send,
> > > > +};
> > > > +
> > > > +static void mtk_vcodec_reset_handler(void *priv)
> > > > +{
> > > > +       struct mtk_vcodec_dev *dev = priv;
> > > > +       struct mtk_vcodec_ctx *ctx;
> > > > +
> > > > +       mtk_v4l2_err("Watchdog timeout!!");
> > > > +
> > > > +       mutex_lock(&dev->dev_mutex);
> > > > +       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > > +               ctx->state = MTK_STATE_ABORT;
> > > > +               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> > > > +                               ctx->id);
> > > > +       }
> > > > +       mutex_unlock(&dev->dev_mutex);
> > > > +}
> > > > +
> > > > +struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> > > > +                                          enum mtk_vcodec_fw_type type,
> > > > +                                          enum rst_id rst_id)
> > > > +{
> > > > +       const struct mtk_vcodec_fw_ops *ops;
> > > > +       struct mtk_vcodec_fw *fw;
> > > > +       struct platform_device *fw_pdev = NULL;
> > > > +
> > > > +       switch (type) {
> > > > +       case VPU:
> > > > +               ops = &mtk_vcodec_vpu_msg;
> > > > +               fw_pdev = vpu_get_plat_device(dev->plat_dev);
> > > > +               if (!fw_pdev) {
> > > > +                       mtk_v4l2_err("firmware device is not ready");
> > > > +                       return ERR_PTR(-EINVAL);
> > > > +               }
> > > > +               vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
> > > > +                                   dev, rst_id);
> > > > +               break;
> > > > +       default:
> > > > +               mtk_v4l2_err("invalid vcodec fw type");
> > > > +               return ERR_PTR(-EINVAL);
> > > > +       }
> > > > +
> > > > +       fw = devm_kzalloc(&dev->plat_dev->dev, sizeof(*fw), GFP_KERNEL);
> > > > +       if (!fw)
> > > > +               return ERR_PTR(-EINVAL);
> > > > +
> > > > +       fw->type = type;
> > > > +       fw->ops = ops;
> > > > +       fw->pdev = fw_pdev;
> > > > +
> > > > +       return fw;
> > > > +}
> > > > +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_select);
> > > > +
> > > > +void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
> > > > +{
> > > > +       switch (fw->type) {
> > > > +       case VPU:
> > > > +               put_device(&fw->pdev->dev);
> > > > +               break;
> > > > +       }
> > > > +}
> > > > +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_release);
> > > > +
> > >
> > > What are these symbols exported for?
> >
> > This driver is made of three modules: mtk_vcodec_common,
> > mtk_vcodec_dec and mtk_vcodec_enc. These functions are used by both
> > the encoder and decoder module, so they need to be exported.
> >
> > There is mistake in the patch though that I noticed while
> > investigating your comment: the file defining these functions
> > (mtk_vcodec_fw.c) is linked against the mtk_vcodec_dec module while it
> > should be part of mtk_vcodec_common, so thanks for asking. :)
>
> Oh, I missed that. Out of curiosity, what's the design decision motivating
> this split?
>
> FWIW, I've had a somewhat painful experience with splitted
> modules, so I try to minimize that if possible.
>
> If you decide to keep the split, how about you consider
> also splitting the user option. Something around two visible
> ones: VIDEO_MEDIATEK_VCODEC_ENC/DEC
> and one hidden for the _COMMON one.
>
> That would make the design more obvious, I think.
>
> Minor nitpick, currently the Kconfig help states:
>
> """
>             To compile this driver as a module, choose M here: the
>             module will be called mtk-vcodec
> """
>

Ah, one more thing.

In order to build this with COMPILE_TEST, I believe you need
something along these lines:

diff --git a/drivers/remoteproc/Kconfig b/drivers/remoteproc/Kconfig
index c4d1731295eb..ee83f407b9a3 100644
--- a/drivers/remoteproc/Kconfig
+++ b/drivers/remoteproc/Kconfig
@@ -34,7 +34,7 @@ config INGENIC_VPU_RPROC

 config MTK_SCP
        tristate "Mediatek SCP support"
-       depends on ARCH_MEDIATEK
+       depends on ARCH_MEDIATEK || COMPILE_TEST
        select RPMSG_MTK_SCP
        help
          Say y here to support Mediatek's System Companion Processor (SCP) via

Thanks,
Ezequiel

Re: [PATCH v3 01/16] media: mtk-vcodec: abstract firmware interface

[Alexandre Courbot]

On Mon, Jul 27, 2020 at 11:24 PM Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Mon, 27 Jul 2020 at 06:06, Alexandre Courbot <acourbot@chromium.org> wrote:
> >
> > On Thu, Jul 23, 2020 at 6:23 AM Ezequiel Garcia
> > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
> > > >
> > > > From: Yunfei Dong <yunfei.dong@mediatek.com>
> > > >
> > > > MT8183's codec firwmare is run by a different remote processor from
> > > > MT8173. While the firmware interface is basically the same, the way to
> > > > invoke it differs. Abstract all firmware calls under a layer that will
> > > > allow us to handle both firmware types transparently.
> > > >
> > > > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > > > [acourbot: refactor, cleanup and split]
> > > > Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> > > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > > > [pihsun: fix error path and add mtk_vcodec_fw_release]
> > > > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > > > Reviewed-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > > ---
> > > >  drivers/media/platform/mtk-vcodec/Makefile    |   4 +-
> > > >  .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  50 ++---
> > > >  .../platform/mtk-vcodec/mtk_vcodec_dec_pm.c   |   1 -
> > > >  .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   5 +-
> > > >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  47 ++---
> > > >  .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   |   2 -
> > > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 172 ++++++++++++++++++
> > > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  36 ++++
> > > >  .../platform/mtk-vcodec/mtk_vcodec_util.c     |   1 -
> > > >  .../platform/mtk-vcodec/vdec/vdec_h264_if.c   |   1 -
> > > >  .../platform/mtk-vcodec/vdec/vdec_vp8_if.c    |   1 -
> > > >  .../platform/mtk-vcodec/vdec/vdec_vp9_if.c    |   1 -
> > > >  .../media/platform/mtk-vcodec/vdec_drv_base.h |   2 -
> > > >  .../media/platform/mtk-vcodec/vdec_drv_if.c   |   1 -
> > > >  .../media/platform/mtk-vcodec/vdec_vpu_if.c   |  12 +-
> > > >  .../media/platform/mtk-vcodec/vdec_vpu_if.h   |  11 +-
> > > >  .../platform/mtk-vcodec/venc/venc_h264_if.c   |  15 +-
> > > >  .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   8 +-
> > > >  .../media/platform/mtk-vcodec/venc_drv_if.c   |   1 -
> > > >  .../media/platform/mtk-vcodec/venc_vpu_if.c   |  17 +-
> > > >  .../media/platform/mtk-vcodec/venc_vpu_if.h   |   5 +-
> > > >  21 files changed, 290 insertions(+), 103 deletions(-)
> > > >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
> > > >
> > > > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > > > index 37b94b555fa1..b8636119ed0a 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > > > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > > > @@ -12,7 +12,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
> > > >                 vdec_vpu_if.o \
> > > >                 mtk_vcodec_dec.o \
> > > >                 mtk_vcodec_dec_pm.o \
> > > > -
> > > > +               mtk_vcodec_fw.o
> > > >
> > > >  mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> > > >                 venc/venc_h264_if.o \
> > > > @@ -25,5 +25,3 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> > > >
> > > >  mtk-vcodec-common-y := mtk_vcodec_intr.o \
> > > >                 mtk_vcodec_util.o\
> > > > -
> > > > -ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > index 97a1b6664c20..4f07a5fcce7f 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > @@ -20,7 +20,7 @@
> > > >  #include "mtk_vcodec_dec_pm.h"
> > > >  #include "mtk_vcodec_intr.h"
> > > >  #include "mtk_vcodec_util.h"
> > > > -#include "mtk_vpu.h"
> > > > +#include "mtk_vcodec_fw.h"
> > > >
> > > >  #define VDEC_HW_ACTIVE 0x10
> > > >  #define VDEC_IRQ_CFG   0x11
> > > > @@ -77,22 +77,6 @@ static irqreturn_t mtk_vcodec_dec_irq_handler(int irq, void *priv)
> > > >         return IRQ_HANDLED;
> > > >  }
> > > >
> > > > -static void mtk_vcodec_dec_reset_handler(void *priv)
> > > > -{
> > > > -       struct mtk_vcodec_dev *dev = priv;
> > > > -       struct mtk_vcodec_ctx *ctx;
> > > > -
> > > > -       mtk_v4l2_err("Watchdog timeout!!");
> > > > -
> > > > -       mutex_lock(&dev->dev_mutex);
> > > > -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > > -               ctx->state = MTK_STATE_ABORT;
> > > > -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR",
> > > > -                               ctx->id);
> > > > -       }
> > > > -       mutex_unlock(&dev->dev_mutex);
> > > > -}
> > > > -
> > > >  static int fops_vcodec_open(struct file *file)
> > > >  {
> > > >         struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > > @@ -144,21 +128,20 @@ static int fops_vcodec_open(struct file *file)
> > > >         if (v4l2_fh_is_singular(&ctx->fh)) {
> > > >                 mtk_vcodec_dec_pw_on(&dev->pm);
> > > >                 /*
> > > > -                * vpu_load_firmware checks if it was loaded already and
> > > > -                * does nothing in that case
> > > > +                * Does nothing if firmware was already loaded.
> > > >                  */
> > > > -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > > > +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
> > > >                 if (ret < 0) {
> > > >                         /*
> > > >                          * Return 0 if downloading firmware successfully,
> > > >                          * otherwise it is failed
> > > >                          */
> > > > -                       mtk_v4l2_err("vpu_load_firmware failed!");
> > > > +                       mtk_v4l2_err("failed to load firmware!");
> > > >                         goto err_load_fw;
> > > >                 }
> > > >
> > > >                 dev->dec_capability =
> > > > -                       vpu_get_vdec_hw_capa(dev->vpu_plat_dev);
> > > > +                       mtk_vcodec_fw_get_vdec_capa(dev->fw_handler);
> > > >                 mtk_v4l2_debug(0, "decoder capability %x", dev->dec_capability);
> > > >         }
> > > >
> > > > @@ -228,6 +211,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         struct mtk_vcodec_dev *dev;
> > > >         struct video_device *vfd_dec;
> > > >         struct resource *res;
> > > > +       phandle rproc_phandle;
> > > > +       enum mtk_vcodec_fw_type fw_type;
> > > >         int i, ret;
> > > >
> > > >         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > > > @@ -237,19 +222,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         INIT_LIST_HEAD(&dev->ctx_list);
> > > >         dev->plat_dev = pdev;
> > > >
> > > > -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > > > -       if (dev->vpu_plat_dev == NULL) {
> > > > -               mtk_v4l2_err("[VPU] vpu device in not ready");
> > > > -               return -EPROBE_DEFER;
> > > > +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > > > +                                 &rproc_phandle)) {
> > > > +               fw_type = VPU;
> > > > +       } else {
> > > > +               mtk_v4l2_err("Could not get vdec IPI device");
> > > > +               return -ENODEV;
> > > >         }
> > > > -
> > > > -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_dec_reset_handler,
> > > > -                       dev, VPU_RST_DEC);
> > > > +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_DEC);
> > > > +       if (IS_ERR(dev->fw_handler))
> > > > +               return PTR_ERR(dev->fw_handler);
> > > >
> > > >         ret = mtk_vcodec_init_dec_pm(dev);
> > > >         if (ret < 0) {
> > > >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source");
> > > > -               return ret;
> > > > +               goto err_dec_pm;
> > > >         }
> > > >
> > > >         for (i = 0; i < NUM_MAX_VDEC_REG_BASE; i++) {
> > > > @@ -352,6 +339,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > >  err_res:
> > > >         mtk_vcodec_release_dec_pm(dev);
> > > > +err_dec_pm:
> > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > >         return ret;
> > > >  }
> > > >
> > > > @@ -376,6 +365,7 @@ static int mtk_vcodec_dec_remove(struct platform_device *pdev)
> > > >
> > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > >         mtk_vcodec_release_dec_pm(dev);
> > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > >         return 0;
> > > >  }
> > > >
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > > index 5a6ec8fb52da..36dfe3fc056a 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > > @@ -12,7 +12,6 @@
> > > >
> > > >  #include "mtk_vcodec_dec_pm.h"
> > > >  #include "mtk_vcodec_util.h"
> > > > -#include "mtk_vpu.h"
> > > >
> > > >  int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *mtkdev)
> > > >  {
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > index 9fd56dee7fd1..e132c4ec463a 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > @@ -309,13 +309,13 @@ struct mtk_vcodec_ctx {
> > > >   * @m2m_dev_dec: m2m device for decoder
> > > >   * @m2m_dev_enc: m2m device for encoder.
> > > >   * @plat_dev: platform device
> > > > - * @vpu_plat_dev: mtk vpu platform device
> > > >   * @ctx_list: list of struct mtk_vcodec_ctx
> > > >   * @irqlock: protect data access by irq handler and work thread
> > > >   * @curr_ctx: The context that is waiting for codec hardware
> > > >   *
> > > >   * @reg_base: Mapped address of MTK Vcodec registers.
> > > >   *
> > > > + * @fw_handler: used to communicate with the firmware.
> > > >   * @id_counter: used to identify current opened instance
> > > >   *
> > > >   * @encode_workqueue: encode work queue
> > > > @@ -344,12 +344,13 @@ struct mtk_vcodec_dev {
> > > >         struct v4l2_m2m_dev *m2m_dev_dec;
> > > >         struct v4l2_m2m_dev *m2m_dev_enc;
> > > >         struct platform_device *plat_dev;
> > > > -       struct platform_device *vpu_plat_dev;
> > > >         struct list_head ctx_list;
> > > >         spinlock_t irqlock;
> > > >         struct mtk_vcodec_ctx *curr_ctx;
> > > >         void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
> > > >
> > > > +       struct mtk_vcodec_fw *fw_handler;
> > > > +
> > > >         unsigned long id_counter;
> > > >
> > > >         struct workqueue_struct *decode_workqueue;
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > index 4d31f1ed113f..4340ea10afd0 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > @@ -21,7 +21,7 @@
> > > >  #include "mtk_vcodec_enc_pm.h"
> > > >  #include "mtk_vcodec_intr.h"
> > > >  #include "mtk_vcodec_util.h"
> > > > -#include "mtk_vpu.h"
> > > > +#include "mtk_vcodec_fw.h"
> > > >
> > > >  module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> > > >  module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> > > > @@ -101,22 +101,6 @@ static irqreturn_t mtk_vcodec_enc_lt_irq_handler(int irq, void *priv)
> > > >         return IRQ_HANDLED;
> > > >  }
> > > >
> > > > -static void mtk_vcodec_enc_reset_handler(void *priv)
> > > > -{
> > > > -       struct mtk_vcodec_dev *dev = priv;
> > > > -       struct mtk_vcodec_ctx *ctx;
> > > > -
> > > > -       mtk_v4l2_debug(0, "Watchdog timeout!!");
> > > > -
> > > > -       mutex_lock(&dev->dev_mutex);
> > > > -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > > -               ctx->state = MTK_STATE_ABORT;
> > > > -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> > > > -                               ctx->id);
> > > > -       }
> > > > -       mutex_unlock(&dev->dev_mutex);
> > > > -}
> > > > -
> > > >  static int fops_vcodec_open(struct file *file)
> > > >  {
> > > >         struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > > @@ -159,10 +143,10 @@ static int fops_vcodec_open(struct file *file)
> > > >
> > > >         if (v4l2_fh_is_singular(&ctx->fh)) {
> > > >                 /*
> > > > -                * vpu_load_firmware checks if it was loaded already and
> > > > +                * load fireware to checks if it was loaded already and
> > > >                  * does nothing in that case
> > > >                  */
> > > > -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > > > +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
> > > >                 if (ret < 0) {
> > > >                         /*
> > > >                          * Return 0 if downloading firmware successfully,
> > > > @@ -173,7 +157,7 @@ static int fops_vcodec_open(struct file *file)
> > > >                 }
> > > >
> > > >                 dev->enc_capability =
> > > > -                       vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > > > +                       mtk_vcodec_fw_get_venc_capa(dev->fw_handler);
> > > >                 mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> > > >         }
> > > >
> > > > @@ -235,6 +219,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         struct mtk_vcodec_dev *dev;
> > > >         struct video_device *vfd_enc;
> > > >         struct resource *res;
> > > > +       phandle rproc_phandle;
> > > > +       enum mtk_vcodec_fw_type fw_type;
> > > >         int i, j, ret;
> > > >
> > > >         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > > > @@ -244,19 +230,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         INIT_LIST_HEAD(&dev->ctx_list);
> > > >         dev->plat_dev = pdev;
> > > >
> > > > -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > > > -       if (dev->vpu_plat_dev == NULL) {
> > > > -               mtk_v4l2_err("[VPU] vpu device in not ready");
> > > > -               return -EPROBE_DEFER;
> > > > +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > > > +                                 &rproc_phandle)) {
> > > > +               fw_type = VPU;
> > > > +       } else {
> > > > +               mtk_v4l2_err("Could not get venc IPI device");
> > > > +               return -ENODEV;
> > > >         }
> > > > -
> > > > -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler,
> > > > -                               dev, VPU_RST_ENC);
> > > > +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_ENC);
> > > > +       if (IS_ERR(dev->fw_handler))
> > > > +               return PTR_ERR(dev->fw_handler);
> > > >
> > > >         ret = mtk_vcodec_init_enc_pm(dev);
> > > >         if (ret < 0) {
> > > >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> > > > -               return ret;
> > > > +               goto err_enc_pm;
> > > >         }
> > > >
> > > >         for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > > > @@ -377,6 +365,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > >  err_res:
> > > >         mtk_vcodec_release_enc_pm(dev);
> > > > +err_enc_pm:
> > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > >         return ret;
> > > >  }
> > > >
> > > > @@ -401,6 +391,7 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
> > > >
> > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > >         mtk_vcodec_release_enc_pm(dev);
> > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > >         return 0;
> > > >  }
> > > >
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > > index 3e2bfded79a6..ee22902aaa71 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > > @@ -12,8 +12,6 @@
> > > >
> > > >  #include "mtk_vcodec_enc_pm.h"
> > > >  #include "mtk_vcodec_util.h"
> > > > -#include "mtk_vpu.h"
> > > > -
> > > >
> > > >  int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > > >  {
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > > new file mode 100644
> > > > index 000000000000..967bb100a990
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > > @@ -0,0 +1,172 @@
> > > > +// SPDX-License-Identifier: GPL-2.0
> > > > +
> > > > +#include "mtk_vcodec_fw.h"
> > > > +#include "mtk_vcodec_util.h"
> > > > +#include "mtk_vcodec_drv.h"
> > > > +
> > > > +struct mtk_vcodec_fw_ops {
> > > > +       int (*load_firmware)(struct mtk_vcodec_fw *fw);
> > > > +       unsigned int (*get_vdec_capa)(struct mtk_vcodec_fw *fw);
> > > > +       unsigned int (*get_venc_capa)(struct mtk_vcodec_fw *fw);
> > > > +       void * (*map_dm_addr)(struct mtk_vcodec_fw *fw, u32 dtcm_dmem_addr);
> > > > +       int (*ipi_register)(struct mtk_vcodec_fw *fw, int id,
> > > > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv);
> > > > +       int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
> > > > +               unsigned int len, unsigned int wait);
> > > > +};
> > > > +
> > > > +struct mtk_vcodec_fw {
> > > > +       enum mtk_vcodec_fw_type type;
> > > > +       const struct mtk_vcodec_fw_ops *ops;
> > > > +       struct platform_device *pdev;
> > > > +};
> > > > +
> > > > +static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
> > > > +{
> > > > +       return vpu_load_firmware(fw->pdev);
> > > > +}
> > > > +
> > > > +static unsigned int mtk_vcodec_vpu_get_vdec_capa(struct mtk_vcodec_fw *fw)
> > > > +{
> > > > +       return vpu_get_vdec_hw_capa(fw->pdev);
> > > > +}
> > > > +
> > > > +static unsigned int mtk_vcodec_vpu_get_venc_capa(struct mtk_vcodec_fw *fw)
> > > > +{
> > > > +       return vpu_get_venc_hw_capa(fw->pdev);
> > > > +}
> > > > +
> > > > +static void *mtk_vcodec_vpu_map_dm_addr(struct mtk_vcodec_fw *fw,
> > > > +                                       u32 dtcm_dmem_addr)
> > > > +{
> > > > +       return vpu_mapping_dm_addr(fw->pdev, dtcm_dmem_addr);
> > > > +}
> > > > +
> > > > +static int mtk_vcodec_vpu_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
> > > > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv)
> > > > +{
> > > > +       /*
> > > > +        * The handler we receive takes a void * as its first argument. We
> > > > +        * cannot change this because it needs to be passed down to the rproc
> > > > +        * subsystem when SCP is used. VPU takes a const argument, which is
> > > > +        * more constrained, so the conversion below is safe.
> > > > +        */
> > > > +       ipi_handler_t handler_const = (ipi_handler_t)handler;
> > > > +
> > > > +       return vpu_ipi_register(fw->pdev, id, handler_const, name, priv);
> > > > +}
> > > > +
> > > > +static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
> > > > +               unsigned int len, unsigned int wait)
> > > > +{
> > > > +       return vpu_ipi_send(fw->pdev, id, buf, len);
> > > > +}
> > > > +
> > > > +static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
> > > > +       .load_firmware = mtk_vcodec_vpu_load_firmware,
> > > > +       .get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
> > > > +       .get_venc_capa = mtk_vcodec_vpu_get_venc_capa,
> > > > +       .map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
> > > > +       .ipi_register = mtk_vcodec_vpu_set_ipi_register,
> > > > +       .ipi_send = mtk_vcodec_vpu_ipi_send,
> > > > +};
> > > > +
> > > > +static void mtk_vcodec_reset_handler(void *priv)
> > > > +{
> > > > +       struct mtk_vcodec_dev *dev = priv;
> > > > +       struct mtk_vcodec_ctx *ctx;
> > > > +
> > > > +       mtk_v4l2_err("Watchdog timeout!!");
> > > > +
> > > > +       mutex_lock(&dev->dev_mutex);
> > > > +       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > > +               ctx->state = MTK_STATE_ABORT;
> > > > +               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> > > > +                               ctx->id);
> > > > +       }
> > > > +       mutex_unlock(&dev->dev_mutex);
> > > > +}
> > > > +
> > > > +struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> > > > +                                          enum mtk_vcodec_fw_type type,
> > > > +                                          enum rst_id rst_id)
> > > > +{
> > > > +       const struct mtk_vcodec_fw_ops *ops;
> > > > +       struct mtk_vcodec_fw *fw;
> > > > +       struct platform_device *fw_pdev = NULL;
> > > > +
> > > > +       switch (type) {
> > > > +       case VPU:
> > > > +               ops = &mtk_vcodec_vpu_msg;
> > > > +               fw_pdev = vpu_get_plat_device(dev->plat_dev);
> > > > +               if (!fw_pdev) {
> > > > +                       mtk_v4l2_err("firmware device is not ready");
> > > > +                       return ERR_PTR(-EINVAL);
> > > > +               }
> > > > +               vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
> > > > +                                   dev, rst_id);
> > > > +               break;
> > > > +       default:
> > > > +               mtk_v4l2_err("invalid vcodec fw type");
> > > > +               return ERR_PTR(-EINVAL);
> > > > +       }
> > > > +
> > > > +       fw = devm_kzalloc(&dev->plat_dev->dev, sizeof(*fw), GFP_KERNEL);
> > > > +       if (!fw)
> > > > +               return ERR_PTR(-EINVAL);
> > > > +
> > > > +       fw->type = type;
> > > > +       fw->ops = ops;
> > > > +       fw->pdev = fw_pdev;
> > > > +
> > > > +       return fw;
> > > > +}
> > > > +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_select);
> > > > +
> > > > +void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
> > > > +{
> > > > +       switch (fw->type) {
> > > > +       case VPU:
> > > > +               put_device(&fw->pdev->dev);
> > > > +               break;
> > > > +       }
> > > > +}
> > > > +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_release);
> > > > +
> > >
> > > What are these symbols exported for?
> >
> > This driver is made of three modules: mtk_vcodec_common,
> > mtk_vcodec_dec and mtk_vcodec_enc. These functions are used by both
> > the encoder and decoder module, so they need to be exported.
> >
> > There is mistake in the patch though that I noticed while
> > investigating your comment: the file defining these functions
> > (mtk_vcodec_fw.c) is linked against the mtk_vcodec_dec module while it
> > should be part of mtk_vcodec_common, so thanks for asking. :)
>
> Oh, I missed that. Out of curiosity, what's the design decision motivating
> this split?

I don't know, but it was there. I suppose this is because the decoder
and encoder are two different devices with their own probe functions?

>
> FWIW, I've had a somewhat painful experience with splitted
> modules, so I try to minimize that if possible.
>
> If you decide to keep the split, how about you consider
> also splitting the user option. Something around two visible
> ones: VIDEO_MEDIATEK_VCODEC_ENC/DEC
> and one hidden for the _COMMON one.
>
> That would make the design more obvious, I think.

Agreed, let me do this outside of this series.

>
> Minor nitpick, currently the Kconfig help states:
>
> """
>             To compile this driver as a module, choose M here: the
>             module will be called mtk-vcodec
> """

Ah, nice catch! Fixed in a new patch.

>
> Thanks,
> Ezequiel

Re: [PATCH v3 01/16] media: mtk-vcodec: abstract firmware interface

[Alexandre Courbot]

On Mon, Jul 27, 2020 at 11:25 PM Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Mon, 27 Jul 2020 at 11:24, Ezequiel Garcia
> <ezequiel@vanguardiasur.com.ar> wrote:
> >
> > On Mon, 27 Jul 2020 at 06:06, Alexandre Courbot <acourbot@chromium.org> wrote:
> > >
> > > On Thu, Jul 23, 2020 at 6:23 AM Ezequiel Garcia
> > > <ezequiel@vanguardiasur.com.ar> wrote:
> > > >
> > > > On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
> > > > >
> > > > > From: Yunfei Dong <yunfei.dong@mediatek.com>
> > > > >
> > > > > MT8183's codec firwmare is run by a different remote processor from
> > > > > MT8173. While the firmware interface is basically the same, the way to
> > > > > invoke it differs. Abstract all firmware calls under a layer that will
> > > > > allow us to handle both firmware types transparently.
> > > > >
> > > > > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > > > > [acourbot: refactor, cleanup and split]
> > > > > Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> > > > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > > > > [pihsun: fix error path and add mtk_vcodec_fw_release]
> > > > > Signed-off-by: Pi-Hsun Shih <pihsun@chromium.org>
> > > > > Reviewed-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > > > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > > > ---
> > > > >  drivers/media/platform/mtk-vcodec/Makefile    |   4 +-
> > > > >  .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  50 ++---
> > > > >  .../platform/mtk-vcodec/mtk_vcodec_dec_pm.c   |   1 -
> > > > >  .../platform/mtk-vcodec/mtk_vcodec_drv.h      |   5 +-
> > > > >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  47 ++---
> > > > >  .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   |   2 -
> > > > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 172 ++++++++++++++++++
> > > > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  36 ++++
> > > > >  .../platform/mtk-vcodec/mtk_vcodec_util.c     |   1 -
> > > > >  .../platform/mtk-vcodec/vdec/vdec_h264_if.c   |   1 -
> > > > >  .../platform/mtk-vcodec/vdec/vdec_vp8_if.c    |   1 -
> > > > >  .../platform/mtk-vcodec/vdec/vdec_vp9_if.c    |   1 -
> > > > >  .../media/platform/mtk-vcodec/vdec_drv_base.h |   2 -
> > > > >  .../media/platform/mtk-vcodec/vdec_drv_if.c   |   1 -
> > > > >  .../media/platform/mtk-vcodec/vdec_vpu_if.c   |  12 +-
> > > > >  .../media/platform/mtk-vcodec/vdec_vpu_if.h   |  11 +-
> > > > >  .../platform/mtk-vcodec/venc/venc_h264_if.c   |  15 +-
> > > > >  .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   8 +-
> > > > >  .../media/platform/mtk-vcodec/venc_drv_if.c   |   1 -
> > > > >  .../media/platform/mtk-vcodec/venc_vpu_if.c   |  17 +-
> > > > >  .../media/platform/mtk-vcodec/venc_vpu_if.h   |   5 +-
> > > > >  21 files changed, 290 insertions(+), 103 deletions(-)
> > > > >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > > >  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.h
> > > > >
> > > > > diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile
> > > > > index 37b94b555fa1..b8636119ed0a 100644
> > > > > --- a/drivers/media/platform/mtk-vcodec/Makefile
> > > > > +++ b/drivers/media/platform/mtk-vcodec/Makefile
> > > > > @@ -12,7 +12,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
> > > > >                 vdec_vpu_if.o \
> > > > >                 mtk_vcodec_dec.o \
> > > > >                 mtk_vcodec_dec_pm.o \
> > > > > -
> > > > > +               mtk_vcodec_fw.o
> > > > >
> > > > >  mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> > > > >                 venc/venc_h264_if.o \
> > > > > @@ -25,5 +25,3 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
> > > > >
> > > > >  mtk-vcodec-common-y := mtk_vcodec_intr.o \
> > > > >                 mtk_vcodec_util.o\
> > > > > -
> > > > > -ccflags-y += -I$(srctree)/drivers/media/platform/mtk-vpu
> > > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > > index 97a1b6664c20..4f07a5fcce7f 100644
> > > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > > @@ -20,7 +20,7 @@
> > > > >  #include "mtk_vcodec_dec_pm.h"
> > > > >  #include "mtk_vcodec_intr.h"
> > > > >  #include "mtk_vcodec_util.h"
> > > > > -#include "mtk_vpu.h"
> > > > > +#include "mtk_vcodec_fw.h"
> > > > >
> > > > >  #define VDEC_HW_ACTIVE 0x10
> > > > >  #define VDEC_IRQ_CFG   0x11
> > > > > @@ -77,22 +77,6 @@ static irqreturn_t mtk_vcodec_dec_irq_handler(int irq, void *priv)
> > > > >         return IRQ_HANDLED;
> > > > >  }
> > > > >
> > > > > -static void mtk_vcodec_dec_reset_handler(void *priv)
> > > > > -{
> > > > > -       struct mtk_vcodec_dev *dev = priv;
> > > > > -       struct mtk_vcodec_ctx *ctx;
> > > > > -
> > > > > -       mtk_v4l2_err("Watchdog timeout!!");
> > > > > -
> > > > > -       mutex_lock(&dev->dev_mutex);
> > > > > -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > > > -               ctx->state = MTK_STATE_ABORT;
> > > > > -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ERROR",
> > > > > -                               ctx->id);
> > > > > -       }
> > > > > -       mutex_unlock(&dev->dev_mutex);
> > > > > -}
> > > > > -
> > > > >  static int fops_vcodec_open(struct file *file)
> > > > >  {
> > > > >         struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > > > @@ -144,21 +128,20 @@ static int fops_vcodec_open(struct file *file)
> > > > >         if (v4l2_fh_is_singular(&ctx->fh)) {
> > > > >                 mtk_vcodec_dec_pw_on(&dev->pm);
> > > > >                 /*
> > > > > -                * vpu_load_firmware checks if it was loaded already and
> > > > > -                * does nothing in that case
> > > > > +                * Does nothing if firmware was already loaded.
> > > > >                  */
> > > > > -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > > > > +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
> > > > >                 if (ret < 0) {
> > > > >                         /*
> > > > >                          * Return 0 if downloading firmware successfully,
> > > > >                          * otherwise it is failed
> > > > >                          */
> > > > > -                       mtk_v4l2_err("vpu_load_firmware failed!");
> > > > > +                       mtk_v4l2_err("failed to load firmware!");
> > > > >                         goto err_load_fw;
> > > > >                 }
> > > > >
> > > > >                 dev->dec_capability =
> > > > > -                       vpu_get_vdec_hw_capa(dev->vpu_plat_dev);
> > > > > +                       mtk_vcodec_fw_get_vdec_capa(dev->fw_handler);
> > > > >                 mtk_v4l2_debug(0, "decoder capability %x", dev->dec_capability);
> > > > >         }
> > > > >
> > > > > @@ -228,6 +211,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > > >         struct mtk_vcodec_dev *dev;
> > > > >         struct video_device *vfd_dec;
> > > > >         struct resource *res;
> > > > > +       phandle rproc_phandle;
> > > > > +       enum mtk_vcodec_fw_type fw_type;
> > > > >         int i, ret;
> > > > >
> > > > >         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > > > > @@ -237,19 +222,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > > >         INIT_LIST_HEAD(&dev->ctx_list);
> > > > >         dev->plat_dev = pdev;
> > > > >
> > > > > -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > > > > -       if (dev->vpu_plat_dev == NULL) {
> > > > > -               mtk_v4l2_err("[VPU] vpu device in not ready");
> > > > > -               return -EPROBE_DEFER;
> > > > > +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > > > > +                                 &rproc_phandle)) {
> > > > > +               fw_type = VPU;
> > > > > +       } else {
> > > > > +               mtk_v4l2_err("Could not get vdec IPI device");
> > > > > +               return -ENODEV;
> > > > >         }
> > > > > -
> > > > > -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_dec_reset_handler,
> > > > > -                       dev, VPU_RST_DEC);
> > > > > +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_DEC);
> > > > > +       if (IS_ERR(dev->fw_handler))
> > > > > +               return PTR_ERR(dev->fw_handler);
> > > > >
> > > > >         ret = mtk_vcodec_init_dec_pm(dev);
> > > > >         if (ret < 0) {
> > > > >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source");
> > > > > -               return ret;
> > > > > +               goto err_dec_pm;
> > > > >         }
> > > > >
> > > > >         for (i = 0; i < NUM_MAX_VDEC_REG_BASE; i++) {
> > > > > @@ -352,6 +339,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > > >  err_res:
> > > > >         mtk_vcodec_release_dec_pm(dev);
> > > > > +err_dec_pm:
> > > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > > >         return ret;
> > > > >  }
> > > > >
> > > > > @@ -376,6 +365,7 @@ static int mtk_vcodec_dec_remove(struct platform_device *pdev)
> > > > >
> > > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > > >         mtk_vcodec_release_dec_pm(dev);
> > > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > > >         return 0;
> > > > >  }
> > > > >
> > > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > > > index 5a6ec8fb52da..36dfe3fc056a 100644
> > > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_pm.c
> > > > > @@ -12,7 +12,6 @@
> > > > >
> > > > >  #include "mtk_vcodec_dec_pm.h"
> > > > >  #include "mtk_vcodec_util.h"
> > > > > -#include "mtk_vpu.h"
> > > > >
> > > > >  int mtk_vcodec_init_dec_pm(struct mtk_vcodec_dev *mtkdev)
> > > > >  {
> > > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > > index 9fd56dee7fd1..e132c4ec463a 100644
> > > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > > @@ -309,13 +309,13 @@ struct mtk_vcodec_ctx {
> > > > >   * @m2m_dev_dec: m2m device for decoder
> > > > >   * @m2m_dev_enc: m2m device for encoder.
> > > > >   * @plat_dev: platform device
> > > > > - * @vpu_plat_dev: mtk vpu platform device
> > > > >   * @ctx_list: list of struct mtk_vcodec_ctx
> > > > >   * @irqlock: protect data access by irq handler and work thread
> > > > >   * @curr_ctx: The context that is waiting for codec hardware
> > > > >   *
> > > > >   * @reg_base: Mapped address of MTK Vcodec registers.
> > > > >   *
> > > > > + * @fw_handler: used to communicate with the firmware.
> > > > >   * @id_counter: used to identify current opened instance
> > > > >   *
> > > > >   * @encode_workqueue: encode work queue
> > > > > @@ -344,12 +344,13 @@ struct mtk_vcodec_dev {
> > > > >         struct v4l2_m2m_dev *m2m_dev_dec;
> > > > >         struct v4l2_m2m_dev *m2m_dev_enc;
> > > > >         struct platform_device *plat_dev;
> > > > > -       struct platform_device *vpu_plat_dev;
> > > > >         struct list_head ctx_list;
> > > > >         spinlock_t irqlock;
> > > > >         struct mtk_vcodec_ctx *curr_ctx;
> > > > >         void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
> > > > >
> > > > > +       struct mtk_vcodec_fw *fw_handler;
> > > > > +
> > > > >         unsigned long id_counter;
> > > > >
> > > > >         struct workqueue_struct *decode_workqueue;
> > > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > > index 4d31f1ed113f..4340ea10afd0 100644
> > > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > > @@ -21,7 +21,7 @@
> > > > >  #include "mtk_vcodec_enc_pm.h"
> > > > >  #include "mtk_vcodec_intr.h"
> > > > >  #include "mtk_vcodec_util.h"
> > > > > -#include "mtk_vpu.h"
> > > > > +#include "mtk_vcodec_fw.h"
> > > > >
> > > > >  module_param(mtk_v4l2_dbg_level, int, S_IRUGO | S_IWUSR);
> > > > >  module_param(mtk_vcodec_dbg, bool, S_IRUGO | S_IWUSR);
> > > > > @@ -101,22 +101,6 @@ static irqreturn_t mtk_vcodec_enc_lt_irq_handler(int irq, void *priv)
> > > > >         return IRQ_HANDLED;
> > > > >  }
> > > > >
> > > > > -static void mtk_vcodec_enc_reset_handler(void *priv)
> > > > > -{
> > > > > -       struct mtk_vcodec_dev *dev = priv;
> > > > > -       struct mtk_vcodec_ctx *ctx;
> > > > > -
> > > > > -       mtk_v4l2_debug(0, "Watchdog timeout!!");
> > > > > -
> > > > > -       mutex_lock(&dev->dev_mutex);
> > > > > -       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > > > -               ctx->state = MTK_STATE_ABORT;
> > > > > -               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> > > > > -                               ctx->id);
> > > > > -       }
> > > > > -       mutex_unlock(&dev->dev_mutex);
> > > > > -}
> > > > > -
> > > > >  static int fops_vcodec_open(struct file *file)
> > > > >  {
> > > > >         struct mtk_vcodec_dev *dev = video_drvdata(file);
> > > > > @@ -159,10 +143,10 @@ static int fops_vcodec_open(struct file *file)
> > > > >
> > > > >         if (v4l2_fh_is_singular(&ctx->fh)) {
> > > > >                 /*
> > > > > -                * vpu_load_firmware checks if it was loaded already and
> > > > > +                * load fireware to checks if it was loaded already and
> > > > >                  * does nothing in that case
> > > > >                  */
> > > > > -               ret = vpu_load_firmware(dev->vpu_plat_dev);
> > > > > +               ret = mtk_vcodec_fw_load_firmware(dev->fw_handler);
> > > > >                 if (ret < 0) {
> > > > >                         /*
> > > > >                          * Return 0 if downloading firmware successfully,
> > > > > @@ -173,7 +157,7 @@ static int fops_vcodec_open(struct file *file)
> > > > >                 }
> > > > >
> > > > >                 dev->enc_capability =
> > > > > -                       vpu_get_venc_hw_capa(dev->vpu_plat_dev);
> > > > > +                       mtk_vcodec_fw_get_venc_capa(dev->fw_handler);
> > > > >                 mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
> > > > >         }
> > > > >
> > > > > @@ -235,6 +219,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > > >         struct mtk_vcodec_dev *dev;
> > > > >         struct video_device *vfd_enc;
> > > > >         struct resource *res;
> > > > > +       phandle rproc_phandle;
> > > > > +       enum mtk_vcodec_fw_type fw_type;
> > > > >         int i, j, ret;
> > > > >
> > > > >         dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > > > > @@ -244,19 +230,21 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > > >         INIT_LIST_HEAD(&dev->ctx_list);
> > > > >         dev->plat_dev = pdev;
> > > > >
> > > > > -       dev->vpu_plat_dev = vpu_get_plat_device(dev->plat_dev);
> > > > > -       if (dev->vpu_plat_dev == NULL) {
> > > > > -               mtk_v4l2_err("[VPU] vpu device in not ready");
> > > > > -               return -EPROBE_DEFER;
> > > > > +       if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > > > > +                                 &rproc_phandle)) {
> > > > > +               fw_type = VPU;
> > > > > +       } else {
> > > > > +               mtk_v4l2_err("Could not get venc IPI device");
> > > > > +               return -ENODEV;
> > > > >         }
> > > > > -
> > > > > -       vpu_wdt_reg_handler(dev->vpu_plat_dev, mtk_vcodec_enc_reset_handler,
> > > > > -                               dev, VPU_RST_ENC);
> > > > > +       dev->fw_handler = mtk_vcodec_fw_select(dev, fw_type, VPU_RST_ENC);
> > > > > +       if (IS_ERR(dev->fw_handler))
> > > > > +               return PTR_ERR(dev->fw_handler);
> > > > >
> > > > >         ret = mtk_vcodec_init_enc_pm(dev);
> > > > >         if (ret < 0) {
> > > > >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> > > > > -               return ret;
> > > > > +               goto err_enc_pm;
> > > > >         }
> > > > >
> > > > >         for (i = VENC_SYS, j = 0; i < NUM_MAX_VCODEC_REG_BASE; i++, j++) {
> > > > > @@ -377,6 +365,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > > >  err_res:
> > > > >         mtk_vcodec_release_enc_pm(dev);
> > > > > +err_enc_pm:
> > > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > > >         return ret;
> > > > >  }
> > > > >
> > > > > @@ -401,6 +391,7 @@ static int mtk_vcodec_enc_remove(struct platform_device *pdev)
> > > > >
> > > > >         v4l2_device_unregister(&dev->v4l2_dev);
> > > > >         mtk_vcodec_release_enc_pm(dev);
> > > > > +       mtk_vcodec_fw_release(dev->fw_handler);
> > > > >         return 0;
> > > > >  }
> > > > >
> > > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > > > index 3e2bfded79a6..ee22902aaa71 100644
> > > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c
> > > > > @@ -12,8 +12,6 @@
> > > > >
> > > > >  #include "mtk_vcodec_enc_pm.h"
> > > > >  #include "mtk_vcodec_util.h"
> > > > > -#include "mtk_vpu.h"
> > > > > -
> > > > >
> > > > >  int mtk_vcodec_init_enc_pm(struct mtk_vcodec_dev *mtkdev)
> > > > >  {
> > > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > > > new file mode 100644
> > > > > index 000000000000..967bb100a990
> > > > > --- /dev/null
> > > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > > > @@ -0,0 +1,172 @@
> > > > > +// SPDX-License-Identifier: GPL-2.0
> > > > > +
> > > > > +#include "mtk_vcodec_fw.h"
> > > > > +#include "mtk_vcodec_util.h"
> > > > > +#include "mtk_vcodec_drv.h"
> > > > > +
> > > > > +struct mtk_vcodec_fw_ops {
> > > > > +       int (*load_firmware)(struct mtk_vcodec_fw *fw);
> > > > > +       unsigned int (*get_vdec_capa)(struct mtk_vcodec_fw *fw);
> > > > > +       unsigned int (*get_venc_capa)(struct mtk_vcodec_fw *fw);
> > > > > +       void * (*map_dm_addr)(struct mtk_vcodec_fw *fw, u32 dtcm_dmem_addr);
> > > > > +       int (*ipi_register)(struct mtk_vcodec_fw *fw, int id,
> > > > > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv);
> > > > > +       int (*ipi_send)(struct mtk_vcodec_fw *fw, int id, void *buf,
> > > > > +               unsigned int len, unsigned int wait);
> > > > > +};
> > > > > +
> > > > > +struct mtk_vcodec_fw {
> > > > > +       enum mtk_vcodec_fw_type type;
> > > > > +       const struct mtk_vcodec_fw_ops *ops;
> > > > > +       struct platform_device *pdev;
> > > > > +};
> > > > > +
> > > > > +static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
> > > > > +{
> > > > > +       return vpu_load_firmware(fw->pdev);
> > > > > +}
> > > > > +
> > > > > +static unsigned int mtk_vcodec_vpu_get_vdec_capa(struct mtk_vcodec_fw *fw)
> > > > > +{
> > > > > +       return vpu_get_vdec_hw_capa(fw->pdev);
> > > > > +}
> > > > > +
> > > > > +static unsigned int mtk_vcodec_vpu_get_venc_capa(struct mtk_vcodec_fw *fw)
> > > > > +{
> > > > > +       return vpu_get_venc_hw_capa(fw->pdev);
> > > > > +}
> > > > > +
> > > > > +static void *mtk_vcodec_vpu_map_dm_addr(struct mtk_vcodec_fw *fw,
> > > > > +                                       u32 dtcm_dmem_addr)
> > > > > +{
> > > > > +       return vpu_mapping_dm_addr(fw->pdev, dtcm_dmem_addr);
> > > > > +}
> > > > > +
> > > > > +static int mtk_vcodec_vpu_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
> > > > > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv)
> > > > > +{
> > > > > +       /*
> > > > > +        * The handler we receive takes a void * as its first argument. We
> > > > > +        * cannot change this because it needs to be passed down to the rproc
> > > > > +        * subsystem when SCP is used. VPU takes a const argument, which is
> > > > > +        * more constrained, so the conversion below is safe.
> > > > > +        */
> > > > > +       ipi_handler_t handler_const = (ipi_handler_t)handler;
> > > > > +
> > > > > +       return vpu_ipi_register(fw->pdev, id, handler_const, name, priv);
> > > > > +}
> > > > > +
> > > > > +static int mtk_vcodec_vpu_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
> > > > > +               unsigned int len, unsigned int wait)
> > > > > +{
> > > > > +       return vpu_ipi_send(fw->pdev, id, buf, len);
> > > > > +}
> > > > > +
> > > > > +static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
> > > > > +       .load_firmware = mtk_vcodec_vpu_load_firmware,
> > > > > +       .get_vdec_capa = mtk_vcodec_vpu_get_vdec_capa,
> > > > > +       .get_venc_capa = mtk_vcodec_vpu_get_venc_capa,
> > > > > +       .map_dm_addr = mtk_vcodec_vpu_map_dm_addr,
> > > > > +       .ipi_register = mtk_vcodec_vpu_set_ipi_register,
> > > > > +       .ipi_send = mtk_vcodec_vpu_ipi_send,
> > > > > +};
> > > > > +
> > > > > +static void mtk_vcodec_reset_handler(void *priv)
> > > > > +{
> > > > > +       struct mtk_vcodec_dev *dev = priv;
> > > > > +       struct mtk_vcodec_ctx *ctx;
> > > > > +
> > > > > +       mtk_v4l2_err("Watchdog timeout!!");
> > > > > +
> > > > > +       mutex_lock(&dev->dev_mutex);
> > > > > +       list_for_each_entry(ctx, &dev->ctx_list, list) {
> > > > > +               ctx->state = MTK_STATE_ABORT;
> > > > > +               mtk_v4l2_debug(0, "[%d] Change to state MTK_STATE_ABORT",
> > > > > +                               ctx->id);
> > > > > +       }
> > > > > +       mutex_unlock(&dev->dev_mutex);
> > > > > +}
> > > > > +
> > > > > +struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> > > > > +                                          enum mtk_vcodec_fw_type type,
> > > > > +                                          enum rst_id rst_id)
> > > > > +{
> > > > > +       const struct mtk_vcodec_fw_ops *ops;
> > > > > +       struct mtk_vcodec_fw *fw;
> > > > > +       struct platform_device *fw_pdev = NULL;
> > > > > +
> > > > > +       switch (type) {
> > > > > +       case VPU:
> > > > > +               ops = &mtk_vcodec_vpu_msg;
> > > > > +               fw_pdev = vpu_get_plat_device(dev->plat_dev);
> > > > > +               if (!fw_pdev) {
> > > > > +                       mtk_v4l2_err("firmware device is not ready");
> > > > > +                       return ERR_PTR(-EINVAL);
> > > > > +               }
> > > > > +               vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
> > > > > +                                   dev, rst_id);
> > > > > +               break;
> > > > > +       default:
> > > > > +               mtk_v4l2_err("invalid vcodec fw type");
> > > > > +               return ERR_PTR(-EINVAL);
> > > > > +       }
> > > > > +
> > > > > +       fw = devm_kzalloc(&dev->plat_dev->dev, sizeof(*fw), GFP_KERNEL);
> > > > > +       if (!fw)
> > > > > +               return ERR_PTR(-EINVAL);
> > > > > +
> > > > > +       fw->type = type;
> > > > > +       fw->ops = ops;
> > > > > +       fw->pdev = fw_pdev;
> > > > > +
> > > > > +       return fw;
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_select);
> > > > > +
> > > > > +void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
> > > > > +{
> > > > > +       switch (fw->type) {
> > > > > +       case VPU:
> > > > > +               put_device(&fw->pdev->dev);
> > > > > +               break;
> > > > > +       }
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(mtk_vcodec_fw_release);
> > > > > +
> > > >
> > > > What are these symbols exported for?
> > >
> > > This driver is made of three modules: mtk_vcodec_common,
> > > mtk_vcodec_dec and mtk_vcodec_enc. These functions are used by both
> > > the encoder and decoder module, so they need to be exported.
> > >
> > > There is mistake in the patch though that I noticed while
> > > investigating your comment: the file defining these functions
> > > (mtk_vcodec_fw.c) is linked against the mtk_vcodec_dec module while it
> > > should be part of mtk_vcodec_common, so thanks for asking. :)
> >
> > Oh, I missed that. Out of curiosity, what's the design decision motivating
> > this split?
> >
> > FWIW, I've had a somewhat painful experience with splitted
> > modules, so I try to minimize that if possible.
> >
> > If you decide to keep the split, how about you consider
> > also splitting the user option. Something around two visible
> > ones: VIDEO_MEDIATEK_VCODEC_ENC/DEC
> > and one hidden for the _COMMON one.
> >
> > That would make the design more obvious, I think.
> >
> > Minor nitpick, currently the Kconfig help states:
> >
> > """
> >             To compile this driver as a module, choose M here: the
> >             module will be called mtk-vcodec
> > """
> >
>
> Ah, one more thing.
>
> In order to build this with COMPILE_TEST, I believe you need
> something along these lines:
>
> diff --git a/drivers/remoteproc/Kconfig b/drivers/remoteproc/Kconfig
> index c4d1731295eb..ee83f407b9a3 100644
> --- a/drivers/remoteproc/Kconfig
> +++ b/drivers/remoteproc/Kconfig
> @@ -34,7 +34,7 @@ config INGENIC_VPU_RPROC
>
>  config MTK_SCP
>         tristate "Mediatek SCP support"
> -       depends on ARCH_MEDIATEK
> +       depends on ARCH_MEDIATEK || COMPILE_TEST
>         select RPMSG_MTK_SCP
>         help
>           Say y here to support Mediatek's System Companion Processor (SCP) via

Sure, but the file you pointed at is for the SCP which is untouched by
this patch. The VCODEC entry has COMPILE_TEST enabled.

You are right nonetheless, so I'll submit another patch for that.

>
> Thanks,
> Ezequiel

Re: [PATCH v3 02/16] dt-bindings: media: mtk-vcodec: document SCP node

[Alexandre Courbot]

On Mon, Jul 13, 2020 at 3:20 PM Chen-Yu Tsai <wens@kernel.org> wrote:
>
> On Mon, Jul 13, 2020 at 2:09 PM Alexandre Courbot <acourbot@chromium.org> wrote:
> >
> > The mediatek codecs can use either the VPU or the SCP as their interface
> > to firmware. Reflect this in the DT bindings.
> >
> > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > ---
> >  Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > index b6b5dde6abd8..7aef0a4fe207 100644
> > --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt
> > @@ -19,7 +19,9 @@ Required properties:
> >  - iommus : should point to the respective IOMMU block with master port as
> >    argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.txt
> >    for details.
> > -- mediatek,vpu : the node of video processor unit
> > +One of the two following nodes:
> > +- mediatek,vpu : the node of the video processor unit, if using VPU.
> > +- mediatek,scp : the noode of the SCP unit, if using SCP.
>
>                          ^ typo / extra o

Fixed, thanks!

Re: [PATCH v3 03/16] media: mtk-vcodec: add SCP firmware ops

[Alexandre Courbot]

On Mon, Jul 27, 2020 at 11:09 PM Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> On Mon, 27 Jul 2020 at 06:06, Alexandre Courbot <acourbot@chromium.org> wrote:
> >
> > On Thu, Jul 23, 2020 at 6:40 AM Ezequiel Garcia
> > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > On Mon, 13 Jul 2020 at 03:09, Alexandre Courbot <acourbot@chromium.org> wrote:
> > > >
> > > > From: Yunfei Dong <yunfei.dong@mediatek.com>
> > > >
> > > > Add support for communicating with the SCP firmware, which will be used
> > > > by MT8183.
> > > >
> > > > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > > > [acourbot: refactor, cleanup and split]
> > > > Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> > > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > > > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > > ---
> > > >  drivers/media/platform/Kconfig                |  1 +
> > > >  .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  3 +
> > > >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  3 +
> > > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.c | 56 +++++++++++++++++++
> > > >  .../media/platform/mtk-vcodec/mtk_vcodec_fw.h |  2 +
> > > >  5 files changed, 65 insertions(+)
> > > >
> > > > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > > > index c57ee78fa99d..f0dbe048efea 100644
> > > > --- a/drivers/media/platform/Kconfig
> > > > +++ b/drivers/media/platform/Kconfig
> > > > @@ -256,6 +256,7 @@ config VIDEO_MEDIATEK_VCODEC
> > > >         select VIDEOBUF2_DMA_CONTIG
> > > >         select V4L2_MEM2MEM_DEV
> > > >         select VIDEO_MEDIATEK_VPU
> > > > +       select MTK_SCP
> > > >         help
> > > >             Mediatek video codec driver provides HW capability to
> > > >             encode and decode in a range of video formats
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > index 4f07a5fcce7f..5b5765b98e57 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_drv.c
> > > > @@ -225,6 +225,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > > >                                   &rproc_phandle)) {
> > > >                 fw_type = VPU;
> > > > +       } else if (!of_property_read_u32(pdev->dev.of_node, "mediatek,scp",
> > > > +                                        &rproc_phandle)) {
> > > > +               fw_type = SCP;
> > > >         } else {
> > > >                 mtk_v4l2_err("Could not get vdec IPI device");
> > > >                 return -ENODEV;
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > index 4340ea10afd0..42530cd01a30 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > @@ -233,6 +233,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         if (!of_property_read_u32(pdev->dev.of_node, "mediatek,vpu",
> > > >                                   &rproc_phandle)) {
> > > >                 fw_type = VPU;
> > > > +       } else if (!of_property_read_u32(pdev->dev.of_node, "mediatek,scp",
> > > > +                                        &rproc_phandle)) {
> > > > +               fw_type = SCP;
> > > >         } else {
> > > >                 mtk_v4l2_err("Could not get venc IPI device");
> > > >                 return -ENODEV;
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > > index 967bb100a990..f2a62ea62fc6 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_fw.c
> > > > @@ -19,6 +19,7 @@ struct mtk_vcodec_fw {
> > > >         enum mtk_vcodec_fw_type type;
> > > >         const struct mtk_vcodec_fw_ops *ops;
> > > >         struct platform_device *pdev;
> > > > +       struct mtk_scp *scp;
> > > >  };
> > > >
> > > >  static int mtk_vcodec_vpu_load_firmware(struct mtk_vcodec_fw *fw)
> > > > @@ -71,6 +72,48 @@ static const struct mtk_vcodec_fw_ops mtk_vcodec_vpu_msg = {
> > > >         .ipi_send = mtk_vcodec_vpu_ipi_send,
> > > >  };
> > > >
> > > > +static int mtk_vcodec_scp_load_firmware(struct mtk_vcodec_fw *fw)
> > > > +{
> > > > +       return rproc_boot(scp_get_rproc(fw->scp));
> > > > +}
> > > > +
> > > > +static unsigned int mtk_vcodec_scp_get_vdec_capa(struct mtk_vcodec_fw *fw)
> > > > +{
> > > > +       return scp_get_vdec_hw_capa(fw->scp);
> > > > +}
> > > > +
> > > > +static unsigned int mtk_vcodec_scp_get_venc_capa(struct mtk_vcodec_fw *fw)
> > > > +{
> > > > +       return scp_get_venc_hw_capa(fw->scp);
> > > > +}
> > > > +
> > > > +static void *mtk_vcodec_vpu_scp_dm_addr(struct mtk_vcodec_fw *fw,
> > > > +                                       u32 dtcm_dmem_addr)
> > > > +{
> > > > +       return scp_mapping_dm_addr(fw->scp, dtcm_dmem_addr);
> > > > +}
> > > > +
> > > > +static int mtk_vcodec_scp_set_ipi_register(struct mtk_vcodec_fw *fw, int id,
> > > > +               mtk_vcodec_ipi_handler handler, const char *name, void *priv)
> > > > +{
> > > > +       return scp_ipi_register(fw->scp, id, handler, priv);
> > > > +}
> > > > +
> > > > +static int mtk_vcodec_scp_ipi_send(struct mtk_vcodec_fw *fw, int id, void *buf,
> > > > +               unsigned int len, unsigned int wait)
> > > > +{
> > > > +       return scp_ipi_send(fw->scp, id, buf, len, wait);
> > > > +}
> > > > +
> > > > +static const struct mtk_vcodec_fw_ops mtk_vcodec_rproc_msg = {
> > > > +       .load_firmware = mtk_vcodec_scp_load_firmware,
> > > > +       .get_vdec_capa = mtk_vcodec_scp_get_vdec_capa,
> > > > +       .get_venc_capa = mtk_vcodec_scp_get_venc_capa,
> > > > +       .map_dm_addr = mtk_vcodec_vpu_scp_dm_addr,
> > > > +       .ipi_register = mtk_vcodec_scp_set_ipi_register,
> > > > +       .ipi_send = mtk_vcodec_scp_ipi_send,
> > > > +};
> > > > +
> > > >  static void mtk_vcodec_reset_handler(void *priv)
> > > >  {
> > > >         struct mtk_vcodec_dev *dev = priv;
> > > > @@ -94,6 +137,7 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> > > >         const struct mtk_vcodec_fw_ops *ops;
> > > >         struct mtk_vcodec_fw *fw;
> > > >         struct platform_device *fw_pdev = NULL;
> > > > +       struct mtk_scp *scp = NULL;
> > > >
> > > >         switch (type) {
> > > >         case VPU:
> > > > @@ -106,6 +150,14 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> > > >                 vpu_wdt_reg_handler(fw_pdev, mtk_vcodec_reset_handler,
> > > >                                     dev, rst_id);
> > > >                 break;
> > > > +       case SCP:
> > > > +               ops = &mtk_vcodec_rproc_msg;
> > > > +               scp = scp_get(dev->plat_dev);
> > > > +               if (!scp) {
> > > > +                       mtk_v4l2_err("could not get vdec scp handle");
> > > > +                       return ERR_PTR(-EPROBE_DEFER);
> > >
> > > I suspect the EPROBE_DEFER should be returned by scp_get
> > > itself instead.
> >
> > scp_get() is a function of of mtk_scp remoteproc driver, so even if we
> > decide this is desirable (which I am not convinced, as the current
> > code leaves the freedom to decide how the absence of SCP should be
> > interpreted to the driver) this is beyond the scope of this series.
> >
>
> How can the consumer decide if it should defer probing or not?

Fair point. Looking at the code of scp_get(), the only failure points
are of_parse_phandle() and of_find_device_by_node(), which both return
NULL upon failure. So I cannot think of a way that could make
scp_get() decide to return EPROBE_DEFER vs. another error code, which
is probably why it just returns NULL for the moment.

Now that means that the code above also has no reason to make that
decision, and EPROBE_DEFER is probably not valid for all cases. For
now I don't see how we could improve this however.


>
> > >
> > > > +               }
> > > > +               break;
> > > >         default:
> > > >                 mtk_v4l2_err("invalid vcodec fw type");
> > > >                 return ERR_PTR(-EINVAL);
> > > > @@ -118,6 +170,7 @@ struct mtk_vcodec_fw *mtk_vcodec_fw_select(struct mtk_vcodec_dev *dev,
> > > >         fw->type = type;
> > > >         fw->ops = ops;
> > > >         fw->pdev = fw_pdev;
> > > > +       fw->scp = scp;
> > > >
> > > >         return fw;
> > > >  }
> > > > @@ -129,6 +182,9 @@ void mtk_vcodec_fw_release(struct mtk_vcodec_fw *fw)
> > > >         case VPU:
> > > >                 put_device(&fw->pdev->dev);
> > > >                 break;
> > > > +       case SCP:
> > > > +               scp_put(fw->scp);
> > >
> > > Interestingly scp_put is a wrapper around put_device :-)
> > > Perhaps not a reason to violate the layering.
> >
> > I don't see what is wrong with the current code? If SCP is in use, we
> > use SCP functions to manage it. If in the future SCP involves in such
> > a way that we need to do more than a put_device(), we are covered. Or
> > am I missing something?
>
> Oh, nothing wrong. I just found it interesting that scp_put was
> just wrapping put_device. Nothing to fix really.
>
> Thanks!
> Ezequiel

Re: [PATCH v3 04/16] media: mtk-vcodec: venc: support SCP firmware

[Alexandre Courbot]

On Mon, Jul 27, 2020 at 11:08 PM Ezequiel Garcia
<ezequiel@vanguardiasur.com.ar> wrote:
>
> .
>
> On Mon, 27 Jul 2020 at 06:06, Alexandre Courbot <acourbot@chromium.org> wrote:
> >
> > On Sat, Jul 25, 2020 at 6:13 AM Ezequiel Garcia
> > <ezequiel@vanguardiasur.com.ar> wrote:
> > >
> > > Hi Alexandre,
> > >
> > > I'm slowly making progress on the series. Here's some more comments.
> > >
> > > On Mon, 13 Jul 2020 at 03:10, Alexandre Courbot <acourbot@chromium.org> wrote:
> > > >
> > > > From: Yunfei Dong <yunfei.dong@mediatek.com>
> > > >
> > > > Support the new extended firmware used by MT8183's encoder.
> > > >
> > >
> > > If you could add some more information about the MT8183 encoder
> > > and the extended firmware interface, it would make the review
> > > easier.
> >
> > Sure, I'll try to expand that description a bit.
> >
> > >
> > > Some comments below, I have no idea how this hardware
> > > works, so maybe I won't make sense :-)
> > >
> > > > Signed-off-by: Yunfei Dong <yunfei.dong@mediatek.com>
> > > > [acourbot: refactor, cleanup and split]
> > > > Co-developed-by: Alexandre Courbot <acourbot@chromium.org>
> > > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > > > Acked-by: Tiffany Lin <tiffany.lin@mediatek.com>
> > > > ---
> > > >  .../platform/mtk-vcodec/mtk_vcodec_drv.h      | 12 +++
> > > >  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 34 ++++---
> > > >  .../platform/mtk-vcodec/venc/venc_h264_if.c   | 65 +++++++++++--
> > > >  .../platform/mtk-vcodec/venc/venc_vp8_if.c    |  3 +-
> > > >  .../media/platform/mtk-vcodec/venc_drv_if.h   |  6 ++
> > > >  .../media/platform/mtk-vcodec/venc_ipi_msg.h  | 15 ++-
> > > >  .../media/platform/mtk-vcodec/venc_vpu_if.c   | 97 +++++++++++++------
> > > >  .../media/platform/mtk-vcodec/venc_vpu_if.h   |  3 +-
> > > >  8 files changed, 181 insertions(+), 54 deletions(-)
> > > >
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > index e132c4ec463a..45c8adfc6a0c 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h
> > > > @@ -300,6 +300,17 @@ struct mtk_vcodec_ctx {
> > > >
> > > >  };
> > > >
> > > > +/**
> > > > + * struct mtk_vcodec_enc_pdata - compatible data for each IC
> > > > + *
> > > > + * @uses_ext: whether the encoder uses the extended firmware messaging format
> > > > + * @has_lt_irq: whether the encoder uses the LT irq
> > > > + */
> > > > +struct mtk_vcodec_enc_pdata {
> > > > +       bool uses_ext;
> > > > +       bool has_lt_irq;
> > >
> > > Instead of this boolean here, I think it might be cleaner
> > > to have a small array of strings, listing the interrupts
> > > each variant requires.
> > >
> > > See drivers/staging/media/hantro/rk3288_vpu_hw.c
> > >
> > > It's future-proof and even if you don't plan to see
> > > future works: a) we never know and b) cleaner code,
> > > easier to maintain.
> >
> > I tend to agree. Note however that this boolean is supposed to
> > disappear soon as there is another patch (not part of this series)
> > that will split the decoder into two nodes. The reason why we have
> > this boolean is because MT8173 actually controls *2* instances of this
> > encoder (one for H.264, one for VP8) that are actually separate
> > hardware.
> >
>
> Well, if you are planning to get rid of this boolean,
> then ignore the comment.
>
> > MT8183 only has one instance of the encoder (for H.264, that chip has
> > no VP8 support), so I have added this boolean to control this. Once
> > the DT node split happens it will go away.
>
> Will this DT change break compatibility?

It probably will, but the conclusion on the thread that discussed this
(sorry, can't find it) was that there is no other way. Basically this
DT update is triggered by a IOMMU change that is incompatible with the
current DT. This is not a convenience change for this driver.

>
> > In the meantime, maybe we
> > can try to handle this better by looking at the number of interrupts
> > provided by the DT and enabling/disabling that extra interrupt based
> > on that. It will be 2 for MT8173 and 1 for MT8183, so that should work
> > fine.
> >
> > >
> > > > +};
> > > > +
> > > >  /**
> > > >   * struct mtk_vcodec_dev - driver data
> > > >   * @v4l2_dev: V4L2 device to register video devices for.
> > > > @@ -348,6 +359,7 @@ struct mtk_vcodec_dev {
> > > >         spinlock_t irqlock;
> > > >         struct mtk_vcodec_ctx *curr_ctx;
> > > >         void __iomem *reg_base[NUM_MAX_VCODEC_REG_BASE];
> > > > +       const struct mtk_vcodec_enc_pdata *venc_pdata;
> > > >
> > > >         struct mtk_vcodec_fw *fw_handler;
> > > >
> > > > diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > index 42530cd01a30..922bc8883811 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c
> > > > @@ -244,6 +244,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         if (IS_ERR(dev->fw_handler))
> > > >                 return PTR_ERR(dev->fw_handler);
> > > >
> > > > +       dev->venc_pdata = of_device_get_match_data(&pdev->dev);
> > > >         ret = mtk_vcodec_init_enc_pm(dev);
> > > >         if (ret < 0) {
> > > >                 dev_err(&pdev->dev, "Failed to get mt vcodec clock source!");
> > > > @@ -278,21 +279,24 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >                 ret = -EINVAL;
> > > >                 goto err_res;
> > > >         }
> > > > +       disable_irq(dev->enc_irq);
> > > >
> > >
> > > I am aware this is not an issue with this patch:
> > > requesting an interrupt and then immediately
> > > disabling the interrupt looks racy.
> > >
> > > Now, I haven't need this pattern myself,
> > > but from a quick grep I think you want to do:
> > >
> > >         irq_set_status_flags(irq, IRQ_NOAUTOEN);
> > >         ret = devm_request_irq(dev, irq, ...
> > >
> > > Perhaps something to fix (probably in another patch,
> > > before this one).
> >
> > Nice suggestion, thanks. I will test and add a patch to this series.
> >
> > >
> > > > -       dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > > > -       ret = devm_request_irq(&pdev->dev,
> > > > -                              dev->enc_lt_irq, mtk_vcodec_enc_lt_irq_handler,
> > > > -                              0, pdev->name, dev);
> > > > -       if (ret) {
> > > > -               dev_err(&pdev->dev,
> > > > -                       "Failed to install dev->enc_lt_irq %d (%d)",
> > > > -                       dev->enc_lt_irq, ret);
> > > > -               ret = -EINVAL;
> > > > -               goto err_res;
> > > > +       if (dev->venc_pdata->has_lt_irq) {
> > > > +               dev->enc_lt_irq = platform_get_irq(pdev, 1);
> > > > +               ret = devm_request_irq(&pdev->dev,
> > > > +                                      dev->enc_lt_irq,
> > > > +                                      mtk_vcodec_enc_lt_irq_handler,
> > > > +                                      0, pdev->name, dev);
> > > > +               if (ret) {
> > > > +                       dev_err(&pdev->dev,
> > > > +                               "Failed to install dev->enc_lt_irq %d (%d)",
> > > > +                               dev->enc_lt_irq, ret);
> > > > +                       ret = -EINVAL;
> > > > +                       goto err_res;
> > > > +               }
> > > > +               disable_irq(dev->enc_lt_irq); /* VENC_LT */
> > > >         }
> > > >
> > > > -       disable_irq(dev->enc_irq);
> > > > -       disable_irq(dev->enc_lt_irq); /* VENC_LT */
> > > >         mutex_init(&dev->enc_mutex);
> > > >         mutex_init(&dev->dev_mutex);
> > > >         spin_lock_init(&dev->irqlock);
> > > > @@ -373,8 +377,12 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
> > > >         return ret;
> > > >  }
> > > >
> > > > +static const struct mtk_vcodec_enc_pdata mt8173_pdata = {
> > > > +       .has_lt_irq = true,
> > > > +};
> > > > +
> > > >  static const struct of_device_id mtk_vcodec_enc_match[] = {
> > > > -       {.compatible = "mediatek,mt8173-vcodec-enc",},
> > > > +       {.compatible = "mediatek,mt8173-vcodec-enc", .data = &mt8173_pdata},
> > > >         {},
> > > >  };
> > > >  MODULE_DEVICE_TABLE(of, mtk_vcodec_enc_match);
> > > > diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> > > > index 7a00f050ec36..050787b2896c 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> > > > @@ -24,6 +24,16 @@ static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc};
> > > >  #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker)
> > > >  #define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
> > > >
> > > > +/*
> > > > + * enum venc_h264_frame_type - h264 encoder output bitstream frame type
> > > > + */
> > > > +enum venc_h264_frame_type {
> > > > +       VENC_H264_IDR_FRM,
> > > > +       VENC_H264_I_FRM,
> > > > +       VENC_H264_P_FRM,
> > > > +       VENC_H264_B_FRM,
> > > > +};
> > > > +
> > > >  /*
> > > >   * enum venc_h264_vpu_work_buf - h264 encoder buffer index
> > > >   */
> > > > @@ -137,7 +147,8 @@ struct venc_h264_inst {
> > > >         struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX];
> > > >         struct mtk_vcodec_mem pps_buf;
> > > >         bool work_buf_allocated;
> > > > -       unsigned int frm_cnt;
> > > > +       u32 frm_cnt;    /* declared as u32 to properly overflow */
> > >
> > > This looks like it's fixing some issue not related to the 8183
> > > or the extended firmware. Perhaps you can add a
> > > more details comment?
> >
> > Mmm, let me check that a bit closer because I know MT8183 communicates
> > with the firmware a bit differently, so this may require these fields.
> > But maybe we can split that part into another patch.

Upon second look I think I can refrain from touching this member
altogether. By keeping the original logic we don't need the overflow
property and I think it's better that way. :)

> >
> > >
> > > > +       unsigned int skip_frm_cnt;
> > > >         unsigned int prepend_hdr;
> > > >         struct venc_vpu_inst vpu_inst;
> > > >         struct venc_h264_vsi *vsi;
> > > > @@ -327,6 +338,22 @@ static unsigned int h264_enc_wait_venc_done(struct venc_h264_inst *inst)
> > > >         return irq_status;
> > > >  }
> > > >
> > > > +static int h264_frame_type(struct venc_h264_inst *inst)
> > > > +{
> > > > +       if ((inst->vsi->config.gop_size != 0 &&
> > > > +               (inst->frm_cnt % inst->vsi->config.gop_size) == 0) ||
> > > > +               (inst->frm_cnt == 0 && inst->vsi->config.gop_size == 0)) {
> > > > +               /* IDR frame */
> > > > +               return VENC_H264_IDR_FRM;
> > > > +       } else if ((inst->vsi->config.intra_period != 0 &&
> > > > +               (inst->frm_cnt % inst->vsi->config.intra_period) == 0) ||
> > > > +               (inst->frm_cnt == 0 && inst->vsi->config.intra_period == 0)) {
> > > > +               /* I frame */
> > > > +               return VENC_H264_I_FRM;
> > > > +       } else {
> > > > +               return VENC_H264_P_FRM;  /* Note: B frames are not supported */
> > > > +       }
> > > > +}
> > > >  static int h264_encode_sps(struct venc_h264_inst *inst,
> > > >                            struct mtk_vcodec_mem *bs_buf,
> > > >                            unsigned int *bs_size)
> > > > @@ -337,7 +364,7 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
> > > >         mtk_vcodec_debug_enter(inst);
> > > >
> > > >         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL,
> > > > -                            bs_buf, bs_size);
> > > > +                            bs_buf, bs_size, NULL);
> > > >         if (ret)
> > > >                 return ret;
> > > >
> > > > @@ -364,7 +391,7 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
> > > >         mtk_vcodec_debug_enter(inst);
> > > >
> > > >         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL,
> > > > -                            bs_buf, bs_size);
> > > > +                            bs_buf, bs_size, NULL);
> > > >         if (ret)
> > > >                 return ret;
> > > >
> > > > @@ -410,13 +437,24 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
> > > >  {
> > > >         int ret = 0;
> > > >         unsigned int irq_status;
> > > > +       struct venc_frame_info frame_info;
> > > >
> > > >         mtk_vcodec_debug_enter(inst);
> > > > -
> > > > +       /* Overflowing back to 0 is ok and expected here */
> > > > +       inst->frm_cnt++;
> > > > +       mtk_vcodec_debug(inst, "frm_cnt++ = %d\n ", inst->frm_cnt);
> > > > +       frame_info.frm_cnt = inst->frm_cnt;
> > > > +       frame_info.skip_frm_cnt = inst->skip_frm_cnt;
> > > > +       frame_info.frm_type = h264_frame_type(inst);
> > >
> > > Ditto: Is this chunk related to the new support,
> > > or is this addressing some current issue?
> >
> > I will need to re-read this part to understand which one it is, but
> > will split the patch if it turns out to make sense.

frame_info is introduced by this patch, so I think this block belongs here.

> >
> > >
> > > > +       mtk_vcodec_debug(inst, "frm_cnt++ = %d,skip_frm_cnt =%d,frm_type=%d.\n",
> > > > +               frame_info.frm_cnt, frame_info.skip_frm_cnt,
> > > > +               frame_info.frm_type);
> > > >         ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf,
> > > > -                            bs_buf, bs_size);
> > > > -       if (ret)
> > > > +                            bs_buf, bs_size, &frame_info);
> > > > +       if (ret) {
> > > > +               inst->frm_cnt--;
> > > >                 return ret;
> > > > +       }
> > > >
> > > >         /*
> > > >          * skip frame case: The skip frame buffer is composed by vpu side only,
> > > > @@ -427,19 +465,19 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
> > > >                 memcpy(bs_buf->va,
> > > >                        inst->work_bufs[VENC_H264_VPU_WORK_BUF_SKIP_FRAME].va,
> > > >                        *bs_size);
> > > > -               ++inst->frm_cnt;
> > > > +               ++inst->skip_frm_cnt;
> > > >                 return ret;
> > > >         }
> > > >
> > > >         irq_status = h264_enc_wait_venc_done(inst);
> > > >         if (irq_status != MTK_VENC_IRQ_STATUS_FRM) {
> > > >                 mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
> > > > +               inst->frm_cnt--;
> > > >                 return -EIO;
> > > >         }
> > > >
> > > >         *bs_size = h264_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT);
> > > >
> > > > -       ++inst->frm_cnt;
> > > >         mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-",
> > > >                          inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm);
> > > >
> > > > @@ -464,6 +502,7 @@ static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,
> > > >
> > > >  static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
> > > >  {
> > > > +       const bool is_ext = ctx->dev->venc_pdata->uses_ext;
> > > >         int ret = 0;
> > > >         struct venc_h264_inst *inst;
> > > >
> > > > @@ -473,8 +512,9 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
> > > >
> > > >         inst->ctx = ctx;
> > > >         inst->vpu_inst.ctx = ctx;
> > > > -       inst->vpu_inst.id = IPI_VENC_H264;
> > > > +       inst->vpu_inst.id = is_ext ? SCP_IPI_VENC_H264 : IPI_VENC_H264;
> > > >         inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
> > > > +       inst->frm_cnt = 0xffffffff;
> > > >
> > > >         mtk_vcodec_debug_enter(inst);
> > > >
> > > > @@ -629,7 +669,12 @@ static int h264_enc_set_param(void *handle,
> > > >                 inst->prepend_hdr = 1;
> > > >                 mtk_vcodec_debug(inst, "set prepend header mode");
> > > >                 break;
> > > > -
> > > > +       case VENC_SET_PARAM_FORCE_INTRA:
> > > > +       case VENC_SET_PARAM_GOP_SIZE:
> > > > +       case VENC_SET_PARAM_INTRA_PERIOD:
> > > > +               inst->frm_cnt = 0xffffffff;
> > > > +               inst->skip_frm_cnt = 0;
> > > > +               fallthrough;
> > > >         default:
> > > >                 ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm);
> > > >                 break;
> > > > diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> > > > index 6426af514526..11abb191ada5 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c
> > > > @@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
> > > >
> > > >         mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
> > > >
> > > > -       ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size);
> > > > +       ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size,
> > > > +                            NULL);
> > > >         if (ret)
> > > >                 return ret;
> > > >
> > > > diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > > > index 52fc9cc812fc..51b52625ca22 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > > > +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h
> > > > @@ -92,6 +92,12 @@ struct venc_enc_param {
> > > >         unsigned int gop_size;
> > > >  };
> > > >
> > >
> > > How about a comment here documenting this struct?
> >
> > Will comment if I can figure out what it is useful for. :)
> >
> > >
> > > > +struct venc_frame_info {
> > > > +       unsigned int frm_cnt;           /* per frame update */
> > > > +       unsigned int skip_frm_cnt;      /* per frame update */
> > >
> > > I'd go for s/cnt/count, it's just 2 bytes
> > > and in the long run it'll make our life easier.
> >
> > Ack.
> >
> > >
> > > > +       unsigned int frm_type;          /* per frame update */
> > > > +};
> > > > +
> > > >  /*
> > > >   * struct venc_frm_buf - frame buffer information used in venc_if_encode()
> > > >   * @fb_addr: plane frame buffer addresses
> > > > diff --git a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > > > index 28ee04ca6241..4cafbf92d9cd 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > > > +++ b/drivers/media/platform/mtk-vcodec/venc_ipi_msg.h
> > > > @@ -51,17 +51,22 @@ struct venc_ap_ipi_msg_init {
> > > >   * @vpu_inst_addr:     VPU encoder instance addr
> > > >   *                     (struct venc_vp8_vsi/venc_h264_vsi *)
> > > >   * @param_id:  parameter id (venc_set_param_type)
> > > > - * @data_item: number of items in the data array
> > > > + * @num_data:  number of items in the data array
> > > >   * @data[8]:   data array to store the set parameters
> > > >   */
> > > >  struct venc_ap_ipi_msg_set_param {
> > > >         uint32_t msg_id;
> > > >         uint32_t vpu_inst_addr;
> > > >         uint32_t param_id;
> > > > -       uint32_t data_item;
> > > > +       uint32_t num_data;
> > >
> > > This change looks either out of place, or unneeded.
> > > Or I'm missing something.
> >
> > Yeah, it's just renaming this member to something that makes more
> > sense, "data_item" bringing absolutely no information about what this
> > does. Will make an extra patch for it.
> >
> > >
> > > >         uint32_t data[8];
> > > >  };
> > > >
> > > > +struct venc_ap_ipi_msg_set_param_ext {
> > > > +       struct venc_ap_ipi_msg_set_param base;
> > > > +       uint32_t data_ext[24];
> > > > +};
> > > > +
> > > >  /**
> > > >   * struct venc_ap_ipi_msg_enc - AP to VPU enc cmd structure
> > > >   * @msg_id:    message id (AP_IPIMSG_XXX_ENC_ENCODE)
> > > > @@ -82,6 +87,12 @@ struct venc_ap_ipi_msg_enc {
> > > >         uint32_t bs_size;
> > > >  };
> > > >
> > >
> > > Ditto.
> >
> > Ditto what? :) There is no code change right here.
> >
>
> Oh, sorry. I meant: same feedback about adding a comment
> to the structure.
>
> > >
> > > > +struct venc_ap_ipi_msg_enc_ext {
> > > > +       struct venc_ap_ipi_msg_enc base;
> > > > +       uint32_t data_item;
> > > > +       uint32_t data[32];
> > > > +};
> > > > +
> > > >  /**
> > > >   * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure
> > > >   * @msg_id:    message id (AP_IPIMSG_XXX_ENC_DEINIT)
> > > > diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> > > > index 53854127814b..6c77bf025172 100644
> > > > --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> > > > +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> > > > @@ -116,49 +116,81 @@ int vpu_enc_init(struct venc_vpu_inst *vpu)
> > > >         return 0;
> > > >  }
> > > >
> > > > +static unsigned int venc_enc_param_crop_right(struct venc_vpu_inst *vpu,
> > > > +                                             struct venc_enc_param *enc_prm)
> > > > +{
> > > > +       unsigned int img_crop_right = enc_prm->buf_width - enc_prm->width;
> > > > +
> > > > +       return img_crop_right % 16;
> > > > +}
> > > > +
> > > > +static unsigned int venc_enc_param_crop_bottom(struct venc_enc_param *enc_prm)
> > > > +{
> > > > +       return round_up(enc_prm->height, 16) - enc_prm->height;
> > > > +}
> > > > +
> > > > +static unsigned int venc_enc_param_num_mb(struct venc_enc_param *enc_prm)
> > > > +{
> > > > +       return DIV_ROUND_UP(enc_prm->width, 16) *
> > > > +              DIV_ROUND_UP(enc_prm->height, 16);
> > >
> > > ^^^^
> > > You could define the macroblock size in a macro and
> > > use it in these various places.
> >
> > Ack.
> >
> > >
> > > > +}
> > > > +
> > > >  int vpu_enc_set_param(struct venc_vpu_inst *vpu,
> > > >                       enum venc_set_param_type id,
> > > >                       struct venc_enc_param *enc_param)
> > > >  {
> > > > -       struct venc_ap_ipi_msg_set_param out;
> > > > +       const bool is_ext = vpu->ctx->dev->venc_pdata->uses_ext;
> > > > +       size_t msg_size = is_ext ?
> > >
> > > How about you use vpu->id to check for is_ext?
> >
> > Unfortunately vpu->id can be assigned an ipi_id or scp_ipi_id, which
> > are two different namespaces and which values can thus collide. So we
> > cannot use ID for this.
> >
> > >
> > > I feel like querying the "is_ext" property like this
> > > is a layering violation.
> > >
> > > Or maybe better some firmware interface type
> > > should be set in struct venc_vpu_inst,
> > > so the venc_vpu_if driver is aware of its own type.
> >
> > "is_ext" refers to whether the firmware uses an extended version of
> > the protocol to communicate, so using it here seems to be in place.
>
> Hm, maybe my comment wasn't really clear. I was trying
> to point out that both venc_h264_if.c and venc_vpu_if.c
> are using ctx->dev->venc_pdata->uses_ext to get
> the "is_ext".
>
> I.e. I would imagine it's cleaner to:
>
> diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> index 050787b2896c..1ad25b59356a 100644
> --- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c
> @@ -513,6 +513,7 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx)
>         inst->ctx = ctx;
>         inst->vpu_inst.ctx = ctx;
>         inst->vpu_inst.id = is_ext ? SCP_IPI_VENC_H264 : IPI_VENC_H264;
> +       inst->vpu_inst.is_ext = is_ext;
>         inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS);
>         inst->frm_cnt = 0xffffffff;
>
> diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> index 472503701003..137453d66d8e 100644
> --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c
> @@ -160,7 +160,7 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
>                       enum venc_set_param_type id,
>                       struct venc_enc_param *enc_param)
>  {
> -       const bool is_ext = vpu->ctx->dev->venc_pdata->uses_ext;
> +       const bool is_ext = vpu->is_ext;
>         size_t msg_size = is_ext ?
>                 sizeof(struct venc_ap_ipi_msg_set_param_ext) :
>                 sizeof(struct venc_ap_ipi_msg_set_param);
> @@ -228,7 +228,7 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu,
> unsigned int bs_mode,
>                    unsigned int *bs_size,
>                    struct venc_frame_info *frame_info)
>  {
> -       const bool is_ext = vpu->ctx->dev->venc_pdata->uses_ext;
> +       const bool is_ext = vpu->is_ext;
>         size_t msg_size = is_ext ?
>                 sizeof(struct venc_ap_ipi_msg_enc_ext) :
>                 sizeof(struct venc_ap_ipi_msg_enc);
>
> IMHO, this makes the structure cleaner, but
> possibly this is just bikeshedding, so feel free to ignore.

Ah, I see now. Well it certainly makes the data more accessible, on
the other hand it also duplicates the same information in different
places. In this case this is not a problem since it doesn't change at
runtime, but as a matter of principle I prefer to avoid such
duplication, lest I acquire the habit of doing it in less safe
contexts. :) However if we want to improve legibility I think a macro
would be a good fit.