[RESEND PATCH v2 0/4] Add auto suspend/resume,YUV422 format,SPS/PPS generation for each IDR

[RESEND PATCH v2 0/4] Add auto suspend/resume,YUV422 format,SPS/PPS generation for each IDR

[jackson.lee]

From: "Jackson.lee" <jackson.lee@chipsnmedia.com>

The wave5 codec driver is a stateful encoder/decoder.
The following patches is for supporting yuv422 inpuy format, supporting
runtime suspend/resume feature and extra things.

v4l2-compliance results:
========================

v4l2-compliance 1.24.1, 64 bits, 64-bit time_t

Buffer ioctls:
            warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported
            warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not supported
    test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
    test VIDIOC_EXPBUF: OK
    test Requests: OK (Not Supported)

Total for wave5-dec device /dev/video0: 45, Succeeded: 45, Failed: 0, Warnings: 2
Total for wave5-enc device /dev/video1: 45, Succeeded: 45, Failed: 0, Warnings: 0

Fluster test results:
=====================

Running test suite JCT-VC-HEVC_V1 with decoder GStreamer-H.265-V4L2-Gst1.0 Using 1 parallel job(s)
Ran 132/147 tests successfully               in 94.782 secs

(1 test fails because of not supporting to parse multi frames, 1 test fails because of a missing frame and slight corruption, 2 tests fail because of sizes which are incompatible with the IP, 11 tests fail because of unsupported 10 bit format)

Running test suite JVT-AVC_V1 with decoder GStreamer-H.264-V4L2-Gst1.0 Using 1 parallel job(s)
Ran 77/135 tests successfully               in 37.364 secs

(58 fail because the hardware is unable to decode  MBAFF / FMO / Field / Extended profile streams.)


Chnage since v1:
=================

* For [PATCH v2 0/4] media: chips-media: wave5: Support SPS/PPS generation for each IDR
 - define a macro for register addresses

* For [PATCH v2 1/4] media: chips-media: wave5: Support runtime suspend/resume
 - add auto suspend/resume

* For [PATCH v2 2/4] media: chips-media: wave5: Use helpers to calculate bytesperline and sizeimage
 - use helper functions to calculate bytesperline and sizeimage

* For [PATCH v2 3/4] media: chips-media: wave5: Support YUV422 raw pixel-formats on the encoder
 - remove unnecessary codes

Change since v0:
=================
The DEFAULT_SRC_SIZE macro was defined using multiple lines,
To make a simple define, tab and multiple lines has been removed,
The macro is defined using one line.

Jackson.lee (4):
  media: chips-media: wave5: Support SPS/PPS generation for each IDR
  media: chips-media: wave5: Support runtime suspend/resume
  media: chips-media: wave5: Use helpers to calculate bytesperline and
    sizeimage.
  media: chips-media: wave5: Support YUV422 raw pixel-formats on the
    encoder.

 .../platform/chips-media/wave5/wave5-helper.c |  24 ++
 .../platform/chips-media/wave5/wave5-helper.h |   4 +
 .../platform/chips-media/wave5/wave5-hw.c     |  23 +-
 .../chips-media/wave5/wave5-vpu-dec.c         | 261 +++++-------------
 .../chips-media/wave5/wave5-vpu-enc.c         | 260 +++++++++--------
 .../platform/chips-media/wave5/wave5-vpu.c    |  43 +++
 .../platform/chips-media/wave5/wave5-vpu.h    |   4 -
 .../platform/chips-media/wave5/wave5-vpuapi.c |  14 +-
 .../platform/chips-media/wave5/wave5-vpuapi.h |   1 +
 .../chips-media/wave5/wave5-vpuconfig.h       |  25 +-
 .../media/platform/chips-media/wave5/wave5.h  |   3 +
 11 files changed, 329 insertions(+), 333 deletions(-)

-- 
2.43.0

[RESEND PATCH v2 1/4] media: chips-media: wave5: Support SPS/PPS generation for each IDR

[jackson.lee]

From: "Jackson.lee" <jackson.lee@chipsnmedia.com>

Provide a control to toggle (0 = off / 1 = on), whether the SPS and
PPS are generated for every IDR.

Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
 .../platform/chips-media/wave5/wave5-hw.c     | 19 +++++++++++++++----
 .../chips-media/wave5/wave5-vpu-enc.c         |  7 +++++++
 .../platform/chips-media/wave5/wave5-vpuapi.h |  1 +
 3 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/chips-media/wave5/wave5-hw.c b/drivers/media/platform/chips-media/wave5/wave5-hw.c
index f1e022fb148e..4a262822bf17 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-hw.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-hw.c
@@ -23,6 +23,15 @@
 #define FEATURE_AVC_ENCODER		BIT(1)
 #define FEATURE_HEVC_ENCODER		BIT(0)
 
+#define ENC_AVC_INTRA_IDR_PARAM_MASK	0x7ff
+#define ENC_AVC_INTRA_PERIOD		6
+#define ENC_AVC_IDR_PERIOD		17
+#define ENC_AVC_FORCED_IDR_HEADER	28
+
+#define ENC_HEVC_INTRA_QP		3
+#define ENC_HEVC_FORCED_IDR_HEADER	9
+#define ENC_HEVC_INTRA_PERIOD		16
+
 /* Decoder support fields */
 #define FEATURE_AVC_DECODER		BIT(3)
 #define FEATURE_HEVC_DECODER		BIT(2)
@@ -1601,12 +1610,14 @@ int wave5_vpu_enc_init_seq(struct vpu_instance *inst)
 
 	if (inst->std == W_AVC_ENC)
 		vpu_write_reg(inst->dev, W5_CMD_ENC_SEQ_INTRA_PARAM, p_param->intra_qp |
-				((p_param->intra_period & 0x7ff) << 6) |
-				((p_param->avc_idr_period & 0x7ff) << 17));
+				((p_param->intra_period & ENC_AVC_INTRA_IDR_PARAM_MASK) << ENC_AVC_INTRA_PERIOD) |
+				((p_param->avc_idr_period & ENC_AVC_INTRA_IDR_PARAM_MASK) << ENC_AVC_IDR_PERIOD) |
+				(p_param->forced_idr_header_enable << ENC_AVC_FORCED_IDR_HEADER));
 	else if (inst->std == W_HEVC_ENC)
 		vpu_write_reg(inst->dev, W5_CMD_ENC_SEQ_INTRA_PARAM,
-			      p_param->decoding_refresh_type | (p_param->intra_qp << 3) |
-				(p_param->intra_period << 16));
+			      p_param->decoding_refresh_type | (p_param->intra_qp << ENC_HEVC_INTRA_QP) |
+			      (p_param->forced_idr_header_enable << ENC_HEVC_FORCED_IDR_HEADER) |
+			      (p_param->intra_period << ENC_HEVC_INTRA_PERIOD));
 
 	reg_val = (p_param->rdo_skip << 2) |
 		(p_param->lambda_scaling_enable << 3) |
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
index f29cfa3af94a..f04baa93a9b7 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
@@ -1061,6 +1061,9 @@ static int wave5_vpu_enc_s_ctrl(struct v4l2_ctrl *ctrl)
 	case V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE:
 		inst->enc_param.entropy_coding_mode = ctrl->val;
 		break;
+	case V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR:
+		inst->enc_param.forced_idr_header_enable = ctrl->val;
+		break;
 	case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
 		break;
 	default:
@@ -1219,6 +1222,7 @@ static void wave5_set_enc_openparam(struct enc_open_param *open_param,
 		else
 			open_param->wave_param.intra_refresh_arg = num_ctu_row;
 	}
+	open_param->wave_param.forced_idr_header_enable = input.forced_idr_header_enable;
 }
 
 static int initialize_sequence(struct vpu_instance *inst)
@@ -1702,6 +1706,9 @@ static int wave5_vpu_open_enc(struct file *filp)
 			  0, 1, 1, 0);
 	v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave5_vpu_enc_ctrl_ops,
 			  V4L2_CID_MIN_BUFFERS_FOR_OUTPUT, 1, 32, 1, 1);
+	v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave5_vpu_enc_ctrl_ops,
+			  V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR,
+			  0, 1, 1, 0);
 
 	if (v4l2_ctrl_hdl->error) {
 		ret = -ENODEV;
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
index 352f6e904e50..3ad6118550ac 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
@@ -566,6 +566,7 @@ struct enc_wave_param {
 	u32 lambda_scaling_enable: 1; /* enable lambda scaling using custom GOP */
 	u32 transform8x8_enable: 1; /* enable 8x8 intra prediction and 8x8 transform */
 	u32 mb_level_rc_enable: 1; /* enable MB-level rate control */
+	u32 forced_idr_header_enable: 1; /* enable header encoding before IDR frame */
 };
 
 struct enc_open_param {
-- 
2.43.0

[RESEND PATCH v2 2/4] media: chips-media: wave5: Support runtime suspend/resume

[jackson.lee]

From: "Jackson.lee" <jackson.lee@chipsnmedia.com>

For saving a power resource, we support runtime suspend/resume for an encoder/decoder.
So our vpu module's power turns on only if an encoder/decoder is used.

Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
 .../platform/chips-media/wave5/wave5-hw.c     |  4 +-
 .../chips-media/wave5/wave5-vpu-dec.c         | 16 ++++++-
 .../chips-media/wave5/wave5-vpu-enc.c         | 15 +++++++
 .../platform/chips-media/wave5/wave5-vpu.c    | 43 +++++++++++++++++++
 .../platform/chips-media/wave5/wave5-vpuapi.c | 14 ++++--
 .../media/platform/chips-media/wave5/wave5.h  |  3 ++
 6 files changed, 88 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/chips-media/wave5/wave5-hw.c b/drivers/media/platform/chips-media/wave5/wave5-hw.c
index 4a262822bf17..826b92b7b582 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-hw.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-hw.c
@@ -1084,8 +1084,8 @@ int wave5_vpu_re_init(struct device *dev, u8 *fw, size_t size)
 	return setup_wave5_properties(dev);
 }
 
-static int wave5_vpu_sleep_wake(struct device *dev, bool i_sleep_wake, const uint16_t *code,
-				size_t size)
+int wave5_vpu_sleep_wake(struct device *dev, bool i_sleep_wake, const uint16_t *code,
+			 size_t size)
 {
 	u32 reg_val;
 	struct vpu_buf *common_vb;
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
index ef227af72348..a199877c643b 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
@@ -5,6 +5,7 @@
  * Copyright (C) 2021-2023 CHIPS&MEDIA INC
  */
 
+#include <linux/pm_runtime.h>
 #include "wave5-helper.h"
 
 #define VPU_DEC_DEV_NAME "C&M Wave5 VPU decoder"
@@ -518,6 +519,8 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst)
 	if (q_status.report_queue_count == 0 &&
 	    (q_status.instance_queue_count == 0 || dec_info.sequence_changed)) {
 		dev_dbg(inst->dev->dev, "%s: finishing job.\n", __func__);
+		pm_runtime_mark_last_busy(inst->dev->dev);
+		pm_runtime_put_autosuspend(inst->dev->dev);
 		v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
 	}
 }
@@ -1382,6 +1385,7 @@ static int wave5_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count
 	int ret = 0;
 
 	dev_dbg(inst->dev->dev, "%s: type: %u\n", __func__, q->type);
+	pm_runtime_resume_and_get(inst->dev->dev);
 
 	v4l2_m2m_update_start_streaming_state(m2m_ctx, q);
 
@@ -1425,13 +1429,15 @@ static int wave5_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count
 			}
 		}
 	}
-
+	pm_runtime_mark_last_busy(inst->dev->dev);
+	pm_runtime_put_autosuspend(inst->dev->dev);
 	return ret;
 
 free_bitstream_vbuf:
 	wave5_vdi_free_dma_memory(inst->dev, &inst->bitstream_vbuf);
 return_buffers:
 	wave5_return_bufs(q, VB2_BUF_STATE_QUEUED);
+	pm_runtime_put_autosuspend(inst->dev->dev);
 	return ret;
 }
 
@@ -1517,6 +1523,7 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)
 	bool check_cmd = TRUE;
 
 	dev_dbg(inst->dev->dev, "%s: type: %u\n", __func__, q->type);
+	pm_runtime_resume_and_get(inst->dev->dev);
 
 	while (check_cmd) {
 		struct queue_status_info q_status;
@@ -1540,6 +1547,9 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)
 		streamoff_output(q);
 	else
 		streamoff_capture(q);
+
+	pm_runtime_mark_last_busy(inst->dev->dev);
+	pm_runtime_put_autosuspend(inst->dev->dev);
 }
 
 static const struct vb2_ops wave5_vpu_dec_vb2_ops = {
@@ -1626,7 +1636,7 @@ static void wave5_vpu_dec_device_run(void *priv)
 	int ret = 0;
 
 	dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new bitstream data", __func__);
-
+	pm_runtime_resume_and_get(inst->dev->dev);
 	ret = fill_ringbuffer(inst);
 	if (ret) {
 		dev_warn(inst->dev->dev, "Filling ring buffer failed\n");
@@ -1709,6 +1719,8 @@ static void wave5_vpu_dec_device_run(void *priv)
 
 finish_job_and_return:
 	dev_dbg(inst->dev->dev, "%s: leave and finish job", __func__);
+	pm_runtime_mark_last_busy(inst->dev->dev);
+	pm_runtime_put_autosuspend(inst->dev->dev);
 	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
 }
 
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
index f04baa93a9b7..013e2bb37fbb 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
@@ -5,6 +5,7 @@
  * Copyright (C) 2021-2023 CHIPS&MEDIA INC
  */
 
+#include <linux/pm_runtime.h>
 #include "wave5-helper.h"
 
 #define VPU_ENC_DEV_NAME "C&M Wave5 VPU encoder"
@@ -1310,6 +1311,7 @@ static int wave5_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int count
 	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
 	int ret = 0;
 
+	pm_runtime_resume_and_get(inst->dev->dev);
 	v4l2_m2m_update_start_streaming_state(m2m_ctx, q);
 
 	if (inst->state == VPU_INST_STATE_NONE && q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
@@ -1364,9 +1366,13 @@ static int wave5_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int count
 	if (ret)
 		goto return_buffers;
 
+	pm_runtime_mark_last_busy(inst->dev->dev);
+	pm_runtime_put_autosuspend(inst->dev->dev);
 	return 0;
 return_buffers:
 	wave5_return_bufs(q, VB2_BUF_STATE_QUEUED);
+	pm_runtime_mark_last_busy(inst->dev->dev);
+	pm_runtime_put_autosuspend(inst->dev->dev);
 	return ret;
 }
 
@@ -1408,6 +1414,7 @@ static void wave5_vpu_enc_stop_streaming(struct vb2_queue *q)
 	 */
 
 	dev_dbg(inst->dev->dev, "%s: type: %u\n", __func__, q->type);
+	pm_runtime_resume_and_get(inst->dev->dev);
 
 	if (wave5_vpu_both_queues_are_streaming(inst))
 		switch_state(inst, VPU_INST_STATE_STOP);
@@ -1432,6 +1439,9 @@ static void wave5_vpu_enc_stop_streaming(struct vb2_queue *q)
 		streamoff_output(inst, q);
 	else
 		streamoff_capture(inst, q);
+
+	pm_runtime_mark_last_busy(inst->dev->dev);
+	pm_runtime_put_autosuspend(inst->dev->dev);
 }
 
 static const struct vb2_ops wave5_vpu_enc_vb2_ops = {
@@ -1478,6 +1488,7 @@ static void wave5_vpu_enc_device_run(void *priv)
 	u32 fail_res = 0;
 	int ret = 0;
 
+	pm_runtime_resume_and_get(inst->dev->dev);
 	switch (inst->state) {
 	case VPU_INST_STATE_PIC_RUN:
 		ret = start_encode(inst, &fail_res);
@@ -1491,6 +1502,8 @@ static void wave5_vpu_enc_device_run(void *priv)
 			break;
 		}
 		dev_dbg(inst->dev->dev, "%s: leave with active job", __func__);
+		pm_runtime_mark_last_busy(inst->dev->dev);
+		pm_runtime_put_autosuspend(inst->dev->dev);
 		return;
 	default:
 		WARN(1, "Execution of a job in state %s is invalid.\n",
@@ -1498,6 +1511,8 @@ static void wave5_vpu_enc_device_run(void *priv)
 		break;
 	}
 	dev_dbg(inst->dev->dev, "%s: leave and finish job", __func__);
+	pm_runtime_mark_last_busy(inst->dev->dev);
+	pm_runtime_put_autosuspend(inst->dev->dev);
 	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
 }
 
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu.c b/drivers/media/platform/chips-media/wave5/wave5-vpu.c
index 0d90b5820bef..8e08461b3515 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu.c
@@ -10,6 +10,7 @@
 #include <linux/clk.h>
 #include <linux/firmware.h>
 #include <linux/interrupt.h>
+#include <linux/pm_runtime.h>
 #include "wave5-vpu.h"
 #include "wave5-regdefine.h"
 #include "wave5-vpuconfig.h"
@@ -117,6 +118,38 @@ static int wave5_vpu_load_firmware(struct device *dev, const char *fw_name,
 	return 0;
 }
 
+static int wave5_pm_suspend(struct device *dev)
+{
+	struct vpu_device *vpu = dev_get_drvdata(dev);
+
+	if (pm_runtime_suspended(dev))
+		return 0;
+
+	wave5_vpu_sleep_wake(dev, true, NULL, 0);
+	clk_bulk_disable_unprepare(vpu->num_clks, vpu->clks);
+
+	return 0;
+}
+
+static int wave5_pm_resume(struct device *dev)
+{
+	struct vpu_device *vpu = dev_get_drvdata(dev);
+	int ret = 0;
+
+	wave5_vpu_sleep_wake(dev, false, NULL, 0);
+	ret = clk_bulk_prepare_enable(vpu->num_clks, vpu->clks);
+	if (ret) {
+		dev_err(dev, "Enabling clocks, fail: %d\n", ret);
+		return ret;
+	}
+
+	return ret;
+}
+
+static const struct dev_pm_ops wave5_pm_ops = {
+	SET_RUNTIME_PM_OPS(wave5_pm_suspend, wave5_pm_resume, NULL)
+};
+
 static int wave5_vpu_probe(struct platform_device *pdev)
 {
 	int ret;
@@ -232,6 +265,12 @@ static int wave5_vpu_probe(struct platform_device *pdev)
 		 (match_data->flags & WAVE5_IS_DEC) ? "'DECODE'" : "");
 	dev_info(&pdev->dev, "Product Code:      0x%x\n", dev->product_code);
 	dev_info(&pdev->dev, "Firmware Revision: %u\n", fw_revision);
+
+	pm_runtime_set_autosuspend_delay(&pdev->dev, 5000);
+	pm_runtime_use_autosuspend(&pdev->dev);
+	pm_runtime_enable(&pdev->dev);
+	wave5_vpu_sleep_wake(&pdev->dev, true, NULL, 0);
+
 	return 0;
 
 err_enc_unreg:
@@ -254,6 +293,9 @@ static int wave5_vpu_remove(struct platform_device *pdev)
 {
 	struct vpu_device *dev = dev_get_drvdata(&pdev->dev);
 
+	pm_runtime_put_sync(&pdev->dev);
+	pm_runtime_disable(&pdev->dev);
+
 	mutex_destroy(&dev->dev_lock);
 	mutex_destroy(&dev->hw_lock);
 	clk_bulk_disable_unprepare(dev->num_clks, dev->clks);
@@ -281,6 +323,7 @@ static struct platform_driver wave5_vpu_driver = {
 	.driver = {
 		.name = VPU_PLATFORM_DEVICE_NAME,
 		.of_match_table = of_match_ptr(wave5_dt_ids),
+		.pm = &wave5_pm_ops,
 		},
 	.probe = wave5_vpu_probe,
 	.remove = wave5_vpu_remove,
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
index 1a3efb638dde..b0911fef232f 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
@@ -6,6 +6,8 @@
  */
 
 #include <linux/bug.h>
+#include <linux/pm_runtime.h>
+#include <linux/delay.h>
 #include "wave5-vpuapi.h"
 #include "wave5-regdefine.h"
 #include "wave5.h"
@@ -200,9 +202,13 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
 	if (!inst->codec_info)
 		return -EINVAL;
 
+	pm_runtime_resume_and_get(inst->dev->dev);
+
 	ret = mutex_lock_interruptible(&vpu_dev->hw_lock);
-	if (ret)
+	if (ret) {
+		pm_runtime_put_sync(inst->dev->dev);
 		return ret;
+	}
 
 	do {
 		ret = wave5_vpu_dec_finish_seq(inst, fail_res);
@@ -234,7 +240,7 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
 
 unlock_and_return:
 	mutex_unlock(&vpu_dev->hw_lock);
-
+	pm_runtime_put_sync(inst->dev->dev);
 	return ret;
 }
 
@@ -702,6 +708,8 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
 	if (!inst->codec_info)
 		return -EINVAL;
 
+	pm_runtime_resume_and_get(inst->dev->dev);
+
 	ret = mutex_lock_interruptible(&vpu_dev->hw_lock);
 	if (ret)
 		return ret;
@@ -733,9 +741,9 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
 	}
 
 	wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info->vb_task);
-
 	mutex_unlock(&vpu_dev->hw_lock);
 
+	pm_runtime_put_sync(inst->dev->dev);
 	return 0;
 }
 
diff --git a/drivers/media/platform/chips-media/wave5/wave5.h b/drivers/media/platform/chips-media/wave5/wave5.h
index 063028eccd3b..6125eff938a8 100644
--- a/drivers/media/platform/chips-media/wave5/wave5.h
+++ b/drivers/media/platform/chips-media/wave5/wave5.h
@@ -56,6 +56,9 @@ int wave5_vpu_get_version(struct vpu_device *vpu_dev, u32 *revision);
 
 int wave5_vpu_init(struct device *dev, u8 *fw, size_t size);
 
+int wave5_vpu_sleep_wake(struct device *dev, bool i_sleep_wake, const uint16_t *code,
+			 size_t size);
+
 int wave5_vpu_reset(struct device *dev, enum sw_reset_mode reset_mode);
 
 int wave5_vpu_build_up_dec_param(struct vpu_instance *inst, struct dec_open_param *param);
-- 
2.43.0

[RESEND PATCH v2 3/4] media: chips-media: wave5: Use helpers to calculate bytesperline and sizeimage.

[jackson.lee]

From: "Jackson.lee" <jackson.lee@chipsnmedia.com>

Use v4l2-common helper functions to calculate bytesperline and sizeimage, instead of calculating in a wave5 driver directly.
In case of raw(YUV) v4l2_pix_format, the wave5 driver updates v4l2_pix_format_mplane struct through v4l2_fill_pixfmt_mp() function.
Encoder and Decoder need same bytesperline and sizeimage values for same v4l2_pix_format.
So, a wave5_update_pix_fmt is refactored to support both together.

Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
 .../platform/chips-media/wave5/wave5-helper.c |  24 ++
 .../platform/chips-media/wave5/wave5-helper.h |   4 +
 .../chips-media/wave5/wave5-vpu-dec.c         | 245 ++++--------------
 .../chips-media/wave5/wave5-vpu-enc.c         | 179 +++++--------
 .../platform/chips-media/wave5/wave5-vpu.h    |   4 -
 .../chips-media/wave5/wave5-vpuconfig.h       |  25 +-
 6 files changed, 164 insertions(+), 317 deletions(-)

diff --git a/drivers/media/platform/chips-media/wave5/wave5-helper.c b/drivers/media/platform/chips-media/wave5/wave5-helper.c
index 8433ecab230c..53cad4d17aa7 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-helper.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-helper.c
@@ -7,6 +7,8 @@
 
 #include "wave5-helper.h"
 
+#define DEFAULT_BS_SIZE(width, height) ((width) * (height) / 8 * 3)
+
 const char *state_to_str(enum vpu_instance_state state)
 {
 	switch (state) {
@@ -211,3 +213,25 @@ void wave5_return_bufs(struct vb2_queue *q, u32 state)
 		v4l2_m2m_buf_done(vbuf, state);
 	}
 }
+
+void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
+			  unsigned int width,
+			  unsigned int height,
+			  const struct v4l2_frmsize_stepwise frmsize)
+{
+	v4l2_apply_frmsize_constraints(&width, &height, &frmsize);
+
+	if (pix_mp->pixelformat == V4L2_PIX_FMT_HEVC ||
+	    pix_mp->pixelformat == V4L2_PIX_FMT_H264) {
+		pix_mp->width = width;
+		pix_mp->height = height;
+		pix_mp->num_planes = 1;
+		pix_mp->plane_fmt[0].bytesperline = 0;
+		pix_mp->plane_fmt[0].sizeimage = max(DEFAULT_BS_SIZE(width, height),
+						     pix_mp->plane_fmt[0].sizeimage);
+	} else {
+		v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat, width, height);
+	}
+	pix_mp->flags = 0;
+	pix_mp->field = V4L2_FIELD_NONE;
+}
diff --git a/drivers/media/platform/chips-media/wave5/wave5-helper.h b/drivers/media/platform/chips-media/wave5/wave5-helper.h
index 6cee1c14d3ce..a8ecd1920207 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-helper.h
+++ b/drivers/media/platform/chips-media/wave5/wave5-helper.h
@@ -28,4 +28,8 @@ const struct vpu_format *wave5_find_vpu_fmt_by_idx(unsigned int idx,
 						   const struct vpu_format fmt_list[MAX_FMTS]);
 enum wave_std wave5_to_vpu_std(unsigned int v4l2_pix_fmt, enum vpu_instance_type type);
 void wave5_return_bufs(struct vb2_queue *q, u32 state);
+void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
+			  unsigned int width,
+			  unsigned int height,
+			  const struct v4l2_frmsize_stepwise frmsize);
 #endif
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
index a199877c643b..43d0a21193dd 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
@@ -11,115 +11,74 @@
 #define VPU_DEC_DEV_NAME "C&M Wave5 VPU decoder"
 #define VPU_DEC_DRV_NAME "wave5-dec"
 
-#define DEFAULT_SRC_SIZE(width, height) ({			\
-	(width) * (height) / 8 * 3;					\
-})
-
 static const struct vpu_format dec_fmt_list[FMT_TYPES][MAX_FMTS] = {
 	[VPU_FMT_TYPE_CODEC] = {
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_H264,
-			.max_width = 8192,
-			.min_width = 32,
-			.max_height = 4320,
-			.min_height = 32,
 		},
 	},
 	[VPU_FMT_TYPE_RAW] = {
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422P,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422M,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16M,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61M,
-			.max_width = 8192,
-			.min_width = 8,
-			.max_height = 4320,
-			.min_height = 8,
 		},
 	}
 };
 
+static const struct v4l2_frmsize_stepwise dec_frmsize[FMT_TYPES] = {
+	[VPU_FMT_TYPE_CODEC] = {
+		.min_width = W5_MIN_DEC_PIC_WIDTH,
+		.max_width = W5_MAX_DEC_PIC_WIDTH,
+		.step_width = W5_DEC_CODEC_STEP_WIDTH,
+		.min_height = W5_MIN_DEC_PIC_HEIGHT,
+		.max_height = W5_MAX_DEC_PIC_HEIGHT,
+		.step_height = W5_DEC_CODEC_STEP_HEIGHT,
+	},
+	[VPU_FMT_TYPE_RAW] = {
+		.min_width = W5_MIN_DEC_PIC_WIDTH,
+		.max_width = W5_MAX_DEC_PIC_WIDTH,
+		.step_width = W5_DEC_RAW_STEP_WIDTH,
+		.min_height = W5_MIN_DEC_PIC_HEIGHT,
+		.max_height = W5_MAX_DEC_PIC_HEIGHT,
+		.step_height = W5_DEC_RAW_STEP_HEIGHT,
+	},
+};
+
 /*
  * Make sure that the state switch is allowed and add logging for debugging
  * purposes
@@ -234,74 +193,6 @@ static void wave5_handle_src_buffer(struct vpu_instance *inst, dma_addr_t rd_ptr
 	inst->remaining_consumed_bytes = consumed_bytes;
 }
 
-static void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp, unsigned int width,
-				 unsigned int height)
-{
-	switch (pix_mp->pixelformat) {
-	case V4L2_PIX_FMT_YUV420:
-	case V4L2_PIX_FMT_NV12:
-	case V4L2_PIX_FMT_NV21:
-		pix_mp->width = round_up(width, 32);
-		pix_mp->height = round_up(height, 16);
-		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[0].sizeimage = width * height * 3 / 2;
-		break;
-	case V4L2_PIX_FMT_YUV422P:
-	case V4L2_PIX_FMT_NV16:
-	case V4L2_PIX_FMT_NV61:
-		pix_mp->width = round_up(width, 32);
-		pix_mp->height = round_up(height, 16);
-		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[0].sizeimage = width * height * 2;
-		break;
-	case V4L2_PIX_FMT_YUV420M:
-		pix_mp->width = round_up(width, 32);
-		pix_mp->height = round_up(height, 16);
-		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[0].sizeimage = width * height;
-		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
-		pix_mp->plane_fmt[1].sizeimage = width * height / 4;
-		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
-		pix_mp->plane_fmt[2].sizeimage = width * height / 4;
-		break;
-	case V4L2_PIX_FMT_NV12M:
-	case V4L2_PIX_FMT_NV21M:
-		pix_mp->width = round_up(width, 32);
-		pix_mp->height = round_up(height, 16);
-		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[0].sizeimage = width * height;
-		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[1].sizeimage = width * height / 2;
-		break;
-	case V4L2_PIX_FMT_YUV422M:
-		pix_mp->width = round_up(width, 32);
-		pix_mp->height = round_up(height, 16);
-		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[0].sizeimage = width * height;
-		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
-		pix_mp->plane_fmt[1].sizeimage = width * height / 2;
-		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
-		pix_mp->plane_fmt[2].sizeimage = width * height / 2;
-		break;
-	case V4L2_PIX_FMT_NV16M:
-	case V4L2_PIX_FMT_NV61M:
-		pix_mp->width = round_up(width, 32);
-		pix_mp->height = round_up(height, 16);
-		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[0].sizeimage = width * height;
-		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[1].sizeimage = width * height;
-		break;
-	default:
-		pix_mp->width = width;
-		pix_mp->height = height;
-		pix_mp->plane_fmt[0].bytesperline = 0;
-		pix_mp->plane_fmt[0].sizeimage = max(DEFAULT_SRC_SIZE(width, height),
-						     pix_mp->plane_fmt[0].sizeimage);
-		break;
-	}
-}
-
 static int start_decode(struct vpu_instance *inst, u32 *fail_res)
 {
 	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
@@ -397,9 +288,11 @@ static int handle_dynamic_resolution_change(struct vpu_instance *inst)
 			initial_info->pic_crop_rect.top - initial_info->pic_crop_rect.bottom;
 
 		wave5_update_pix_fmt(&inst->src_fmt, initial_info->pic_width,
-				     initial_info->pic_height);
+						     initial_info->pic_height,
+						     dec_frmsize[VPU_FMT_TYPE_CODEC]);
 		wave5_update_pix_fmt(&inst->dst_fmt, initial_info->pic_width,
-				     initial_info->pic_height);
+						     initial_info->pic_height,
+						     dec_frmsize[VPU_FMT_TYPE_RAW]);
 	}
 
 	v4l2_event_queue_fh(fh, &vpu_event_src_ch);
@@ -548,12 +441,7 @@ static int wave5_vpu_dec_enum_framesizes(struct file *f, void *fh, struct v4l2_f
 	}
 
 	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
-	fsize->stepwise.min_width = vpu_fmt->min_width;
-	fsize->stepwise.max_width = vpu_fmt->max_width;
-	fsize->stepwise.step_width = 1;
-	fsize->stepwise.min_height = vpu_fmt->min_height;
-	fsize->stepwise.max_height = vpu_fmt->max_height;
-	fsize->stepwise.step_height = 1;
+	fsize->stepwise = dec_frmsize[VPU_FMT_TYPE_CODEC];
 
 	return 0;
 }
@@ -589,14 +477,10 @@ static int wave5_vpu_dec_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
 		width = inst->dst_fmt.width;
 		height = inst->dst_fmt.height;
 		f->fmt.pix_mp.pixelformat = inst->dst_fmt.pixelformat;
-		f->fmt.pix_mp.num_planes = inst->dst_fmt.num_planes;
 	} else {
-		const struct v4l2_format_info *info = v4l2_format_info(vpu_fmt->v4l2_pix_fmt);
-
-		width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width, vpu_fmt->max_width);
-		height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height, vpu_fmt->max_height);
+		width = f->fmt.pix_mp.width;
+		height = f->fmt.pix_mp.height;
 		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
-		f->fmt.pix_mp.num_planes = info->mem_planes;
 	}
 
 	if (p_dec_info->initial_info_obtained) {
@@ -604,9 +488,9 @@ static int wave5_vpu_dec_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
 		height = inst->dst_fmt.height;
 	}
 
-	wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
-	f->fmt.pix_mp.flags = 0;
-	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
+					     height,
+					     dec_frmsize[VPU_FMT_TYPE_RAW]);
 	f->fmt.pix_mp.colorspace = inst->colorspace;
 	f->fmt.pix_mp.ycbcr_enc = inst->ycbcr_enc;
 	f->fmt.pix_mp.quantization = inst->quantization;
@@ -719,6 +603,7 @@ static int wave5_vpu_dec_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
 {
 	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
 	const struct vpu_format *vpu_fmt;
+	int width, height;
 
 	dev_dbg(inst->dev->dev,
 		"%s: fourcc: %u width: %u height: %u num_planes: %u colorspace: %u field: %u\n",
@@ -727,20 +612,18 @@ static int wave5_vpu_dec_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
 
 	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat, dec_fmt_list[VPU_FMT_TYPE_CODEC]);
 	if (!vpu_fmt) {
+		width = inst->src_fmt.width;
+		height = inst->src_fmt.height;
 		f->fmt.pix_mp.pixelformat = inst->src_fmt.pixelformat;
-		f->fmt.pix_mp.num_planes = inst->src_fmt.num_planes;
-		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->src_fmt.width, inst->src_fmt.height);
 	} else {
-		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width, vpu_fmt->max_width);
-		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height, vpu_fmt->max_height);
-
+		width = f->fmt.pix_mp.width;
+		height = f->fmt.pix_mp.height;
 		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
-		f->fmt.pix_mp.num_planes = 1;
-		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
 	}
 
-	f->fmt.pix_mp.flags = 0;
-	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
+					     height,
+					     dec_frmsize[VPU_FMT_TYPE_CODEC]);
 
 	return 0;
 }
@@ -782,7 +665,9 @@ static int wave5_vpu_dec_s_fmt_out(struct file *file, void *fh, struct v4l2_form
 	inst->quantization = f->fmt.pix_mp.quantization;
 	inst->xfer_func = f->fmt.pix_mp.xfer_func;
 
-	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width, f->fmt.pix_mp.height);
+	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width,
+					     f->fmt.pix_mp.height,
+					     dec_frmsize[VPU_FMT_TYPE_RAW]);
 
 	return 0;
 }
@@ -1005,6 +890,7 @@ static int wave5_vpu_dec_queue_setup(struct vb2_queue *q, unsigned int *num_buff
 	struct vpu_instance *inst = vb2_get_drv_priv(q);
 	struct v4l2_pix_format_mplane inst_format =
 		(q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) ? inst->src_fmt : inst->dst_fmt;
+	unsigned int i;
 
 	dev_dbg(inst->dev->dev, "%s: num_buffers: %u | num_planes: %u | type: %u\n", __func__,
 		*num_buffers, *num_planes, q->type);
@@ -1018,31 +904,9 @@ static int wave5_vpu_dec_queue_setup(struct vb2_queue *q, unsigned int *num_buff
 		if (*num_buffers < inst->fbc_buf_count)
 			*num_buffers = inst->fbc_buf_count;
 
-		if (*num_planes == 1) {
-			if (inst->output_format == FORMAT_422)
-				sizes[0] = inst_format.width * inst_format.height * 2;
-			else
-				sizes[0] = inst_format.width * inst_format.height * 3 / 2;
-			dev_dbg(inst->dev->dev, "%s: size[0]: %u\n", __func__, sizes[0]);
-		} else if (*num_planes == 2) {
-			sizes[0] = inst_format.width * inst_format.height;
-			if (inst->output_format == FORMAT_422)
-				sizes[1] = inst_format.width * inst_format.height;
-			else
-				sizes[1] = inst_format.width * inst_format.height / 2;
-			dev_dbg(inst->dev->dev, "%s: size[0]: %u | size[1]: %u\n",
-				__func__, sizes[0], sizes[1]);
-		} else if (*num_planes == 3) {
-			sizes[0] = inst_format.width * inst_format.height;
-			if (inst->output_format == FORMAT_422) {
-				sizes[1] = inst_format.width * inst_format.height / 2;
-				sizes[2] = inst_format.width * inst_format.height / 2;
-			} else {
-				sizes[1] = inst_format.width * inst_format.height / 4;
-				sizes[2] = inst_format.width * inst_format.height / 4;
-			}
-			dev_dbg(inst->dev->dev, "%s: size[0]: %u | size[1]: %u | size[2]: %u\n",
-				__func__, sizes[0], sizes[1], sizes[2]);
+		for (i = 0; i < *num_planes; i++) {
+			sizes[i] = inst_format.plane_fmt[i].sizeimage;
+			dev_dbg(inst->dev->dev, "%s: size[%u]: %u\n", __func__, i, sizes[i]);
 		}
 	}
 
@@ -1564,20 +1428,15 @@ static const struct vb2_ops wave5_vpu_dec_vb2_ops = {
 static void wave5_set_default_format(struct v4l2_pix_format_mplane *src_fmt,
 				     struct v4l2_pix_format_mplane *dst_fmt)
 {
-	unsigned int dst_pix_fmt = dec_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
-	const struct v4l2_format_info *dst_fmt_info = v4l2_format_info(dst_pix_fmt);
-
 	src_fmt->pixelformat = dec_fmt_list[VPU_FMT_TYPE_CODEC][0].v4l2_pix_fmt;
-	src_fmt->field = V4L2_FIELD_NONE;
-	src_fmt->flags = 0;
-	src_fmt->num_planes = 1;
-	wave5_update_pix_fmt(src_fmt, 720, 480);
-
-	dst_fmt->pixelformat = dst_pix_fmt;
-	dst_fmt->field = V4L2_FIELD_NONE;
-	dst_fmt->flags = 0;
-	dst_fmt->num_planes = dst_fmt_info->mem_planes;
-	wave5_update_pix_fmt(dst_fmt, 736, 480);
+	wave5_update_pix_fmt(src_fmt, W5_DEF_DEC_PIC_WIDTH,
+				      W5_DEF_DEC_PIC_HEIGHT,
+				      dec_frmsize[VPU_FMT_TYPE_CODEC]);
+
+	dst_fmt->pixelformat = dec_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
+	wave5_update_pix_fmt(dst_fmt, W5_DEF_DEC_PIC_WIDTH,
+				      W5_DEF_DEC_PIC_HEIGHT,
+				      dec_frmsize[VPU_FMT_TYPE_RAW]);
 }
 
 static int wave5_vpu_dec_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
index 013e2bb37fbb..5a32bb138158 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
@@ -15,65 +15,52 @@ static const struct vpu_format enc_fmt_list[FMT_TYPES][MAX_FMTS] = {
 	[VPU_FMT_TYPE_CODEC] = {
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
-			.max_width = W5_MAX_ENC_PIC_WIDTH,
-			.min_width = W5_MIN_ENC_PIC_WIDTH,
-			.max_height = W5_MAX_ENC_PIC_HEIGHT,
-			.min_height = W5_MIN_ENC_PIC_HEIGHT,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_H264,
-			.max_width = W5_MAX_ENC_PIC_WIDTH,
-			.min_width = W5_MIN_ENC_PIC_WIDTH,
-			.max_height = W5_MAX_ENC_PIC_HEIGHT,
-			.min_height = W5_MIN_ENC_PIC_HEIGHT,
 		},
 	},
 	[VPU_FMT_TYPE_RAW] = {
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
-			.max_width = W5_MAX_ENC_PIC_WIDTH,
-			.min_width = W5_MIN_ENC_PIC_WIDTH,
-			.max_height = W5_MAX_ENC_PIC_HEIGHT,
-			.min_height = W5_MIN_ENC_PIC_HEIGHT,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
-			.max_width = W5_MAX_ENC_PIC_WIDTH,
-			.min_width = W5_MIN_ENC_PIC_WIDTH,
-			.max_height = W5_MAX_ENC_PIC_HEIGHT,
-			.min_height = W5_MIN_ENC_PIC_HEIGHT,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
-			.max_width = W5_MAX_ENC_PIC_WIDTH,
-			.min_width = W5_MIN_ENC_PIC_WIDTH,
-			.max_height = W5_MAX_ENC_PIC_HEIGHT,
-			.min_height = W5_MIN_ENC_PIC_HEIGHT,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
-			.max_width = W5_MAX_ENC_PIC_WIDTH,
-			.min_width = W5_MIN_ENC_PIC_WIDTH,
-			.max_height = W5_MAX_ENC_PIC_HEIGHT,
-			.min_height = W5_MIN_ENC_PIC_HEIGHT,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
-			.max_width = W5_MAX_ENC_PIC_WIDTH,
-			.min_width = W5_MIN_ENC_PIC_WIDTH,
-			.max_height = W5_MAX_ENC_PIC_HEIGHT,
-			.min_height = W5_MIN_ENC_PIC_HEIGHT,
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
-			.max_width = W5_MAX_ENC_PIC_WIDTH,
-			.min_width = W5_MIN_ENC_PIC_WIDTH,
-			.max_height = W5_MAX_ENC_PIC_HEIGHT,
-			.min_height = W5_MIN_ENC_PIC_HEIGHT,
 		},
 	}
 };
 
+static const struct v4l2_frmsize_stepwise enc_frmsize[FMT_TYPES] = {
+	[VPU_FMT_TYPE_CODEC] = {
+		.min_width = W5_MIN_ENC_PIC_WIDTH,
+		.max_width = W5_MAX_ENC_PIC_WIDTH,
+		.step_width = W5_ENC_CODEC_STEP_WIDTH,
+		.min_height = W5_MIN_ENC_PIC_HEIGHT,
+		.max_height = W5_MAX_ENC_PIC_HEIGHT,
+		.step_height = W5_ENC_CODEC_STEP_HEIGHT,
+	},
+	[VPU_FMT_TYPE_RAW] = {
+		.min_width = W5_MIN_ENC_PIC_WIDTH,
+		.max_width = W5_MAX_ENC_PIC_WIDTH,
+		.step_width = W5_ENC_RAW_STEP_WIDTH,
+		.min_height = W5_MIN_ENC_PIC_HEIGHT,
+		.max_height = W5_MAX_ENC_PIC_HEIGHT,
+		.step_height = W5_ENC_RAW_STEP_HEIGHT,
+	},
+};
+
 static int switch_state(struct vpu_instance *inst, enum vpu_instance_state state)
 {
 	switch (state) {
@@ -106,46 +93,6 @@ static int switch_state(struct vpu_instance *inst, enum vpu_instance_state state
 	return -EINVAL;
 }
 
-static void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp, unsigned int width,
-				 unsigned int height)
-{
-	switch (pix_mp->pixelformat) {
-	case V4L2_PIX_FMT_YUV420:
-	case V4L2_PIX_FMT_NV12:
-	case V4L2_PIX_FMT_NV21:
-		pix_mp->width = width;
-		pix_mp->height = height;
-		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height * 3 / 2;
-		break;
-	case V4L2_PIX_FMT_YUV420M:
-		pix_mp->width = width;
-		pix_mp->height = height;
-		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height;
-		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
-		pix_mp->plane_fmt[1].sizeimage = round_up(width, 32) * height / 4;
-		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
-		pix_mp->plane_fmt[2].sizeimage = round_up(width, 32) * height / 4;
-		break;
-	case V4L2_PIX_FMT_NV12M:
-	case V4L2_PIX_FMT_NV21M:
-		pix_mp->width = width;
-		pix_mp->height = height;
-		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height;
-		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
-		pix_mp->plane_fmt[1].sizeimage = round_up(width, 32) * height / 2;
-		break;
-	default:
-		pix_mp->width = width;
-		pix_mp->height = height;
-		pix_mp->plane_fmt[0].bytesperline = 0;
-		pix_mp->plane_fmt[0].sizeimage = width * height / 8 * 3;
-		break;
-	}
-}
-
 static int start_encode(struct vpu_instance *inst, u32 *fail_res)
 {
 	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
@@ -360,13 +307,8 @@ static int wave5_vpu_enc_enum_framesizes(struct file *f, void *fh, struct v4l2_f
 			return -EINVAL;
 	}
 
-	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
-	fsize->stepwise.min_width = vpu_fmt->min_width;
-	fsize->stepwise.max_width = vpu_fmt->max_width;
-	fsize->stepwise.step_width = 1;
-	fsize->stepwise.min_height = vpu_fmt->min_height;
-	fsize->stepwise.max_height = vpu_fmt->max_height;
-	fsize->stepwise.step_height = 1;
+	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+	fsize->stepwise = enc_frmsize[VPU_FMT_TYPE_CODEC];
 
 	return 0;
 }
@@ -392,6 +334,7 @@ static int wave5_vpu_enc_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
 {
 	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
 	const struct vpu_format *vpu_fmt;
+	int width, height;
 
 	dev_dbg(inst->dev->dev, "%s: fourcc: %u width: %u height: %u num_planes: %u field: %u\n",
 		__func__, f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width, f->fmt.pix_mp.height,
@@ -399,20 +342,18 @@ static int wave5_vpu_enc_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
 
 	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat, enc_fmt_list[VPU_FMT_TYPE_CODEC]);
 	if (!vpu_fmt) {
+		width = inst->dst_fmt.width;
+		height = inst->dst_fmt.height;
 		f->fmt.pix_mp.pixelformat = inst->dst_fmt.pixelformat;
-		f->fmt.pix_mp.num_planes = inst->dst_fmt.num_planes;
-		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->dst_fmt.width, inst->dst_fmt.height);
 	} else {
-		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width, vpu_fmt->max_width);
-		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height, vpu_fmt->max_height);
-
+		width = f->fmt.pix_mp.width;
+		height = f->fmt.pix_mp.height;
 		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
-		f->fmt.pix_mp.num_planes = 1;
-		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
 	}
 
-	f->fmt.pix_mp.flags = 0;
-	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
+					     height,
+					     enc_frmsize[VPU_FMT_TYPE_CODEC]);
 	f->fmt.pix_mp.colorspace = inst->colorspace;
 	f->fmt.pix_mp.ycbcr_enc = inst->ycbcr_enc;
 	f->fmt.pix_mp.quantization = inst->quantization;
@@ -500,6 +441,7 @@ static int wave5_vpu_enc_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
 {
 	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
 	const struct vpu_format *vpu_fmt;
+	int width, height;
 
 	dev_dbg(inst->dev->dev, "%s: fourcc: %u width: %u height: %u num_planes: %u field: %u\n",
 		__func__, f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width, f->fmt.pix_mp.height,
@@ -507,21 +449,18 @@ static int wave5_vpu_enc_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
 
 	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat, enc_fmt_list[VPU_FMT_TYPE_RAW]);
 	if (!vpu_fmt) {
+		width = inst->src_fmt.width;
+		height = inst->src_fmt.height;
 		f->fmt.pix_mp.pixelformat = inst->src_fmt.pixelformat;
-		f->fmt.pix_mp.num_planes = inst->src_fmt.num_planes;
-		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->src_fmt.width, inst->src_fmt.height);
 	} else {
-		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width, vpu_fmt->max_width);
-		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height, vpu_fmt->max_height);
-		const struct v4l2_format_info *info = v4l2_format_info(vpu_fmt->v4l2_pix_fmt);
-
+		width = f->fmt.pix_mp.width;
+		height = f->fmt.pix_mp.height;
 		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
-		f->fmt.pix_mp.num_planes = info->mem_planes;
-		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
 	}
 
-	f->fmt.pix_mp.flags = 0;
-	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
+					     height,
+					     enc_frmsize[VPU_FMT_TYPE_RAW]);
 
 	return 0;
 }
@@ -568,7 +507,11 @@ static int wave5_vpu_enc_s_fmt_out(struct file *file, void *fh, struct v4l2_form
 	inst->quantization = f->fmt.pix_mp.quantization;
 	inst->xfer_func = f->fmt.pix_mp.xfer_func;
 
-	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width, f->fmt.pix_mp.height);
+	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width,
+					     f->fmt.pix_mp.height,
+					     enc_frmsize[VPU_FMT_TYPE_CODEC]);
+	inst->conf_win.width = inst->dst_fmt.width;
+	inst->conf_win.height = inst->dst_fmt.height;
 
 	return 0;
 }
@@ -584,12 +527,17 @@ static int wave5_vpu_enc_g_selection(struct file *file, void *fh, struct v4l2_se
 	switch (s->target) {
 	case V4L2_SEL_TGT_CROP_DEFAULT:
 	case V4L2_SEL_TGT_CROP_BOUNDS:
-	case V4L2_SEL_TGT_CROP:
 		s->r.left = 0;
 		s->r.top = 0;
 		s->r.width = inst->dst_fmt.width;
 		s->r.height = inst->dst_fmt.height;
 		break;
+	case V4L2_SEL_TGT_CROP:
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = inst->conf_win.width;
+		s->r.height = inst->conf_win.height;
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -612,8 +560,10 @@ static int wave5_vpu_enc_s_selection(struct file *file, void *fh, struct v4l2_se
 
 	s->r.left = 0;
 	s->r.top = 0;
-	s->r.width = inst->src_fmt.width;
-	s->r.height = inst->src_fmt.height;
+	s->r.width = min(s->r.width, inst->dst_fmt.width);
+	s->r.height = min(s->r.height, inst->dst_fmt.height);
+
+	inst->conf_win = s->r;
 
 	return 0;
 }
@@ -1151,8 +1101,8 @@ static void wave5_set_enc_openparam(struct enc_open_param *open_param,
 	open_param->wave_param.lambda_scaling_enable = 1;
 
 	open_param->line_buf_int_en = true;
-	open_param->pic_width = inst->dst_fmt.width;
-	open_param->pic_height = inst->dst_fmt.height;
+	open_param->pic_width = inst->conf_win.width;
+	open_param->pic_height = inst->conf_win.height;
 	open_param->frame_rate_info = inst->frame_rate;
 	open_param->rc_enable = inst->rc_enable;
 	if (inst->rc_enable) {
@@ -1456,20 +1406,15 @@ static const struct vb2_ops wave5_vpu_enc_vb2_ops = {
 static void wave5_set_default_format(struct v4l2_pix_format_mplane *src_fmt,
 				     struct v4l2_pix_format_mplane *dst_fmt)
 {
-	unsigned int src_pix_fmt = enc_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
-	const struct v4l2_format_info *src_fmt_info = v4l2_format_info(src_pix_fmt);
-
-	src_fmt->pixelformat = src_pix_fmt;
-	src_fmt->field = V4L2_FIELD_NONE;
-	src_fmt->flags = 0;
-	src_fmt->num_planes = src_fmt_info->mem_planes;
-	wave5_update_pix_fmt(src_fmt, 416, 240);
+	src_fmt->pixelformat = enc_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
+	wave5_update_pix_fmt(src_fmt, W5_DEF_ENC_PIC_WIDTH,
+				      W5_DEF_ENC_PIC_HEIGHT,
+				      enc_frmsize[VPU_FMT_TYPE_RAW]);
 
 	dst_fmt->pixelformat = enc_fmt_list[VPU_FMT_TYPE_CODEC][0].v4l2_pix_fmt;
-	dst_fmt->field = V4L2_FIELD_NONE;
-	dst_fmt->flags = 0;
-	dst_fmt->num_planes = 1;
-	wave5_update_pix_fmt(dst_fmt, 416, 240);
+	wave5_update_pix_fmt(dst_fmt, W5_DEF_ENC_PIC_WIDTH,
+				      W5_DEF_ENC_PIC_HEIGHT,
+				      enc_frmsize[VPU_FMT_TYPE_CODEC]);
 }
 
 static int wave5_vpu_enc_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
@@ -1734,6 +1679,8 @@ static int wave5_vpu_open_enc(struct file *filp)
 	v4l2_ctrl_handler_setup(v4l2_ctrl_hdl);
 
 	wave5_set_default_format(&inst->src_fmt, &inst->dst_fmt);
+	inst->conf_win.width = inst->dst_fmt.width;
+	inst->conf_win.height = inst->dst_fmt.height;
 	inst->colorspace = V4L2_COLORSPACE_REC709;
 	inst->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
 	inst->quantization = V4L2_QUANTIZATION_DEFAULT;
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu.h b/drivers/media/platform/chips-media/wave5/wave5-vpu.h
index 32b7fd3730b5..691d6341fcda 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu.h
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu.h
@@ -38,10 +38,6 @@ enum vpu_fmt_type {
 
 struct vpu_format {
 	unsigned int v4l2_pix_fmt;
-	unsigned int max_width;
-	unsigned int min_width;
-	unsigned int max_height;
-	unsigned int min_height;
 };
 
 static inline struct vpu_instance *wave5_to_vpu_inst(struct v4l2_fh *vfh)
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h b/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
index d9751eedb0f9..e6a34ae7084e 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
@@ -30,10 +30,27 @@
 
 #define MAX_NUM_INSTANCE                32
 
-#define W5_MIN_ENC_PIC_WIDTH            256
-#define W5_MIN_ENC_PIC_HEIGHT           128
-#define W5_MAX_ENC_PIC_WIDTH            8192
-#define W5_MAX_ENC_PIC_HEIGHT           8192
+#define W5_DEF_DEC_PIC_WIDTH            720U
+#define W5_DEF_DEC_PIC_HEIGHT           480U
+#define W5_MIN_DEC_PIC_WIDTH            32U
+#define W5_MIN_DEC_PIC_HEIGHT           32U
+#define W5_MAX_DEC_PIC_WIDTH            8192U
+#define W5_MAX_DEC_PIC_HEIGHT           4320U
+#define W5_DEC_CODEC_STEP_WIDTH         1U
+#define W5_DEC_CODEC_STEP_HEIGHT        1U
+#define W5_DEC_RAW_STEP_WIDTH           32U
+#define W5_DEC_RAW_STEP_HEIGHT          16U
+
+#define W5_DEF_ENC_PIC_WIDTH            416U
+#define W5_DEF_ENC_PIC_HEIGHT           240U
+#define W5_MIN_ENC_PIC_WIDTH            256U
+#define W5_MIN_ENC_PIC_HEIGHT           128U
+#define W5_MAX_ENC_PIC_WIDTH            8192U
+#define W5_MAX_ENC_PIC_HEIGHT           8192U
+#define W5_ENC_CODEC_STEP_WIDTH         8U
+#define W5_ENC_CODEC_STEP_HEIGHT        8U
+#define W5_ENC_RAW_STEP_WIDTH           32U
+#define W5_ENC_RAW_STEP_HEIGHT          16U
 
 //  application specific configuration
 #define VPU_ENC_TIMEOUT                 60000
-- 
2.43.0

[RESEND PATCH v2 4/4] media: chips-media: wave5: Support YUV422 raw pixel-formats on the encoder.

[jackson.lee]

From: "Jackson.lee" <jackson.lee@chipsnmedia.com>

Add support for the YUV422P, NV16, NV61, YUV422M, NV16M, NV61M raw pixel-formats to the Wave5 encoder.
All these formats have a chroma subsampling ratio of 4:2:2 and therefore require a new image size calculation as the driver previously only handled a ratio of 4:2:0.

Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
---
 .../chips-media/wave5/wave5-vpu-enc.c         | 59 +++++++++++++++++--
 1 file changed, 54 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
index 5a32bb138158..77657f63a169 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
@@ -39,6 +39,24 @@ static const struct vpu_format enc_fmt_list[FMT_TYPES][MAX_FMTS] = {
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
 		},
+		{
+			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422P,
+		},
+		{
+			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16,
+		},
+		{
+			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61,
+		},
+		{
+			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422M,
+		},
+		{
+			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16M,
+		},
+		{
+			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61M,
+		},
 	}
 };
 
@@ -101,13 +119,30 @@ static int start_encode(struct vpu_instance *inst, u32 *fail_res)
 	struct vb2_v4l2_buffer *dst_buf;
 	struct frame_buffer frame_buf;
 	struct enc_param pic_param;
-	u32 stride = ALIGN(inst->dst_fmt.width, 32);
-	u32 luma_size = (stride * inst->dst_fmt.height);
-	u32 chroma_size = ((stride / 2) * (inst->dst_fmt.height / 2));
+	u32 stride = inst->src_fmt.plane_fmt[0].bytesperline;
+	u32 luma_size = (stride * inst->src_fmt.height);
+	u32 chroma_size = 0;
 
 	memset(&pic_param, 0, sizeof(struct enc_param));
 	memset(&frame_buf, 0, sizeof(struct frame_buffer));
 
+	if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV420 ||
+	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV420M)
+		chroma_size = luma_size / 4;
+	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12 ||
+		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21 ||
+		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12M ||
+		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21M)
+		chroma_size = luma_size / 2;
+	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422P ||
+		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422M)
+		chroma_size = luma_size / 2;
+	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16 ||
+		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61 ||
+		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16M ||
+		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61M)
+		chroma_size = luma_size;
+
 	dst_buf = v4l2_m2m_next_dst_buf(m2m_ctx);
 	if (!dst_buf) {
 		dev_dbg(inst->dev->dev, "%s: No destination buffer found\n", __func__);
@@ -490,11 +525,15 @@ static int wave5_vpu_enc_s_fmt_out(struct file *file, void *fh, struct v4l2_form
 	}
 
 	if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12 ||
-	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12M) {
+	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12M ||
+	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16 ||
+	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16M) {
 		inst->cbcr_interleave = true;
 		inst->nv21 = false;
 	} else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21 ||
-		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21M) {
+		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21M ||
+		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61 ||
+		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61M) {
 		inst->cbcr_interleave = true;
 		inst->nv21 = true;
 	} else {
@@ -1086,6 +1125,16 @@ static void wave5_set_enc_openparam(struct enc_open_param *open_param,
 	u32 num_ctu_row = ALIGN(inst->dst_fmt.height, 64) / 64;
 	u32 num_mb_row = ALIGN(inst->dst_fmt.height, 16) / 16;
 
+	if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422P ||
+	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16 ||
+	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61 ||
+	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422M ||
+	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16M ||
+	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61M)
+		open_param->src_format = FORMAT_422;
+	else
+		open_param->src_format = FORMAT_420;
+
 	open_param->wave_param.gop_preset_idx = PRESET_IDX_IPP_SINGLE;
 	open_param->wave_param.hvs_qp_scale = 2;
 	open_param->wave_param.hvs_max_delta_qp = 10;
-- 
2.43.0

RE: [RESEND PATCH v2 0/4] Add auto suspend/resume,YUV422 format,SPS/PPS generation for each IDR

[jackson.lee]

Hello Hans, Sebastian, and Nicolas

Can you please review the following patch for the V2 ?

https://www.spinics.net/lists/kernel/msg5136434.html


thanks.
Jackson

> -----Original Message-----
> From: jackson.lee
> Sent: Monday, March 11, 2024 7:57 PM
> To: mchehab@kernel.org; nicolas@ndufresne.ca;
> sebastian.fricke@collabora.com
> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>; lafley.kim
> <lafley.kim@chipsnmedia.com>; b-brnich@ti.com; jackson.lee
> <jackson.lee@chipsnmedia.com>
> Subject: [RESEND PATCH v2 0/4] Add auto suspend/resume,YUV422
> format,SPS/PPS generation for each IDR
> 
> From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> 
> The wave5 codec driver is a stateful encoder/decoder.
> The following patches is for supporting yuv422 inpuy format, supporting
> runtime suspend/resume feature and extra things.
> 
> v4l2-compliance results:
> ========================
> 
> v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
> 
> Buffer ioctls:
>             warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not
> supported
>             warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not
> supported
>     test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>     test VIDIOC_EXPBUF: OK
>     test Requests: OK (Not Supported)
> 
> Total for wave5-dec device /dev/video0: 45, Succeeded: 45, Failed: 0,
> Warnings: 2 Total for wave5-enc device /dev/video1: 45, Succeeded: 45,
> Failed: 0, Warnings: 0
> 
> Fluster test results:
> =====================
> 
> Running test suite JCT-VC-HEVC_V1 with decoder GStreamer-H.265-V4L2-Gst1.0
> Using 1 parallel job(s)
> Ran 132/147 tests successfully               in 94.782 secs
> 
> (1 test fails because of not supporting to parse multi frames, 1 test
> fails because of a missing frame and slight corruption, 2 tests fail
> because of sizes which are incompatible with the IP, 11 tests fail because
> of unsupported 10 bit format)
> 
> Running test suite JVT-AVC_V1 with decoder GStreamer-H.264-V4L2-Gst1.0
> Using 1 parallel job(s)
> Ran 77/135 tests successfully               in 37.364 secs
> 
> (58 fail because the hardware is unable to decode  MBAFF / FMO / Field /
> Extended profile streams.)
> 
> 
> Chnage since v1:
> =================
> 
> * For [PATCH v2 0/4] media: chips-media: wave5: Support SPS/PPS generation
> for each IDR
>  - define a macro for register addresses
> 
> * For [PATCH v2 1/4] media: chips-media: wave5: Support runtime
> suspend/resume
>  - add auto suspend/resume
> 
> * For [PATCH v2 2/4] media: chips-media: wave5: Use helpers to calculate
> bytesperline and sizeimage
>  - use helper functions to calculate bytesperline and sizeimage
> 
> * For [PATCH v2 3/4] media: chips-media: wave5: Support YUV422 raw pixel-
> formats on the encoder
>  - remove unnecessary codes
> 
> Change since v0:
> =================
> The DEFAULT_SRC_SIZE macro was defined using multiple lines, To make a
> simple define, tab and multiple lines has been removed, The macro is
> defined using one line.
> 
> Jackson.lee (4):
>   media: chips-media: wave5: Support SPS/PPS generation for each IDR
>   media: chips-media: wave5: Support runtime suspend/resume
>   media: chips-media: wave5: Use helpers to calculate bytesperline and
>     sizeimage.
>   media: chips-media: wave5: Support YUV422 raw pixel-formats on the
>     encoder.
> 
>  .../platform/chips-media/wave5/wave5-helper.c |  24 ++
>  .../platform/chips-media/wave5/wave5-helper.h |   4 +
>  .../platform/chips-media/wave5/wave5-hw.c     |  23 +-
>  .../chips-media/wave5/wave5-vpu-dec.c         | 261 +++++-------------
>  .../chips-media/wave5/wave5-vpu-enc.c         | 260 +++++++++--------
>  .../platform/chips-media/wave5/wave5-vpu.c    |  43 +++
>  .../platform/chips-media/wave5/wave5-vpu.h    |   4 -
>  .../platform/chips-media/wave5/wave5-vpuapi.c |  14 +-
>  .../platform/chips-media/wave5/wave5-vpuapi.h |   1 +
>  .../chips-media/wave5/wave5-vpuconfig.h       |  25 +-
>  .../media/platform/chips-media/wave5/wave5.h  |   3 +
>  11 files changed, 329 insertions(+), 333 deletions(-)
> 
> --
> 2.43.0

RE: [RESEND PATCH v2 0/4] Add auto suspend/resume,YUV422 format,SPS/PPS generation for each IDR

[jackson.lee]

Hello Sebastian, Hans and Nicolas

Could you please review the below patch series ?


https://patchwork.linuxtv.org/project/linux-media/list/?series=12404


thanks
Jackson

> -----Original Message-----
> From: jackson.lee <jackson.lee@chipsnmedia.com>
> Sent: Monday, March 11, 2024 7:56 PM
> To: mchehab@kernel.org; nicolas@ndufresne.ca;
> sebastian.fricke@collabora.com
> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>; lafley.kim
> <lafley.kim@chipsnmedia.com>; b-brnich@ti.com; jackson.lee
> <jackson.lee@chipsnmedia.com>
> Subject: [RESEND PATCH v2 0/4] Add auto suspend/resume,YUV422
> format,SPS/PPS generation for each IDR
> 
> From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> 
> The wave5 codec driver is a stateful encoder/decoder.
> The following patches is for supporting yuv422 inpuy format, supporting
> runtime suspend/resume feature and extra things.
> 
> v4l2-compliance results:
> ========================
> 
> v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
> 
> Buffer ioctls:
>             warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not
> supported
>             warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not
> supported
>     test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>     test VIDIOC_EXPBUF: OK
>     test Requests: OK (Not Supported)
> 
> Total for wave5-dec device /dev/video0: 45, Succeeded: 45, Failed: 0,
> Warnings: 2 Total for wave5-enc device /dev/video1: 45, Succeeded: 45,
> Failed: 0, Warnings: 0
> 
> Fluster test results:
> =====================
> 
> Running test suite JCT-VC-HEVC_V1 with decoder GStreamer-H.265-V4L2-Gst1.0
> Using 1 parallel job(s)
> Ran 132/147 tests successfully               in 94.782 secs
> 
> (1 test fails because of not supporting to parse multi frames, 1 test
> fails because of a missing frame and slight corruption, 2 tests fail
> because of sizes which are incompatible with the IP, 11 tests fail because
> of unsupported 10 bit format)
> 
> Running test suite JVT-AVC_V1 with decoder GStreamer-H.264-V4L2-Gst1.0
> Using 1 parallel job(s)
> Ran 77/135 tests successfully               in 37.364 secs
> 
> (58 fail because the hardware is unable to decode  MBAFF / FMO / Field /
> Extended profile streams.)
> 
> 
> Chnage since v1:
> =================
> 
> * For [PATCH v2 0/4] media: chips-media: wave5: Support SPS/PPS generation
> for each IDR
>  - define a macro for register addresses
> 
> * For [PATCH v2 1/4] media: chips-media: wave5: Support runtime
> suspend/resume
>  - add auto suspend/resume
> 
> * For [PATCH v2 2/4] media: chips-media: wave5: Use helpers to calculate
> bytesperline and sizeimage
>  - use helper functions to calculate bytesperline and sizeimage
> 
> * For [PATCH v2 3/4] media: chips-media: wave5: Support YUV422 raw pixel-
> formats on the encoder
>  - remove unnecessary codes
> 
> Change since v0:
> =================
> The DEFAULT_SRC_SIZE macro was defined using multiple lines, To make a
> simple define, tab and multiple lines has been removed, The macro is
> defined using one line.
> 
> Jackson.lee (4):
>   media: chips-media: wave5: Support SPS/PPS generation for each IDR
>   media: chips-media: wave5: Support runtime suspend/resume
>   media: chips-media: wave5: Use helpers to calculate bytesperline and
>     sizeimage.
>   media: chips-media: wave5: Support YUV422 raw pixel-formats on the
>     encoder.
> 
>  .../platform/chips-media/wave5/wave5-helper.c |  24 ++
>  .../platform/chips-media/wave5/wave5-helper.h |   4 +
>  .../platform/chips-media/wave5/wave5-hw.c     |  23 +-
>  .../chips-media/wave5/wave5-vpu-dec.c         | 261 +++++-------------
>  .../chips-media/wave5/wave5-vpu-enc.c         | 260 +++++++++--------
>  .../platform/chips-media/wave5/wave5-vpu.c    |  43 +++
>  .../platform/chips-media/wave5/wave5-vpu.h    |   4 -
>  .../platform/chips-media/wave5/wave5-vpuapi.c |  14 +-
>  .../platform/chips-media/wave5/wave5-vpuapi.h |   1 +
>  .../chips-media/wave5/wave5-vpuconfig.h       |  25 +-
>  .../media/platform/chips-media/wave5/wave5.h  |   3 +
>  11 files changed, 329 insertions(+), 333 deletions(-)
> 
> --
> 2.43.0

Re: [RESEND PATCH v2 0/4] Add auto suspend/resume,YUV422 format,SPS/PPS generation for each IDR

[sebastian.fricke]

Hey Jackson,

On 25.03.2024 06:19, jackson.lee wrote:
>Hello Sebastian, Hans and Nicolas
>
>Could you please review the below patch series ?

Sorry for the delay, me and Nicolas are currently quite busy. So, the
review of the patchset will be delayed a bit longer, current goal would
be review this until the 20th of April. I'll try to keep you updated.

Greetings,
Sebastian

>
>
>https://patchwork.linuxtv.org/project/linux-media/list/?series=12404
>
>
>thanks
>Jackson
>
>> -----Original Message-----
>> From: jackson.lee <jackson.lee@chipsnmedia.com>
>> Sent: Monday, March 11, 2024 7:56 PM
>> To: mchehab@kernel.org; nicolas@ndufresne.ca;
>> sebastian.fricke@collabora.com
>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>> hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>; lafley.kim
>> <lafley.kim@chipsnmedia.com>; b-brnich@ti.com; jackson.lee
>> <jackson.lee@chipsnmedia.com>
>> Subject: [RESEND PATCH v2 0/4] Add auto suspend/resume,YUV422
>> format,SPS/PPS generation for each IDR
>>
>> From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
>>
>> The wave5 codec driver is a stateful encoder/decoder.
>> The following patches is for supporting yuv422 inpuy format, supporting
>> runtime suspend/resume feature and extra things.
>>
>> v4l2-compliance results:
>> ========================
>>
>> v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
>>
>> Buffer ioctls:
>>             warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not
>> supported
>>             warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not
>> supported
>>     test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>>     test VIDIOC_EXPBUF: OK
>>     test Requests: OK (Not Supported)
>>
>> Total for wave5-dec device /dev/video0: 45, Succeeded: 45, Failed: 0,
>> Warnings: 2 Total for wave5-enc device /dev/video1: 45, Succeeded: 45,
>> Failed: 0, Warnings: 0
>>
>> Fluster test results:
>> =====================
>>
>> Running test suite JCT-VC-HEVC_V1 with decoder GStreamer-H.265-V4L2-Gst1.0
>> Using 1 parallel job(s)
>> Ran 132/147 tests successfully               in 94.782 secs
>>
>> (1 test fails because of not supporting to parse multi frames, 1 test
>> fails because of a missing frame and slight corruption, 2 tests fail
>> because of sizes which are incompatible with the IP, 11 tests fail because
>> of unsupported 10 bit format)
>>
>> Running test suite JVT-AVC_V1 with decoder GStreamer-H.264-V4L2-Gst1.0
>> Using 1 parallel job(s)
>> Ran 77/135 tests successfully               in 37.364 secs
>>
>> (58 fail because the hardware is unable to decode  MBAFF / FMO / Field /
>> Extended profile streams.)
>>
>>
>> Chnage since v1:
>> =================
>>
>> * For [PATCH v2 0/4] media: chips-media: wave5: Support SPS/PPS generation
>> for each IDR
>>  - define a macro for register addresses
>>
>> * For [PATCH v2 1/4] media: chips-media: wave5: Support runtime
>> suspend/resume
>>  - add auto suspend/resume
>>
>> * For [PATCH v2 2/4] media: chips-media: wave5: Use helpers to calculate
>> bytesperline and sizeimage
>>  - use helper functions to calculate bytesperline and sizeimage
>>
>> * For [PATCH v2 3/4] media: chips-media: wave5: Support YUV422 raw pixel-
>> formats on the encoder
>>  - remove unnecessary codes
>>
>> Change since v0:
>> =================
>> The DEFAULT_SRC_SIZE macro was defined using multiple lines, To make a
>> simple define, tab and multiple lines has been removed, The macro is
>> defined using one line.
>>
>> Jackson.lee (4):
>>   media: chips-media: wave5: Support SPS/PPS generation for each IDR
>>   media: chips-media: wave5: Support runtime suspend/resume
>>   media: chips-media: wave5: Use helpers to calculate bytesperline and
>>     sizeimage.
>>   media: chips-media: wave5: Support YUV422 raw pixel-formats on the
>>     encoder.
>>
>>  .../platform/chips-media/wave5/wave5-helper.c |  24 ++
>>  .../platform/chips-media/wave5/wave5-helper.h |   4 +
>>  .../platform/chips-media/wave5/wave5-hw.c     |  23 +-
>>  .../chips-media/wave5/wave5-vpu-dec.c         | 261 +++++-------------
>>  .../chips-media/wave5/wave5-vpu-enc.c         | 260 +++++++++--------
>>  .../platform/chips-media/wave5/wave5-vpu.c    |  43 +++
>>  .../platform/chips-media/wave5/wave5-vpu.h    |   4 -
>>  .../platform/chips-media/wave5/wave5-vpuapi.c |  14 +-
>>  .../platform/chips-media/wave5/wave5-vpuapi.h |   1 +
>>  .../chips-media/wave5/wave5-vpuconfig.h       |  25 +-
>>  .../media/platform/chips-media/wave5/wave5.h  |   3 +
>>  11 files changed, 329 insertions(+), 333 deletions(-)
>>
>> --
>> 2.43.0
>

RE: [RESEND PATCH v2 0/4] Add auto suspend/resume,YUV422 format,SPS/PPS generation for each IDR

[jackson.lee]

Hello Sebastian

Thanks for your reply.

> -----Original Message-----
> From: sebastian.fricke@collabora.com <sebastian.fricke@collabora.com>
> Sent: Thursday, April 4, 2024 4:47 PM
> To: jackson.lee <jackson.lee@chipsnmedia.com>
> Cc: mchehab@kernel.org; nicolas@ndufresne.ca; hverkuil@xs4all.nl; linux-
> media@vger.kernel.org; linux-kernel@vger.kernel.org; Nas Chung
> <nas.chung@chipsnmedia.com>; lafley.kim <lafley.kim@chipsnmedia.com>; b-
> brnich@ti.com
> Subject: Re: [RESEND PATCH v2 0/4] Add auto suspend/resume,YUV422
> format,SPS/PPS generation for each IDR
> 
> Hey Jackson,
> 
> On 25.03.2024 06:19, jackson.lee wrote:
> >Hello Sebastian, Hans and Nicolas
> >
> >Could you please review the below patch series ?
> 
> Sorry for the delay, me and Nicolas are currently quite busy. So, the
> review of the patchset will be delayed a bit longer, current goal would be
> review this until the 20th of April. I'll try to keep you updated.
> 
> Greetings,
> Sebastian
> 
> >
> >
> >https://patchwork.linuxtv.org/project/linux-media/list/?series=12404
> >
> >
> >thanks
> >Jackson
> >
> >> -----Original Message-----
> >> From: jackson.lee <jackson.lee@chipsnmedia.com>
> >> Sent: Monday, March 11, 2024 7:56 PM
> >> To: mchehab@kernel.org; nicolas@ndufresne.ca;
> >> sebastian.fricke@collabora.com
> >> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> >> hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>; lafley.kim
> >> <lafley.kim@chipsnmedia.com>; b-brnich@ti.com; jackson.lee
> >> <jackson.lee@chipsnmedia.com>
> >> Subject: [RESEND PATCH v2 0/4] Add auto suspend/resume,YUV422
> >> format,SPS/PPS generation for each IDR
> >>
> >> From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> >>
> >> The wave5 codec driver is a stateful encoder/decoder.
> >> The following patches is for supporting yuv422 inpuy format,
> >> supporting runtime suspend/resume feature and extra things.
> >>
> >> v4l2-compliance results:
> >> ========================
> >>
> >> v4l2-compliance 1.24.1, 64 bits, 64-bit time_t
> >>
> >> Buffer ioctls:
> >>             warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not
> >> supported
> >>             warn: v4l2-test-buffers.cpp(693): VIDIOC_CREATE_BUFS not
> >> supported
> >>     test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> >>     test VIDIOC_EXPBUF: OK
> >>     test Requests: OK (Not Supported)
> >>
> >> Total for wave5-dec device /dev/video0: 45, Succeeded: 45, Failed: 0,
> >> Warnings: 2 Total for wave5-enc device /dev/video1: 45, Succeeded:
> >> 45,
> >> Failed: 0, Warnings: 0
> >>
> >> Fluster test results:
> >> =====================
> >>
> >> Running test suite JCT-VC-HEVC_V1 with decoder
> >> GStreamer-H.265-V4L2-Gst1.0 Using 1 parallel job(s)
> >> Ran 132/147 tests successfully               in 94.782 secs
> >>
> >> (1 test fails because of not supporting to parse multi frames, 1 test
> >> fails because of a missing frame and slight corruption, 2 tests fail
> >> because of sizes which are incompatible with the IP, 11 tests fail
> >> because of unsupported 10 bit format)
> >>
> >> Running test suite JVT-AVC_V1 with decoder
> >> GStreamer-H.264-V4L2-Gst1.0 Using 1 parallel job(s)
> >> Ran 77/135 tests successfully               in 37.364 secs
> >>
> >> (58 fail because the hardware is unable to decode  MBAFF / FMO /
> >> Field / Extended profile streams.)
> >>
> >>
> >> Chnage since v1:
> >> =================
> >>
> >> * For [PATCH v2 0/4] media: chips-media: wave5: Support SPS/PPS
> >> generation for each IDR
> >>  - define a macro for register addresses
> >>
> >> * For [PATCH v2 1/4] media: chips-media: wave5: Support runtime
> >> suspend/resume
> >>  - add auto suspend/resume
> >>
> >> * For [PATCH v2 2/4] media: chips-media: wave5: Use helpers to
> >> calculate bytesperline and sizeimage
> >>  - use helper functions to calculate bytesperline and sizeimage
> >>
> >> * For [PATCH v2 3/4] media: chips-media: wave5: Support YUV422 raw
> >> pixel- formats on the encoder
> >>  - remove unnecessary codes
> >>
> >> Change since v0:
> >> =================
> >> The DEFAULT_SRC_SIZE macro was defined using multiple lines, To make
> >> a simple define, tab and multiple lines has been removed, The macro
> >> is defined using one line.
> >>
> >> Jackson.lee (4):
> >>   media: chips-media: wave5: Support SPS/PPS generation for each IDR
> >>   media: chips-media: wave5: Support runtime suspend/resume
> >>   media: chips-media: wave5: Use helpers to calculate bytesperline and
> >>     sizeimage.
> >>   media: chips-media: wave5: Support YUV422 raw pixel-formats on the
> >>     encoder.
> >>
> >>  .../platform/chips-media/wave5/wave5-helper.c |  24 ++
> >>  .../platform/chips-media/wave5/wave5-helper.h |   4 +
> >>  .../platform/chips-media/wave5/wave5-hw.c     |  23 +-
> >>  .../chips-media/wave5/wave5-vpu-dec.c         | 261 +++++-------------
> >>  .../chips-media/wave5/wave5-vpu-enc.c         | 260 +++++++++--------
> >>  .../platform/chips-media/wave5/wave5-vpu.c    |  43 +++
> >>  .../platform/chips-media/wave5/wave5-vpu.h    |   4 -
> >>  .../platform/chips-media/wave5/wave5-vpuapi.c |  14 +-
> >>  .../platform/chips-media/wave5/wave5-vpuapi.h |   1 +
> >>  .../chips-media/wave5/wave5-vpuconfig.h       |  25 +-
> >>  .../media/platform/chips-media/wave5/wave5.h  |   3 +
> >>  11 files changed, 329 insertions(+), 333 deletions(-)
> >>
> >> --
> >> 2.43.0
> >

Re: [RESEND PATCH v2 1/4] media: chips-media: wave5: Support SPS/PPS generation for each IDR

[Nicolas Dufresne]

Le lundi 11 mars 2024 à 19:56 +0900, jackson.lee a écrit :
> From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> 
> Provide a control to toggle (0 = off / 1 = on), whether the SPS and
> PPS are generated for every IDR.
> 
> Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
> Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
> ---
>  .../platform/chips-media/wave5/wave5-hw.c     | 19 +++++++++++++++----
>  .../chips-media/wave5/wave5-vpu-enc.c         |  7 +++++++
>  .../platform/chips-media/wave5/wave5-vpuapi.h |  1 +
>  3 files changed, 23 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-hw.c b/drivers/media/platform/chips-media/wave5/wave5-hw.c
> index f1e022fb148e..4a262822bf17 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-hw.c
> +++ b/drivers/media/platform/chips-media/wave5/wave5-hw.c
> @@ -23,6 +23,15 @@
>  #define FEATURE_AVC_ENCODER		BIT(1)
>  #define FEATURE_HEVC_ENCODER		BIT(0)
>  
> +#define ENC_AVC_INTRA_IDR_PARAM_MASK	0x7ff
> +#define ENC_AVC_INTRA_PERIOD		6
> +#define ENC_AVC_IDR_PERIOD		17
> +#define ENC_AVC_FORCED_IDR_HEADER	28
> +
> +#define ENC_HEVC_INTRA_QP		3
> +#define ENC_HEVC_FORCED_IDR_HEADER	9
> +#define ENC_HEVC_INTRA_PERIOD		16
> +

Perhaps add the suffix _SHIFT to these macro ? I think it can easily be confused
with a default value otherwise.

nit: can happen later, since this is how things got merged in this driver, but
typically, registers offsets, shift and masks are usually consolidated in a
single reg header. It would be nice to consider cleaning this up in future
patches.

>  /* Decoder support fields */
>  #define FEATURE_AVC_DECODER		BIT(3)
>  #define FEATURE_HEVC_DECODER		BIT(2)
> @@ -1601,12 +1610,14 @@ int wave5_vpu_enc_init_seq(struct vpu_instance *inst)
>  
>  	if (inst->std == W_AVC_ENC)
>  		vpu_write_reg(inst->dev, W5_CMD_ENC_SEQ_INTRA_PARAM, p_param->intra_qp |
> -				((p_param->intra_period & 0x7ff) << 6) |
> -				((p_param->avc_idr_period & 0x7ff) << 17));
> +				((p_param->intra_period & ENC_AVC_INTRA_IDR_PARAM_MASK) << ENC_AVC_INTRA_PERIOD) |
> +				((p_param->avc_idr_period & ENC_AVC_INTRA_IDR_PARAM_MASK) << ENC_AVC_IDR_PERIOD) |
> +				(p_param->forced_idr_header_enable << ENC_AVC_FORCED_IDR_HEADER));
>  	else if (inst->std == W_HEVC_ENC)
>  		vpu_write_reg(inst->dev, W5_CMD_ENC_SEQ_INTRA_PARAM,
> -			      p_param->decoding_refresh_type | (p_param->intra_qp << 3) |
> -				(p_param->intra_period << 16));
> +			      p_param->decoding_refresh_type | (p_param->intra_qp << ENC_HEVC_INTRA_QP) |
> +			      (p_param->forced_idr_header_enable << ENC_HEVC_FORCED_IDR_HEADER) |
> +			      (p_param->intra_period << ENC_HEVC_INTRA_PERIOD));
>  
>  	reg_val = (p_param->rdo_skip << 2) |
>  		(p_param->lambda_scaling_enable << 3) |
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> index f29cfa3af94a..f04baa93a9b7 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> @@ -1061,6 +1061,9 @@ static int wave5_vpu_enc_s_ctrl(struct v4l2_ctrl *ctrl)
>  	case V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE:
>  		inst->enc_param.entropy_coding_mode = ctrl->val;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR:
> +		inst->enc_param.forced_idr_header_enable = ctrl->val;
> +		break;
>  	case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
>  		break;
>  	default:
> @@ -1219,6 +1222,7 @@ static void wave5_set_enc_openparam(struct enc_open_param *open_param,
>  		else
>  			open_param->wave_param.intra_refresh_arg = num_ctu_row;
>  	}
> +	open_param->wave_param.forced_idr_header_enable = input.forced_idr_header_enable;

in the long term, there is one too many abstraction in this driver, we should
remove that.

>  }
>  
>  static int initialize_sequence(struct vpu_instance *inst)
> @@ -1702,6 +1706,9 @@ static int wave5_vpu_open_enc(struct file *filp)
>  			  0, 1, 1, 0);
>  	v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave5_vpu_enc_ctrl_ops,
>  			  V4L2_CID_MIN_BUFFERS_FOR_OUTPUT, 1, 32, 1, 1);
> +	v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave5_vpu_enc_ctrl_ops,
> +			  V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR,
> +			  0, 1, 1, 0);
>  
>  	if (v4l2_ctrl_hdl->error) {
>  		ret = -ENODEV;
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
> index 352f6e904e50..3ad6118550ac 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
> +++ b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
> @@ -566,6 +566,7 @@ struct enc_wave_param {
>  	u32 lambda_scaling_enable: 1; /* enable lambda scaling using custom GOP */
>  	u32 transform8x8_enable: 1; /* enable 8x8 intra prediction and 8x8 transform */
>  	u32 mb_level_rc_enable: 1; /* enable MB-level rate control */
> +	u32 forced_idr_header_enable: 1; /* enable header encoding before IDR frame */
>  };
>  
>  struct enc_open_param {

With the suggested _SHIFT suffix (or equivalent) added.

Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Re: [RESEND PATCH v2 2/4] media: chips-media: wave5: Support runtime suspend/resume

[Nicolas Dufresne]

Le lundi 11 mars 2024 à 19:56 +0900, jackson.lee a écrit :
> From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> 
> For saving a power resource, we support runtime suspend/resume for an encoder/decoder.
> So our vpu module's power turns on only if an encoder/decoder is used.

As per guidelines, try to be more direct on what the patch do and make it less
looks like a discussion. Something like (I'm not sure it totally exact)

   Add support for runtime suspend/resume in the encoder and decoder. This is
   achieved by saving the VPU state and powering it off while the VPU idle.

> 
> Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
> Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
> ---
>  .../platform/chips-media/wave5/wave5-hw.c     |  4 +-
>  .../chips-media/wave5/wave5-vpu-dec.c         | 16 ++++++-
>  .../chips-media/wave5/wave5-vpu-enc.c         | 15 +++++++
>  .../platform/chips-media/wave5/wave5-vpu.c    | 43 +++++++++++++++++++
>  .../platform/chips-media/wave5/wave5-vpuapi.c | 14 ++++--
>  .../media/platform/chips-media/wave5/wave5.h  |  3 ++
>  6 files changed, 88 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-hw.c b/drivers/media/platform/chips-media/wave5/wave5-hw.c
> index 4a262822bf17..826b92b7b582 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-hw.c
> +++ b/drivers/media/platform/chips-media/wave5/wave5-hw.c
> @@ -1084,8 +1084,8 @@ int wave5_vpu_re_init(struct device *dev, u8 *fw, size_t size)
>  	return setup_wave5_properties(dev);
>  }
>  
> -static int wave5_vpu_sleep_wake(struct device *dev, bool i_sleep_wake, const uint16_t *code,
> -				size_t size)
> +int wave5_vpu_sleep_wake(struct device *dev, bool i_sleep_wake, const uint16_t *code,
> +			 size_t size)
>  {
>  	u32 reg_val;
>  	struct vpu_buf *common_vb;
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> index ef227af72348..a199877c643b 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> @@ -5,6 +5,7 @@
>   * Copyright (C) 2021-2023 CHIPS&MEDIA INC
>   */
>  
> +#include <linux/pm_runtime.h>
>  #include "wave5-helper.h"
>  
>  #define VPU_DEC_DEV_NAME "C&M Wave5 VPU decoder"
> @@ -518,6 +519,8 @@ static void wave5_vpu_dec_finish_decode(struct vpu_instance *inst)
>  	if (q_status.report_queue_count == 0 &&
>  	    (q_status.instance_queue_count == 0 || dec_info.sequence_changed)) {
>  		dev_dbg(inst->dev->dev, "%s: finishing job.\n", __func__);
> +		pm_runtime_mark_last_busy(inst->dev->dev);
> +		pm_runtime_put_autosuspend(inst->dev->dev);
>  		v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
>  	}
>  }
> @@ -1382,6 +1385,7 @@ static int wave5_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count
>  	int ret = 0;
>  
>  	dev_dbg(inst->dev->dev, "%s: type: %u\n", __func__, q->type);
> +	pm_runtime_resume_and_get(inst->dev->dev);
>  
>  	v4l2_m2m_update_start_streaming_state(m2m_ctx, q);
>  
> @@ -1425,13 +1429,15 @@ static int wave5_vpu_dec_start_streaming(struct vb2_queue *q, unsigned int count
>  			}
>  		}
>  	}
> -
> +	pm_runtime_mark_last_busy(inst->dev->dev);
> +	pm_runtime_put_autosuspend(inst->dev->dev);
>  	return ret;
>  
>  free_bitstream_vbuf:
>  	wave5_vdi_free_dma_memory(inst->dev, &inst->bitstream_vbuf);
>  return_buffers:
>  	wave5_return_bufs(q, VB2_BUF_STATE_QUEUED);
> +	pm_runtime_put_autosuspend(inst->dev->dev);
>  	return ret;
>  }
>  
> @@ -1517,6 +1523,7 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)
>  	bool check_cmd = TRUE;
>  
>  	dev_dbg(inst->dev->dev, "%s: type: %u\n", __func__, q->type);
> +	pm_runtime_resume_and_get(inst->dev->dev);
>  
>  	while (check_cmd) {
>  		struct queue_status_info q_status;
> @@ -1540,6 +1547,9 @@ static void wave5_vpu_dec_stop_streaming(struct vb2_queue *q)
>  		streamoff_output(q);
>  	else
>  		streamoff_capture(q);
> +
> +	pm_runtime_mark_last_busy(inst->dev->dev);
> +	pm_runtime_put_autosuspend(inst->dev->dev);
>  }
>  
>  static const struct vb2_ops wave5_vpu_dec_vb2_ops = {
> @@ -1626,7 +1636,7 @@ static void wave5_vpu_dec_device_run(void *priv)
>  	int ret = 0;
>  
>  	dev_dbg(inst->dev->dev, "%s: Fill the ring buffer with new bitstream data", __func__);
> -
> +	pm_runtime_resume_and_get(inst->dev->dev);
>  	ret = fill_ringbuffer(inst);
>  	if (ret) {
>  		dev_warn(inst->dev->dev, "Filling ring buffer failed\n");
> @@ -1709,6 +1719,8 @@ static void wave5_vpu_dec_device_run(void *priv)
>  
>  finish_job_and_return:
>  	dev_dbg(inst->dev->dev, "%s: leave and finish job", __func__);
> +	pm_runtime_mark_last_busy(inst->dev->dev);
> +	pm_runtime_put_autosuspend(inst->dev->dev);
>  	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
>  }
>  
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> index f04baa93a9b7..013e2bb37fbb 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> @@ -5,6 +5,7 @@
>   * Copyright (C) 2021-2023 CHIPS&MEDIA INC
>   */
>  
> +#include <linux/pm_runtime.h>
>  #include "wave5-helper.h"
>  
>  #define VPU_ENC_DEV_NAME "C&M Wave5 VPU encoder"
> @@ -1310,6 +1311,7 @@ static int wave5_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int count
>  	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
>  	int ret = 0;
>  
> +	pm_runtime_resume_and_get(inst->dev->dev);
>  	v4l2_m2m_update_start_streaming_state(m2m_ctx, q);
>  
>  	if (inst->state == VPU_INST_STATE_NONE && q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> @@ -1364,9 +1366,13 @@ static int wave5_vpu_enc_start_streaming(struct vb2_queue *q, unsigned int count
>  	if (ret)
>  		goto return_buffers;
>  
> +	pm_runtime_mark_last_busy(inst->dev->dev);
> +	pm_runtime_put_autosuspend(inst->dev->dev);
>  	return 0;
>  return_buffers:
>  	wave5_return_bufs(q, VB2_BUF_STATE_QUEUED);
> +	pm_runtime_mark_last_busy(inst->dev->dev);
> +	pm_runtime_put_autosuspend(inst->dev->dev);
>  	return ret;
>  }
>  
> @@ -1408,6 +1414,7 @@ static void wave5_vpu_enc_stop_streaming(struct vb2_queue *q)
>  	 */
>  
>  	dev_dbg(inst->dev->dev, "%s: type: %u\n", __func__, q->type);
> +	pm_runtime_resume_and_get(inst->dev->dev);
>  
>  	if (wave5_vpu_both_queues_are_streaming(inst))
>  		switch_state(inst, VPU_INST_STATE_STOP);
> @@ -1432,6 +1439,9 @@ static void wave5_vpu_enc_stop_streaming(struct vb2_queue *q)
>  		streamoff_output(inst, q);
>  	else
>  		streamoff_capture(inst, q);
> +
> +	pm_runtime_mark_last_busy(inst->dev->dev);
> +	pm_runtime_put_autosuspend(inst->dev->dev);
>  }
>  
>  static const struct vb2_ops wave5_vpu_enc_vb2_ops = {
> @@ -1478,6 +1488,7 @@ static void wave5_vpu_enc_device_run(void *priv)
>  	u32 fail_res = 0;
>  	int ret = 0;
>  
> +	pm_runtime_resume_and_get(inst->dev->dev);
>  	switch (inst->state) {
>  	case VPU_INST_STATE_PIC_RUN:
>  		ret = start_encode(inst, &fail_res);
> @@ -1491,6 +1502,8 @@ static void wave5_vpu_enc_device_run(void *priv)
>  			break;
>  		}
>  		dev_dbg(inst->dev->dev, "%s: leave with active job", __func__);
> +		pm_runtime_mark_last_busy(inst->dev->dev);
> +		pm_runtime_put_autosuspend(inst->dev->dev);
>  		return;
>  	default:
>  		WARN(1, "Execution of a job in state %s is invalid.\n",
> @@ -1498,6 +1511,8 @@ static void wave5_vpu_enc_device_run(void *priv)
>  		break;
>  	}
>  	dev_dbg(inst->dev->dev, "%s: leave and finish job", __func__);
> +	pm_runtime_mark_last_busy(inst->dev->dev);
> +	pm_runtime_put_autosuspend(inst->dev->dev);
>  	v4l2_m2m_job_finish(inst->v4l2_m2m_dev, m2m_ctx);
>  }
>  
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu.c b/drivers/media/platform/chips-media/wave5/wave5-vpu.c
> index 0d90b5820bef..8e08461b3515 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-vpu.c
> +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu.c
> @@ -10,6 +10,7 @@
>  #include <linux/clk.h>
>  #include <linux/firmware.h>
>  #include <linux/interrupt.h>
> +#include <linux/pm_runtime.h>
>  #include "wave5-vpu.h"
>  #include "wave5-regdefine.h"
>  #include "wave5-vpuconfig.h"
> @@ -117,6 +118,38 @@ static int wave5_vpu_load_firmware(struct device *dev, const char *fw_name,
>  	return 0;
>  }
>  
> +static int wave5_pm_suspend(struct device *dev)
> +{
> +	struct vpu_device *vpu = dev_get_drvdata(dev);
> +
> +	if (pm_runtime_suspended(dev))
> +		return 0;
> +
> +	wave5_vpu_sleep_wake(dev, true, NULL, 0);
> +	clk_bulk_disable_unprepare(vpu->num_clks, vpu->clks);
> +
> +	return 0;
> +}
> +
> +static int wave5_pm_resume(struct device *dev)
> +{
> +	struct vpu_device *vpu = dev_get_drvdata(dev);
> +	int ret = 0;
> +
> +	wave5_vpu_sleep_wake(dev, false, NULL, 0);
> +	ret = clk_bulk_prepare_enable(vpu->num_clks, vpu->clks);
> +	if (ret) {
> +		dev_err(dev, "Enabling clocks, fail: %d\n", ret);
> +		return ret;
> +	}
> +
> +	return ret;
> +}
> +
> +static const struct dev_pm_ops wave5_pm_ops = {
> +	SET_RUNTIME_PM_OPS(wave5_pm_suspend, wave5_pm_resume, NULL)
> +};
> +
>  static int wave5_vpu_probe(struct platform_device *pdev)
>  {
>  	int ret;
> @@ -232,6 +265,12 @@ static int wave5_vpu_probe(struct platform_device *pdev)
>  		 (match_data->flags & WAVE5_IS_DEC) ? "'DECODE'" : "");
>  	dev_info(&pdev->dev, "Product Code:      0x%x\n", dev->product_code);
>  	dev_info(&pdev->dev, "Firmware Revision: %u\n", fw_revision);
> +
> +	pm_runtime_set_autosuspend_delay(&pdev->dev, 5000);
> +	pm_runtime_use_autosuspend(&pdev->dev);
> +	pm_runtime_enable(&pdev->dev);
> +	wave5_vpu_sleep_wake(&pdev->dev, true, NULL, 0);
> +
>  	return 0;
>  
>  err_enc_unreg:
> @@ -254,6 +293,9 @@ static int wave5_vpu_remove(struct platform_device *pdev)
>  {
>  	struct vpu_device *dev = dev_get_drvdata(&pdev->dev);
>  
> +	pm_runtime_put_sync(&pdev->dev);
> +	pm_runtime_disable(&pdev->dev);
> +
>  	mutex_destroy(&dev->dev_lock);
>  	mutex_destroy(&dev->hw_lock);
>  	clk_bulk_disable_unprepare(dev->num_clks, dev->clks);
> @@ -281,6 +323,7 @@ static struct platform_driver wave5_vpu_driver = {
>  	.driver = {
>  		.name = VPU_PLATFORM_DEVICE_NAME,
>  		.of_match_table = of_match_ptr(wave5_dt_ids),
> +		.pm = &wave5_pm_ops,
>  		},
>  	.probe = wave5_vpu_probe,
>  	.remove = wave5_vpu_remove,
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
> index 1a3efb638dde..b0911fef232f 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
> +++ b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
> @@ -6,6 +6,8 @@
>   */
>  
>  #include <linux/bug.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/delay.h>
>  #include "wave5-vpuapi.h"
>  #include "wave5-regdefine.h"
>  #include "wave5.h"
> @@ -200,9 +202,13 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
>  	if (!inst->codec_info)
>  		return -EINVAL;
>  
> +	pm_runtime_resume_and_get(inst->dev->dev);
> +
>  	ret = mutex_lock_interruptible(&vpu_dev->hw_lock);
> -	if (ret)
> +	if (ret) {
> +		pm_runtime_put_sync(inst->dev->dev);
>  		return ret;
> +	}
>  
>  	do {
>  		ret = wave5_vpu_dec_finish_seq(inst, fail_res);
> @@ -234,7 +240,7 @@ int wave5_vpu_dec_close(struct vpu_instance *inst, u32 *fail_res)
>  
>  unlock_and_return:
>  	mutex_unlock(&vpu_dev->hw_lock);
> -
> +	pm_runtime_put_sync(inst->dev->dev);
>  	return ret;
>  }
>  
> @@ -702,6 +708,8 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
>  	if (!inst->codec_info)
>  		return -EINVAL;
>  
> +	pm_runtime_resume_and_get(inst->dev->dev);
> +
>  	ret = mutex_lock_interruptible(&vpu_dev->hw_lock);
>  	if (ret)
>  		return ret;
> @@ -733,9 +741,9 @@ int wave5_vpu_enc_close(struct vpu_instance *inst, u32 *fail_res)
>  	}
>  
>  	wave5_vdi_free_dma_memory(vpu_dev, &p_enc_info->vb_task);
> -
>  	mutex_unlock(&vpu_dev->hw_lock);
>  
> +	pm_runtime_put_sync(inst->dev->dev);
>  	return 0;
>  }
>  
> diff --git a/drivers/media/platform/chips-media/wave5/wave5.h b/drivers/media/platform/chips-media/wave5/wave5.h
> index 063028eccd3b..6125eff938a8 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5.h
> +++ b/drivers/media/platform/chips-media/wave5/wave5.h
> @@ -56,6 +56,9 @@ int wave5_vpu_get_version(struct vpu_device *vpu_dev, u32 *revision);
>  
>  int wave5_vpu_init(struct device *dev, u8 *fw, size_t size);
>  
> +int wave5_vpu_sleep_wake(struct device *dev, bool i_sleep_wake, const uint16_t *code,
> +			 size_t size);
> +
>  int wave5_vpu_reset(struct device *dev, enum sw_reset_mode reset_mode);
>  
>  int wave5_vpu_build_up_dec_param(struct vpu_instance *inst, struct dec_open_param *param);


With a better commit message:

Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Re: [RESEND PATCH v2 3/4] media: chips-media: wave5: Use helpers to calculate bytesperline and sizeimage.

[Nicolas Dufresne]

Hi,

very nice cleanup.

Le lundi 11 mars 2024 à 19:56 +0900, jackson.lee a écrit :
> From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> 
> Use v4l2-common helper functions to calculate bytesperline and sizeimage, instead of calculating in a wave5 driver directly.
> In case of raw(YUV) v4l2_pix_format, the wave5 driver updates v4l2_pix_format_mplane struct through v4l2_fill_pixfmt_mp() function.
> Encoder and Decoder need same bytesperline and sizeimage values for same v4l2_pix_format.
> So, a wave5_update_pix_fmt is refactored to support both together.
> 
> Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
> Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
> ---
>  .../platform/chips-media/wave5/wave5-helper.c |  24 ++
>  .../platform/chips-media/wave5/wave5-helper.h |   4 +
>  .../chips-media/wave5/wave5-vpu-dec.c         | 245 ++++--------------
>  .../chips-media/wave5/wave5-vpu-enc.c         | 179 +++++--------
>  .../platform/chips-media/wave5/wave5-vpu.h    |   4 -
>  .../chips-media/wave5/wave5-vpuconfig.h       |  25 +-
>  6 files changed, 164 insertions(+), 317 deletions(-)
> 
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-helper.c b/drivers/media/platform/chips-media/wave5/wave5-helper.c
> index 8433ecab230c..53cad4d17aa7 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-helper.c
> +++ b/drivers/media/platform/chips-media/wave5/wave5-helper.c
> @@ -7,6 +7,8 @@
>  
>  #include "wave5-helper.h"
>  
> +#define DEFAULT_BS_SIZE(width, height) ((width) * (height) / 8 * 3)
> +
>  const char *state_to_str(enum vpu_instance_state state)
>  {
>  	switch (state) {
> @@ -211,3 +213,25 @@ void wave5_return_bufs(struct vb2_queue *q, u32 state)
>  		v4l2_m2m_buf_done(vbuf, state);
>  	}
>  }
> +
> +void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
> +			  unsigned int width,
> +			  unsigned int height,
> +			  const struct v4l2_frmsize_stepwise frmsize)
> +{
> +	v4l2_apply_frmsize_constraints(&width, &height, &frmsize);
> +
> +	if (pix_mp->pixelformat == V4L2_PIX_FMT_HEVC ||
> +	    pix_mp->pixelformat == V4L2_PIX_FMT_H264) {

I notice the caller already index the frmsize between RAW and CODEC. I'd say
pass the information as an argument so we don't have to update code later when
more codecs are added.

> +		pix_mp->width = width;
> +		pix_mp->height = height;
> +		pix_mp->num_planes = 1;
> +		pix_mp->plane_fmt[0].bytesperline = 0;
> +		pix_mp->plane_fmt[0].sizeimage = max(DEFAULT_BS_SIZE(width, height),
> +						     pix_mp->plane_fmt[0].sizeimage);
> +	} else {
> +		v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat, width, height);
> +	}
> +	pix_mp->flags = 0;
> +	pix_mp->field = V4L2_FIELD_NONE;
> +}
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-helper.h b/drivers/media/platform/chips-media/wave5/wave5-helper.h
> index 6cee1c14d3ce..a8ecd1920207 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-helper.h
> +++ b/drivers/media/platform/chips-media/wave5/wave5-helper.h
> @@ -28,4 +28,8 @@ const struct vpu_format *wave5_find_vpu_fmt_by_idx(unsigned int idx,
>  						   const struct vpu_format fmt_list[MAX_FMTS]);
>  enum wave_std wave5_to_vpu_std(unsigned int v4l2_pix_fmt, enum vpu_instance_type type);
>  void wave5_return_bufs(struct vb2_queue *q, u32 state);
> +void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
> +			  unsigned int width,
> +			  unsigned int height,
> +			  const struct v4l2_frmsize_stepwise frmsize);
>  #endif
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> index a199877c643b..43d0a21193dd 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> @@ -11,115 +11,74 @@
>  #define VPU_DEC_DEV_NAME "C&M Wave5 VPU decoder"
>  #define VPU_DEC_DRV_NAME "wave5-dec"
>  
> -#define DEFAULT_SRC_SIZE(width, height) ({			\
> -	(width) * (height) / 8 * 3;					\
> -})
> -
>  static const struct vpu_format dec_fmt_list[FMT_TYPES][MAX_FMTS] = {
>  	[VPU_FMT_TYPE_CODEC] = {
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_H264,
> -			.max_width = 8192,
> -			.min_width = 32,
> -			.max_height = 4320,
> -			.min_height = 32,
>  		},

I notice that min/max was different per codec, but in this patch you actually
make them both identical. Is that an error, otherwise the change should be
underlined in the commit message.

If you really need per codec granularity, you could define the common frmsize
array before, and set a pointer here into the static array. This way you won't
have to type the same thing over and over, but still have per codec granularity.

>  	},
>  	[VPU_FMT_TYPE_RAW] = {
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422P,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422M,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16M,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61M,
> -			.max_width = 8192,
> -			.min_width = 8,
> -			.max_height = 4320,
> -			.min_height = 8,
>  		},
>  	}
>  };
>  
> +static const struct v4l2_frmsize_stepwise dec_frmsize[FMT_TYPES] = {
> +	[VPU_FMT_TYPE_CODEC] = {
> +		.min_width = W5_MIN_DEC_PIC_WIDTH,
> +		.max_width = W5_MAX_DEC_PIC_WIDTH,
> +		.step_width = W5_DEC_CODEC_STEP_WIDTH,
> +		.min_height = W5_MIN_DEC_PIC_HEIGHT,
> +		.max_height = W5_MAX_DEC_PIC_HEIGHT,
> +		.step_height = W5_DEC_CODEC_STEP_HEIGHT,
> +	},
> +	[VPU_FMT_TYPE_RAW] = {
> +		.min_width = W5_MIN_DEC_PIC_WIDTH,
> +		.max_width = W5_MAX_DEC_PIC_WIDTH,
> +		.step_width = W5_DEC_RAW_STEP_WIDTH,
> +		.min_height = W5_MIN_DEC_PIC_HEIGHT,
> +		.max_height = W5_MAX_DEC_PIC_HEIGHT,
> +		.step_height = W5_DEC_RAW_STEP_HEIGHT,
> +	},
> +};
> +
>  /*
>   * Make sure that the state switch is allowed and add logging for debugging
>   * purposes
> @@ -234,74 +193,6 @@ static void wave5_handle_src_buffer(struct vpu_instance *inst, dma_addr_t rd_ptr
>  	inst->remaining_consumed_bytes = consumed_bytes;
>  }
>  
> -static void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp, unsigned int width,
> -				 unsigned int height)
> -{
> -	switch (pix_mp->pixelformat) {
> -	case V4L2_PIX_FMT_YUV420:
> -	case V4L2_PIX_FMT_NV12:
> -	case V4L2_PIX_FMT_NV21:
> -		pix_mp->width = round_up(width, 32);
> -		pix_mp->height = round_up(height, 16);
> -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[0].sizeimage = width * height * 3 / 2;
> -		break;
> -	case V4L2_PIX_FMT_YUV422P:
> -	case V4L2_PIX_FMT_NV16:
> -	case V4L2_PIX_FMT_NV61:
> -		pix_mp->width = round_up(width, 32);
> -		pix_mp->height = round_up(height, 16);
> -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[0].sizeimage = width * height * 2;
> -		break;
> -	case V4L2_PIX_FMT_YUV420M:
> -		pix_mp->width = round_up(width, 32);
> -		pix_mp->height = round_up(height, 16);
> -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[0].sizeimage = width * height;
> -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
> -		pix_mp->plane_fmt[1].sizeimage = width * height / 4;
> -		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
> -		pix_mp->plane_fmt[2].sizeimage = width * height / 4;
> -		break;
> -	case V4L2_PIX_FMT_NV12M:
> -	case V4L2_PIX_FMT_NV21M:
> -		pix_mp->width = round_up(width, 32);
> -		pix_mp->height = round_up(height, 16);
> -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[0].sizeimage = width * height;
> -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[1].sizeimage = width * height / 2;
> -		break;
> -	case V4L2_PIX_FMT_YUV422M:
> -		pix_mp->width = round_up(width, 32);
> -		pix_mp->height = round_up(height, 16);
> -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[0].sizeimage = width * height;
> -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
> -		pix_mp->plane_fmt[1].sizeimage = width * height / 2;
> -		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
> -		pix_mp->plane_fmt[2].sizeimage = width * height / 2;
> -		break;
> -	case V4L2_PIX_FMT_NV16M:
> -	case V4L2_PIX_FMT_NV61M:
> -		pix_mp->width = round_up(width, 32);
> -		pix_mp->height = round_up(height, 16);
> -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[0].sizeimage = width * height;
> -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[1].sizeimage = width * height;
> -		break;
> -	default:
> -		pix_mp->width = width;
> -		pix_mp->height = height;
> -		pix_mp->plane_fmt[0].bytesperline = 0;
> -		pix_mp->plane_fmt[0].sizeimage = max(DEFAULT_SRC_SIZE(width, height),
> -						     pix_mp->plane_fmt[0].sizeimage);
> -		break;
> -	}
> -}
> -
>  static int start_decode(struct vpu_instance *inst, u32 *fail_res)
>  {
>  	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
> @@ -397,9 +288,11 @@ static int handle_dynamic_resolution_change(struct vpu_instance *inst)
>  			initial_info->pic_crop_rect.top - initial_info->pic_crop_rect.bottom;
>  
>  		wave5_update_pix_fmt(&inst->src_fmt, initial_info->pic_width,
> -				     initial_info->pic_height);
> +						     initial_info->pic_height,
> +						     dec_frmsize[VPU_FMT_TYPE_CODEC]);
>  		wave5_update_pix_fmt(&inst->dst_fmt, initial_info->pic_width,
> -				     initial_info->pic_height);
> +						     initial_info->pic_height,
> +						     dec_frmsize[VPU_FMT_TYPE_RAW]);
>  	}
>  
>  	v4l2_event_queue_fh(fh, &vpu_event_src_ch);
> @@ -548,12 +441,7 @@ static int wave5_vpu_dec_enum_framesizes(struct file *f, void *fh, struct v4l2_f
>  	}
>  
>  	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> -	fsize->stepwise.min_width = vpu_fmt->min_width;
> -	fsize->stepwise.max_width = vpu_fmt->max_width;
> -	fsize->stepwise.step_width = 1;
> -	fsize->stepwise.min_height = vpu_fmt->min_height;
> -	fsize->stepwise.max_height = vpu_fmt->max_height;
> -	fsize->stepwise.step_height = 1;
> +	fsize->stepwise = dec_frmsize[VPU_FMT_TYPE_CODEC];
>  
>  	return 0;
>  }
> @@ -589,14 +477,10 @@ static int wave5_vpu_dec_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
>  		width = inst->dst_fmt.width;
>  		height = inst->dst_fmt.height;
>  		f->fmt.pix_mp.pixelformat = inst->dst_fmt.pixelformat;
> -		f->fmt.pix_mp.num_planes = inst->dst_fmt.num_planes;
>  	} else {
> -		const struct v4l2_format_info *info = v4l2_format_info(vpu_fmt->v4l2_pix_fmt);
> -
> -		width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width, vpu_fmt->max_width);
> -		height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height, vpu_fmt->max_height);
> +		width = f->fmt.pix_mp.width;
> +		height = f->fmt.pix_mp.height;
>  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> -		f->fmt.pix_mp.num_planes = info->mem_planes;
>  	}
>  
>  	if (p_dec_info->initial_info_obtained) {
> @@ -604,9 +488,9 @@ static int wave5_vpu_dec_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
>  		height = inst->dst_fmt.height;
>  	}
>  
> -	wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
> -	f->fmt.pix_mp.flags = 0;
> -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> +					     height,
> +					     dec_frmsize[VPU_FMT_TYPE_RAW]);
>  	f->fmt.pix_mp.colorspace = inst->colorspace;
>  	f->fmt.pix_mp.ycbcr_enc = inst->ycbcr_enc;
>  	f->fmt.pix_mp.quantization = inst->quantization;
> @@ -719,6 +603,7 @@ static int wave5_vpu_dec_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
>  {
>  	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
>  	const struct vpu_format *vpu_fmt;
> +	int width, height;
>  
>  	dev_dbg(inst->dev->dev,
>  		"%s: fourcc: %u width: %u height: %u num_planes: %u colorspace: %u field: %u\n",
> @@ -727,20 +612,18 @@ static int wave5_vpu_dec_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
>  
>  	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat, dec_fmt_list[VPU_FMT_TYPE_CODEC]);
>  	if (!vpu_fmt) {
> +		width = inst->src_fmt.width;
> +		height = inst->src_fmt.height;
>  		f->fmt.pix_mp.pixelformat = inst->src_fmt.pixelformat;
> -		f->fmt.pix_mp.num_planes = inst->src_fmt.num_planes;
> -		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->src_fmt.width, inst->src_fmt.height);
>  	} else {
> -		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width, vpu_fmt->max_width);
> -		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height, vpu_fmt->max_height);
> -
> +		width = f->fmt.pix_mp.width;
> +		height = f->fmt.pix_mp.height;
>  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> -		f->fmt.pix_mp.num_planes = 1;
> -		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
>  	}
>  
> -	f->fmt.pix_mp.flags = 0;
> -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> +					     height,
> +					     dec_frmsize[VPU_FMT_TYPE_CODEC]);
>  
>  	return 0;
>  }
> @@ -782,7 +665,9 @@ static int wave5_vpu_dec_s_fmt_out(struct file *file, void *fh, struct v4l2_form
>  	inst->quantization = f->fmt.pix_mp.quantization;
>  	inst->xfer_func = f->fmt.pix_mp.xfer_func;
>  
> -	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width, f->fmt.pix_mp.height);
> +	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width,
> +					     f->fmt.pix_mp.height,
> +					     dec_frmsize[VPU_FMT_TYPE_RAW]);
>  
>  	return 0;
>  }
> @@ -1005,6 +890,7 @@ static int wave5_vpu_dec_queue_setup(struct vb2_queue *q, unsigned int *num_buff
>  	struct vpu_instance *inst = vb2_get_drv_priv(q);
>  	struct v4l2_pix_format_mplane inst_format =
>  		(q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) ? inst->src_fmt : inst->dst_fmt;
> +	unsigned int i;
>  
>  	dev_dbg(inst->dev->dev, "%s: num_buffers: %u | num_planes: %u | type: %u\n", __func__,
>  		*num_buffers, *num_planes, q->type);
> @@ -1018,31 +904,9 @@ static int wave5_vpu_dec_queue_setup(struct vb2_queue *q, unsigned int *num_buff
>  		if (*num_buffers < inst->fbc_buf_count)
>  			*num_buffers = inst->fbc_buf_count;
>  
> -		if (*num_planes == 1) {
> -			if (inst->output_format == FORMAT_422)
> -				sizes[0] = inst_format.width * inst_format.height * 2;
> -			else
> -				sizes[0] = inst_format.width * inst_format.height * 3 / 2;
> -			dev_dbg(inst->dev->dev, "%s: size[0]: %u\n", __func__, sizes[0]);
> -		} else if (*num_planes == 2) {
> -			sizes[0] = inst_format.width * inst_format.height;
> -			if (inst->output_format == FORMAT_422)
> -				sizes[1] = inst_format.width * inst_format.height;
> -			else
> -				sizes[1] = inst_format.width * inst_format.height / 2;
> -			dev_dbg(inst->dev->dev, "%s: size[0]: %u | size[1]: %u\n",
> -				__func__, sizes[0], sizes[1]);
> -		} else if (*num_planes == 3) {
> -			sizes[0] = inst_format.width * inst_format.height;
> -			if (inst->output_format == FORMAT_422) {
> -				sizes[1] = inst_format.width * inst_format.height / 2;
> -				sizes[2] = inst_format.width * inst_format.height / 2;
> -			} else {
> -				sizes[1] = inst_format.width * inst_format.height / 4;
> -				sizes[2] = inst_format.width * inst_format.height / 4;
> -			}
> -			dev_dbg(inst->dev->dev, "%s: size[0]: %u | size[1]: %u | size[2]: %u\n",
> -				__func__, sizes[0], sizes[1], sizes[2]);
> +		for (i = 0; i < *num_planes; i++) {
> +			sizes[i] = inst_format.plane_fmt[i].sizeimage;
> +			dev_dbg(inst->dev->dev, "%s: size[%u]: %u\n", __func__, i, sizes[i]);
>  		}
>  	}
>  
> @@ -1564,20 +1428,15 @@ static const struct vb2_ops wave5_vpu_dec_vb2_ops = {
>  static void wave5_set_default_format(struct v4l2_pix_format_mplane *src_fmt,
>  				     struct v4l2_pix_format_mplane *dst_fmt)
>  {
> -	unsigned int dst_pix_fmt = dec_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> -	const struct v4l2_format_info *dst_fmt_info = v4l2_format_info(dst_pix_fmt);
> -
>  	src_fmt->pixelformat = dec_fmt_list[VPU_FMT_TYPE_CODEC][0].v4l2_pix_fmt;
> -	src_fmt->field = V4L2_FIELD_NONE;
> -	src_fmt->flags = 0;
> -	src_fmt->num_planes = 1;
> -	wave5_update_pix_fmt(src_fmt, 720, 480);
> -
> -	dst_fmt->pixelformat = dst_pix_fmt;
> -	dst_fmt->field = V4L2_FIELD_NONE;
> -	dst_fmt->flags = 0;
> -	dst_fmt->num_planes = dst_fmt_info->mem_planes;
> -	wave5_update_pix_fmt(dst_fmt, 736, 480);
> +	wave5_update_pix_fmt(src_fmt, W5_DEF_DEC_PIC_WIDTH,
> +				      W5_DEF_DEC_PIC_HEIGHT,
> +				      dec_frmsize[VPU_FMT_TYPE_CODEC]);
> +
> +	dst_fmt->pixelformat = dec_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> +	wave5_update_pix_fmt(dst_fmt, W5_DEF_DEC_PIC_WIDTH,
> +				      W5_DEF_DEC_PIC_HEIGHT,
> +				      dec_frmsize[VPU_FMT_TYPE_RAW]);
>  }
>  
>  static int wave5_vpu_dec_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> index 013e2bb37fbb..5a32bb138158 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> @@ -15,65 +15,52 @@ static const struct vpu_format enc_fmt_list[FMT_TYPES][MAX_FMTS] = {
>  	[VPU_FMT_TYPE_CODEC] = {
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
> -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_H264,
> -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
>  		},
>  	},
>  	[VPU_FMT_TYPE_RAW] = {
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
> -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
> -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
> -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
> -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
> -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
>  		},
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
> -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
>  		},
>  	}
>  };
>  
> +static const struct v4l2_frmsize_stepwise enc_frmsize[FMT_TYPES] = {
> +	[VPU_FMT_TYPE_CODEC] = {
> +		.min_width = W5_MIN_ENC_PIC_WIDTH,
> +		.max_width = W5_MAX_ENC_PIC_WIDTH,
> +		.step_width = W5_ENC_CODEC_STEP_WIDTH,
> +		.min_height = W5_MIN_ENC_PIC_HEIGHT,
> +		.max_height = W5_MAX_ENC_PIC_HEIGHT,
> +		.step_height = W5_ENC_CODEC_STEP_HEIGHT,
> +	},
> +	[VPU_FMT_TYPE_RAW] = {
> +		.min_width = W5_MIN_ENC_PIC_WIDTH,
> +		.max_width = W5_MAX_ENC_PIC_WIDTH,
> +		.step_width = W5_ENC_RAW_STEP_WIDTH,
> +		.min_height = W5_MIN_ENC_PIC_HEIGHT,
> +		.max_height = W5_MAX_ENC_PIC_HEIGHT,
> +		.step_height = W5_ENC_RAW_STEP_HEIGHT,
> +	},
> +};
> +
>  static int switch_state(struct vpu_instance *inst, enum vpu_instance_state state)
>  {
>  	switch (state) {
> @@ -106,46 +93,6 @@ static int switch_state(struct vpu_instance *inst, enum vpu_instance_state state
>  	return -EINVAL;
>  }
>  
> -static void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp, unsigned int width,
> -				 unsigned int height)
> -{
> -	switch (pix_mp->pixelformat) {
> -	case V4L2_PIX_FMT_YUV420:
> -	case V4L2_PIX_FMT_NV12:
> -	case V4L2_PIX_FMT_NV21:
> -		pix_mp->width = width;
> -		pix_mp->height = height;
> -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height * 3 / 2;
> -		break;
> -	case V4L2_PIX_FMT_YUV420M:
> -		pix_mp->width = width;
> -		pix_mp->height = height;
> -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height;
> -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
> -		pix_mp->plane_fmt[1].sizeimage = round_up(width, 32) * height / 4;
> -		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
> -		pix_mp->plane_fmt[2].sizeimage = round_up(width, 32) * height / 4;
> -		break;
> -	case V4L2_PIX_FMT_NV12M:
> -	case V4L2_PIX_FMT_NV21M:
> -		pix_mp->width = width;
> -		pix_mp->height = height;
> -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height;
> -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
> -		pix_mp->plane_fmt[1].sizeimage = round_up(width, 32) * height / 2;
> -		break;
> -	default:
> -		pix_mp->width = width;
> -		pix_mp->height = height;
> -		pix_mp->plane_fmt[0].bytesperline = 0;
> -		pix_mp->plane_fmt[0].sizeimage = width * height / 8 * 3;
> -		break;
> -	}
> -}
> -
>  static int start_encode(struct vpu_instance *inst, u32 *fail_res)
>  {
>  	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx;
> @@ -360,13 +307,8 @@ static int wave5_vpu_enc_enum_framesizes(struct file *f, void *fh, struct v4l2_f
>  			return -EINVAL;
>  	}
>  
> -	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> -	fsize->stepwise.min_width = vpu_fmt->min_width;
> -	fsize->stepwise.max_width = vpu_fmt->max_width;
> -	fsize->stepwise.step_width = 1;
> -	fsize->stepwise.min_height = vpu_fmt->min_height;
> -	fsize->stepwise.max_height = vpu_fmt->max_height;
> -	fsize->stepwise.step_height = 1;
> +	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> +	fsize->stepwise = enc_frmsize[VPU_FMT_TYPE_CODEC];
>  
>  	return 0;
>  }
> @@ -392,6 +334,7 @@ static int wave5_vpu_enc_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
>  {
>  	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
>  	const struct vpu_format *vpu_fmt;
> +	int width, height;
>  
>  	dev_dbg(inst->dev->dev, "%s: fourcc: %u width: %u height: %u num_planes: %u field: %u\n",
>  		__func__, f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width, f->fmt.pix_mp.height,
> @@ -399,20 +342,18 @@ static int wave5_vpu_enc_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
>  
>  	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat, enc_fmt_list[VPU_FMT_TYPE_CODEC]);
>  	if (!vpu_fmt) {
> +		width = inst->dst_fmt.width;
> +		height = inst->dst_fmt.height;
>  		f->fmt.pix_mp.pixelformat = inst->dst_fmt.pixelformat;
> -		f->fmt.pix_mp.num_planes = inst->dst_fmt.num_planes;
> -		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->dst_fmt.width, inst->dst_fmt.height);
>  	} else {
> -		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width, vpu_fmt->max_width);
> -		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height, vpu_fmt->max_height);
> -
> +		width = f->fmt.pix_mp.width;
> +		height = f->fmt.pix_mp.height;
>  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> -		f->fmt.pix_mp.num_planes = 1;
> -		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
>  	}
>  
> -	f->fmt.pix_mp.flags = 0;
> -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> +					     height,
> +					     enc_frmsize[VPU_FMT_TYPE_CODEC]);
>  	f->fmt.pix_mp.colorspace = inst->colorspace;
>  	f->fmt.pix_mp.ycbcr_enc = inst->ycbcr_enc;
>  	f->fmt.pix_mp.quantization = inst->quantization;
> @@ -500,6 +441,7 @@ static int wave5_vpu_enc_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
>  {
>  	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
>  	const struct vpu_format *vpu_fmt;
> +	int width, height;
>  
>  	dev_dbg(inst->dev->dev, "%s: fourcc: %u width: %u height: %u num_planes: %u field: %u\n",
>  		__func__, f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width, f->fmt.pix_mp.height,
> @@ -507,21 +449,18 @@ static int wave5_vpu_enc_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
>  
>  	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat, enc_fmt_list[VPU_FMT_TYPE_RAW]);
>  	if (!vpu_fmt) {
> +		width = inst->src_fmt.width;
> +		height = inst->src_fmt.height;
>  		f->fmt.pix_mp.pixelformat = inst->src_fmt.pixelformat;
> -		f->fmt.pix_mp.num_planes = inst->src_fmt.num_planes;
> -		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->src_fmt.width, inst->src_fmt.height);
>  	} else {
> -		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width, vpu_fmt->max_width);
> -		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height, vpu_fmt->max_height);
> -		const struct v4l2_format_info *info = v4l2_format_info(vpu_fmt->v4l2_pix_fmt);
> -
> +		width = f->fmt.pix_mp.width;
> +		height = f->fmt.pix_mp.height;
>  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> -		f->fmt.pix_mp.num_planes = info->mem_planes;
> -		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
>  	}
>  
> -	f->fmt.pix_mp.flags = 0;
> -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> +					     height,
> +					     enc_frmsize[VPU_FMT_TYPE_RAW]);
>  
>  	return 0;
>  }
> @@ -568,7 +507,11 @@ static int wave5_vpu_enc_s_fmt_out(struct file *file, void *fh, struct v4l2_form
>  	inst->quantization = f->fmt.pix_mp.quantization;
>  	inst->xfer_func = f->fmt.pix_mp.xfer_func;
>  
> -	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width, f->fmt.pix_mp.height);
> +	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width,
> +					     f->fmt.pix_mp.height,
> +					     enc_frmsize[VPU_FMT_TYPE_CODEC]);
> +	inst->conf_win.width = inst->dst_fmt.width;
> +	inst->conf_win.height = inst->dst_fmt.height;
>  
>  	return 0;
>  }
> @@ -584,12 +527,17 @@ static int wave5_vpu_enc_g_selection(struct file *file, void *fh, struct v4l2_se
>  	switch (s->target) {
>  	case V4L2_SEL_TGT_CROP_DEFAULT:
>  	case V4L2_SEL_TGT_CROP_BOUNDS:
> -	case V4L2_SEL_TGT_CROP:
>  		s->r.left = 0;
>  		s->r.top = 0;
>  		s->r.width = inst->dst_fmt.width;
>  		s->r.height = inst->dst_fmt.height;
>  		break;
> +	case V4L2_SEL_TGT_CROP:
> +		s->r.left = 0;
> +		s->r.top = 0;
> +		s->r.width = inst->conf_win.width;
> +		s->r.height = inst->conf_win.height;
> +		break;
>  	default:
>  		return -EINVAL;
>  	}
> @@ -612,8 +560,10 @@ static int wave5_vpu_enc_s_selection(struct file *file, void *fh, struct v4l2_se
>  
>  	s->r.left = 0;
>  	s->r.top = 0;
> -	s->r.width = inst->src_fmt.width;
> -	s->r.height = inst->src_fmt.height;
> +	s->r.width = min(s->r.width, inst->dst_fmt.width);
> +	s->r.height = min(s->r.height, inst->dst_fmt.height);
> +
> +	inst->conf_win = s->r;
>  
>  	return 0;
>  }
> @@ -1151,8 +1101,8 @@ static void wave5_set_enc_openparam(struct enc_open_param *open_param,
>  	open_param->wave_param.lambda_scaling_enable = 1;
>  
>  	open_param->line_buf_int_en = true;
> -	open_param->pic_width = inst->dst_fmt.width;
> -	open_param->pic_height = inst->dst_fmt.height;
> +	open_param->pic_width = inst->conf_win.width;
> +	open_param->pic_height = inst->conf_win.height;
>  	open_param->frame_rate_info = inst->frame_rate;
>  	open_param->rc_enable = inst->rc_enable;
>  	if (inst->rc_enable) {
> @@ -1456,20 +1406,15 @@ static const struct vb2_ops wave5_vpu_enc_vb2_ops = {
>  static void wave5_set_default_format(struct v4l2_pix_format_mplane *src_fmt,
>  				     struct v4l2_pix_format_mplane *dst_fmt)
>  {
> -	unsigned int src_pix_fmt = enc_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> -	const struct v4l2_format_info *src_fmt_info = v4l2_format_info(src_pix_fmt);
> -
> -	src_fmt->pixelformat = src_pix_fmt;
> -	src_fmt->field = V4L2_FIELD_NONE;
> -	src_fmt->flags = 0;
> -	src_fmt->num_planes = src_fmt_info->mem_planes;
> -	wave5_update_pix_fmt(src_fmt, 416, 240);
> +	src_fmt->pixelformat = enc_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> +	wave5_update_pix_fmt(src_fmt, W5_DEF_ENC_PIC_WIDTH,
> +				      W5_DEF_ENC_PIC_HEIGHT,
> +				      enc_frmsize[VPU_FMT_TYPE_RAW]);
>  
>  	dst_fmt->pixelformat = enc_fmt_list[VPU_FMT_TYPE_CODEC][0].v4l2_pix_fmt;
> -	dst_fmt->field = V4L2_FIELD_NONE;
> -	dst_fmt->flags = 0;
> -	dst_fmt->num_planes = 1;
> -	wave5_update_pix_fmt(dst_fmt, 416, 240);
> +	wave5_update_pix_fmt(dst_fmt, W5_DEF_ENC_PIC_WIDTH,
> +				      W5_DEF_ENC_PIC_HEIGHT,
> +				      enc_frmsize[VPU_FMT_TYPE_CODEC]);
>  }
>  
>  static int wave5_vpu_enc_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
> @@ -1734,6 +1679,8 @@ static int wave5_vpu_open_enc(struct file *filp)
>  	v4l2_ctrl_handler_setup(v4l2_ctrl_hdl);
>  
>  	wave5_set_default_format(&inst->src_fmt, &inst->dst_fmt);
> +	inst->conf_win.width = inst->dst_fmt.width;
> +	inst->conf_win.height = inst->dst_fmt.height;
>  	inst->colorspace = V4L2_COLORSPACE_REC709;
>  	inst->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
>  	inst->quantization = V4L2_QUANTIZATION_DEFAULT;
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu.h b/drivers/media/platform/chips-media/wave5/wave5-vpu.h
> index 32b7fd3730b5..691d6341fcda 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-vpu.h
> +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu.h
> @@ -38,10 +38,6 @@ enum vpu_fmt_type {
>  
>  struct vpu_format {
>  	unsigned int v4l2_pix_fmt;
> -	unsigned int max_width;
> -	unsigned int min_width;
> -	unsigned int max_height;
> -	unsigned int min_height;
>  };
>  
>  static inline struct vpu_instance *wave5_to_vpu_inst(struct v4l2_fh *vfh)
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h b/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> index d9751eedb0f9..e6a34ae7084e 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> +++ b/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> @@ -30,10 +30,27 @@
>  
>  #define MAX_NUM_INSTANCE                32
>  
> -#define W5_MIN_ENC_PIC_WIDTH            256
> -#define W5_MIN_ENC_PIC_HEIGHT           128
> -#define W5_MAX_ENC_PIC_WIDTH            8192
> -#define W5_MAX_ENC_PIC_HEIGHT           8192
> +#define W5_DEF_DEC_PIC_WIDTH            720U
> +#define W5_DEF_DEC_PIC_HEIGHT           480U
> +#define W5_MIN_DEC_PIC_WIDTH            32U
> +#define W5_MIN_DEC_PIC_HEIGHT           32U
> +#define W5_MAX_DEC_PIC_WIDTH            8192U
> +#define W5_MAX_DEC_PIC_HEIGHT           4320U
> +#define W5_DEC_CODEC_STEP_WIDTH         1U
> +#define W5_DEC_CODEC_STEP_HEIGHT        1U
> +#define W5_DEC_RAW_STEP_WIDTH           32U
> +#define W5_DEC_RAW_STEP_HEIGHT          16U
> +
> +#define W5_DEF_ENC_PIC_WIDTH            416U
> +#define W5_DEF_ENC_PIC_HEIGHT           240U
> +#define W5_MIN_ENC_PIC_WIDTH            256U
> +#define W5_MIN_ENC_PIC_HEIGHT           128U
> +#define W5_MAX_ENC_PIC_WIDTH            8192U
> +#define W5_MAX_ENC_PIC_HEIGHT           8192U
> +#define W5_ENC_CODEC_STEP_WIDTH         8U
> +#define W5_ENC_CODEC_STEP_HEIGHT        8U
> +#define W5_ENC_RAW_STEP_WIDTH           32U
> +#define W5_ENC_RAW_STEP_HEIGHT          16U
>  
>  //  application specific configuration
>  #define VPU_ENC_TIMEOUT                 60000


I'll have to give this one a second pass on next revision, but nice work.

Nicolas

Re: [RESEND PATCH v2 4/4] media: chips-media: wave5: Support YUV422 raw pixel-formats on the encoder.

[Nicolas Dufresne]

Le lundi 11 mars 2024 à 19:56 +0900, jackson.lee a écrit :
> From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> 
> Add support for the YUV422P, NV16, NV61, YUV422M, NV16M, NV61M raw pixel-formats to the Wave5 encoder.
> All these formats have a chroma subsampling ratio of 4:2:2 and therefore require a new image size calculation as the driver previously only handled a ratio of 4:2:0.
> 
> Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
> Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
> ---
>  .../chips-media/wave5/wave5-vpu-enc.c         | 59 +++++++++++++++++--
>  1 file changed, 54 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> index 5a32bb138158..77657f63a169 100644
> --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> @@ -39,6 +39,24 @@ static const struct vpu_format enc_fmt_list[FMT_TYPES][MAX_FMTS] = {
>  		{
>  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
>  		},
> +		{
> +			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422P,
> +		},
> +		{
> +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16,
> +		},
> +		{
> +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61,
> +		},
> +		{
> +			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422M,
> +		},
> +		{
> +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16M,
> +		},
> +		{
> +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61M,
> +		},
>  	}
>  };
>  
> @@ -101,13 +119,30 @@ static int start_encode(struct vpu_instance *inst, u32 *fail_res)
>  	struct vb2_v4l2_buffer *dst_buf;
>  	struct frame_buffer frame_buf;
>  	struct enc_param pic_param;
> -	u32 stride = ALIGN(inst->dst_fmt.width, 32);
> -	u32 luma_size = (stride * inst->dst_fmt.height);
> -	u32 chroma_size = ((stride / 2) * (inst->dst_fmt.height / 2));
> +	u32 stride = inst->src_fmt.plane_fmt[0].bytesperline;
> +	u32 luma_size = (stride * inst->src_fmt.height);
> +	u32 chroma_size = 0;

The helper introduced in previous patch also calculate sizeimage for each
planes, so no need for this code anymore.

>  
>  	memset(&pic_param, 0, sizeof(struct enc_param));
>  	memset(&frame_buf, 0, sizeof(struct frame_buffer));
>  
> +	if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV420 ||
> +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV420M)
> +		chroma_size = luma_size / 4;
> +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12 ||
> +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21 ||
> +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12M ||
> +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21M)
> +		chroma_size = luma_size / 2;
> +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422P ||
> +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422M)
> +		chroma_size = luma_size / 2;
> +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16 ||
> +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61 ||
> +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16M ||
> +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61M)
> +		chroma_size = luma_size;
> +
>  	dst_buf = v4l2_m2m_next_dst_buf(m2m_ctx);
>  	if (!dst_buf) {
>  		dev_dbg(inst->dev->dev, "%s: No destination buffer found\n", __func__);
> @@ -490,11 +525,15 @@ static int wave5_vpu_enc_s_fmt_out(struct file *file, void *fh, struct v4l2_form
>  	}
>  
>  	if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12 ||
> -	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12M) {
> +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12M ||
> +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16 ||
> +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16M) {
>  		inst->cbcr_interleave = true;
>  		inst->nv21 = false;
>  	} else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21 ||
> -		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21M) {
> +		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21M ||
> +		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61 ||
> +		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61M) {
>  		inst->cbcr_interleave = true;
>  		inst->nv21 = true;
>  	} else {
> @@ -1086,6 +1125,16 @@ static void wave5_set_enc_openparam(struct enc_open_param *open_param,
>  	u32 num_ctu_row = ALIGN(inst->dst_fmt.height, 64) / 64;
>  	u32 num_mb_row = ALIGN(inst->dst_fmt.height, 16) / 16;
>  
> +	if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422P ||
> +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16 ||
> +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61 ||
> +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422M ||
> +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16M ||
> +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61M)
> +		open_param->src_format = FORMAT_422;
> +	else
> +		open_param->src_format = FORMAT_420;
> +
>  	open_param->wave_param.gop_preset_idx = PRESET_IDX_IPP_SINGLE;
>  	open_param->wave_param.hvs_qp_scale = 2;
>  	open_param->wave_param.hvs_max_delta_qp = 10;

RE: [RESEND PATCH v2 1/4] media: chips-media: wave5: Support SPS/PPS generation for each IDR

[jackson.lee]

Hi Nicolas

Thanks for your review.

I will add the suffix into next patch.

> -----Original Message-----
> From: Nicolas Dufresne <nicolas@ndufresne.ca>
> Sent: Friday, April 19, 2024 5:43 AM
> To: jackson.lee <jackson.lee@chipsnmedia.com>; mchehab@kernel.org;
> sebastian.fricke@collabora.com
> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>; lafley.kim
> <lafley.kim@chipsnmedia.com>; b-brnich@ti.com
> Subject: Re: [RESEND PATCH v2 1/4] media: chips-media: wave5: Support SPS/PPS
> generation for each IDR
> 
> Le lundi 11 mars 2024 à 19:56 +0900, jackson.lee a écrit :
> > From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> >
> > Provide a control to toggle (0 = off / 1 = on), whether the SPS and
> > PPS are generated for every IDR.
> >
> > Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
> > Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
> > ---
> >  .../platform/chips-media/wave5/wave5-hw.c     | 19 +++++++++++++++----
> >  .../chips-media/wave5/wave5-vpu-enc.c         |  7 +++++++
> >  .../platform/chips-media/wave5/wave5-vpuapi.h |  1 +
> >  3 files changed, 23 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/media/platform/chips-media/wave5/wave5-hw.c
> > b/drivers/media/platform/chips-media/wave5/wave5-hw.c
> > index f1e022fb148e..4a262822bf17 100644
> > --- a/drivers/media/platform/chips-media/wave5/wave5-hw.c
> > +++ b/drivers/media/platform/chips-media/wave5/wave5-hw.c
> > @@ -23,6 +23,15 @@
> >  #define FEATURE_AVC_ENCODER		BIT(1)
> >  #define FEATURE_HEVC_ENCODER		BIT(0)
> >
> > +#define ENC_AVC_INTRA_IDR_PARAM_MASK	0x7ff
> > +#define ENC_AVC_INTRA_PERIOD		6
> > +#define ENC_AVC_IDR_PERIOD		17
> > +#define ENC_AVC_FORCED_IDR_HEADER	28
> > +
> > +#define ENC_HEVC_INTRA_QP		3
> > +#define ENC_HEVC_FORCED_IDR_HEADER	9
> > +#define ENC_HEVC_INTRA_PERIOD		16
> > +
> 
> Perhaps add the suffix _SHIFT to these macro ? I think it can easily be
> confused with a default value otherwise.
> 
> nit: can happen later, since this is how things got merged in this driver,
> but typically, registers offsets, shift and masks are usually consolidated in
> a single reg header. It would be nice to consider cleaning this up in future
> patches.
> 
> >  /* Decoder support fields */
> >  #define FEATURE_AVC_DECODER		BIT(3)
> >  #define FEATURE_HEVC_DECODER		BIT(2)
> > @@ -1601,12 +1610,14 @@ int wave5_vpu_enc_init_seq(struct vpu_instance
> > *inst)
> >
> >  	if (inst->std == W_AVC_ENC)
> >  		vpu_write_reg(inst->dev, W5_CMD_ENC_SEQ_INTRA_PARAM, p_param-
> >intra_qp |
> > -				((p_param->intra_period & 0x7ff) << 6) |
> > -				((p_param->avc_idr_period & 0x7ff) << 17));
> > +				((p_param->intra_period &
> ENC_AVC_INTRA_IDR_PARAM_MASK) << ENC_AVC_INTRA_PERIOD) |
> > +				((p_param->avc_idr_period &
> ENC_AVC_INTRA_IDR_PARAM_MASK) << ENC_AVC_IDR_PERIOD) |
> > +				(p_param->forced_idr_header_enable <<
> > +ENC_AVC_FORCED_IDR_HEADER));
> >  	else if (inst->std == W_HEVC_ENC)
> >  		vpu_write_reg(inst->dev, W5_CMD_ENC_SEQ_INTRA_PARAM,
> > -			      p_param->decoding_refresh_type | (p_param->intra_qp
> << 3) |
> > -				(p_param->intra_period << 16));
> > +			      p_param->decoding_refresh_type | (p_param->intra_qp
> << ENC_HEVC_INTRA_QP) |
> > +			      (p_param->forced_idr_header_enable <<
> ENC_HEVC_FORCED_IDR_HEADER) |
> > +			      (p_param->intra_period << ENC_HEVC_INTRA_PERIOD));
> >
> >  	reg_val = (p_param->rdo_skip << 2) |
> >  		(p_param->lambda_scaling_enable << 3) | diff --git
> > a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > index f29cfa3af94a..f04baa93a9b7 100644
> > --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > @@ -1061,6 +1061,9 @@ static int wave5_vpu_enc_s_ctrl(struct v4l2_ctrl
> *ctrl)
> >  	case V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE:
> >  		inst->enc_param.entropy_coding_mode = ctrl->val;
> >  		break;
> > +	case V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR:
> > +		inst->enc_param.forced_idr_header_enable = ctrl->val;
> > +		break;
> >  	case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
> >  		break;
> >  	default:
> > @@ -1219,6 +1222,7 @@ static void wave5_set_enc_openparam(struct
> enc_open_param *open_param,
> >  		else
> >  			open_param->wave_param.intra_refresh_arg = num_ctu_row;
> >  	}
> > +	open_param->wave_param.forced_idr_header_enable =
> > +input.forced_idr_header_enable;
> 
> in the long term, there is one too many abstraction in this driver, we should
> remove that.
> 
> >  }
> >
> >  static int initialize_sequence(struct vpu_instance *inst) @@ -1702,6
> > +1706,9 @@ static int wave5_vpu_open_enc(struct file *filp)
> >  			  0, 1, 1, 0);
> >  	v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave5_vpu_enc_ctrl_ops,
> >  			  V4L2_CID_MIN_BUFFERS_FOR_OUTPUT, 1, 32, 1, 1);
> > +	v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave5_vpu_enc_ctrl_ops,
> > +			  V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR,
> > +			  0, 1, 1, 0);
> >
> >  	if (v4l2_ctrl_hdl->error) {
> >  		ret = -ENODEV;
> > diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
> > b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
> > index 352f6e904e50..3ad6118550ac 100644
> > --- a/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
> > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
> > @@ -566,6 +566,7 @@ struct enc_wave_param {
> >  	u32 lambda_scaling_enable: 1; /* enable lambda scaling using custom
> GOP */
> >  	u32 transform8x8_enable: 1; /* enable 8x8 intra prediction and 8x8
> transform */
> >  	u32 mb_level_rc_enable: 1; /* enable MB-level rate control */
> > +	u32 forced_idr_header_enable: 1; /* enable header encoding before
> > +IDR frame */
> >  };
> >
> >  struct enc_open_param {
> 
> With the suggested _SHIFT suffix (or equivalent) added.
> 
> Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

RE: [RESEND PATCH v2 3/4] media: chips-media: wave5: Use helpers to calculate bytesperline and sizeimage.

[jackson.lee]

Hi Nicolas

> -----Original Message-----
> From: Nicolas Dufresne <nicolas@ndufresne.ca>
> Sent: Friday, April 19, 2024 6:03 AM
> To: jackson.lee <jackson.lee@chipsnmedia.com>; mchehab@kernel.org;
> sebastian.fricke@collabora.com
> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>; lafley.kim
> <lafley.kim@chipsnmedia.com>; b-brnich@ti.com
> Subject: Re: [RESEND PATCH v2 3/4] media: chips-media: wave5: Use helpers to
> calculate bytesperline and sizeimage.
> 
> Hi,
> 
> very nice cleanup.
> 
> Le lundi 11 mars 2024 à 19:56 +0900, jackson.lee a écrit :
> > From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> >
> > Use v4l2-common helper functions to calculate bytesperline and sizeimage,
> instead of calculating in a wave5 driver directly.
> > In case of raw(YUV) v4l2_pix_format, the wave5 driver updates
> v4l2_pix_format_mplane struct through v4l2_fill_pixfmt_mp() function.
> > Encoder and Decoder need same bytesperline and sizeimage values for same
> v4l2_pix_format.
> > So, a wave5_update_pix_fmt is refactored to support both together.
> >
> > Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
> > Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
> > ---
> >  .../platform/chips-media/wave5/wave5-helper.c |  24 ++
> >  .../platform/chips-media/wave5/wave5-helper.h |   4 +
> >  .../chips-media/wave5/wave5-vpu-dec.c         | 245 ++++--------------
> >  .../chips-media/wave5/wave5-vpu-enc.c         | 179 +++++--------
> >  .../platform/chips-media/wave5/wave5-vpu.h    |   4 -
> >  .../chips-media/wave5/wave5-vpuconfig.h       |  25 +-
> >  6 files changed, 164 insertions(+), 317 deletions(-)
> >
> > diff --git a/drivers/media/platform/chips-media/wave5/wave5-helper.c
> > b/drivers/media/platform/chips-media/wave5/wave5-helper.c
> > index 8433ecab230c..53cad4d17aa7 100644
> > --- a/drivers/media/platform/chips-media/wave5/wave5-helper.c
> > +++ b/drivers/media/platform/chips-media/wave5/wave5-helper.c
> > @@ -7,6 +7,8 @@
> >
> >  #include "wave5-helper.h"
> >
> > +#define DEFAULT_BS_SIZE(width, height) ((width) * (height) / 8 * 3)
> > +
> >  const char *state_to_str(enum vpu_instance_state state)  {
> >  	switch (state) {
> > @@ -211,3 +213,25 @@ void wave5_return_bufs(struct vb2_queue *q, u32 state)
> >  		v4l2_m2m_buf_done(vbuf, state);
> >  	}
> >  }
> > +
> > +void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
> > +			  unsigned int width,
> > +			  unsigned int height,
> > +			  const struct v4l2_frmsize_stepwise frmsize) {
> > +	v4l2_apply_frmsize_constraints(&width, &height, &frmsize);
> > +
> > +	if (pix_mp->pixelformat == V4L2_PIX_FMT_HEVC ||
> > +	    pix_mp->pixelformat == V4L2_PIX_FMT_H264) {
> 
> I notice the caller already index the frmsize between RAW and CODEC. I'd say
> pass the information as an argument so we don't have to update code later
> when more codecs are added.

What you are saying is this ? 
The caller already knows pixel format type if it is codec or raw type.
So I added a variable to describe a pixel format type.

     void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp, int pix_fmt_type, // add pix_fmt_type
			  unsigned int width,
			  unsigned int height,
			  const struct v4l2_frmsize_stepwise frmsize)

		if (pix_fmt_type == VPU_FMT_TYPE_CODEC) {

		}
		else {

		}

> 
> > +		pix_mp->width = width;
> > +		pix_mp->height = height;
> > +		pix_mp->num_planes = 1;
> > +		pix_mp->plane_fmt[0].bytesperline = 0;
> > +		pix_mp->plane_fmt[0].sizeimage = max(DEFAULT_BS_SIZE(width,
> height),
> > +						     pix_mp->plane_fmt[0].sizeimage);
> > +	} else {
> > +		v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat, width, height);
> > +	}
> > +	pix_mp->flags = 0;
> > +	pix_mp->field = V4L2_FIELD_NONE;
> > +}
> > diff --git a/drivers/media/platform/chips-media/wave5/wave5-helper.h
> > b/drivers/media/platform/chips-media/wave5/wave5-helper.h
> > index 6cee1c14d3ce..a8ecd1920207 100644
> > --- a/drivers/media/platform/chips-media/wave5/wave5-helper.h
> > +++ b/drivers/media/platform/chips-media/wave5/wave5-helper.h
> > @@ -28,4 +28,8 @@ const struct vpu_format
> *wave5_find_vpu_fmt_by_idx(unsigned int idx,
> >  						   const struct vpu_format
> fmt_list[MAX_FMTS]);  enum wave_std
> > wave5_to_vpu_std(unsigned int v4l2_pix_fmt, enum vpu_instance_type
> > type);  void wave5_return_bufs(struct vb2_queue *q, u32 state);
> > +void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
> > +			  unsigned int width,
> > +			  unsigned int height,
> > +			  const struct v4l2_frmsize_stepwise frmsize);
> >  #endif
> > diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> > b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> > index a199877c643b..43d0a21193dd 100644
> > --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> > @@ -11,115 +11,74 @@
> >  #define VPU_DEC_DEV_NAME "C&M Wave5 VPU decoder"
> >  #define VPU_DEC_DRV_NAME "wave5-dec"
> >
> > -#define DEFAULT_SRC_SIZE(width, height) ({			\
> > -	(width) * (height) / 8 * 3;					\
> > -})
> > -
> >  static const struct vpu_format dec_fmt_list[FMT_TYPES][MAX_FMTS] = {
> >  	[VPU_FMT_TYPE_CODEC] = {
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_H264,
> > -			.max_width = 8192,
> > -			.min_width = 32,
> > -			.max_height = 4320,
> > -			.min_height = 32,
> >  		},
> 
> I notice that min/max was different per codec, but in this patch you actually
> make them both identical. Is that an error, otherwise the change should be
> underlined in the commit message.
> 
> If you really need per codec granularity, you could define the common frmsize
> array before, and set a pointer here into the static array. This way you
> won't have to type the same thing over and over, but still have per codec
> granularity.
> 
> >  	},
> >  	[VPU_FMT_TYPE_RAW] = {
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422P,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422M,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16M,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61M,
> > -			.max_width = 8192,
> > -			.min_width = 8,
> > -			.max_height = 4320,
> > -			.min_height = 8,
> >  		},
> >  	}
> >  };
> >
> > +static const struct v4l2_frmsize_stepwise dec_frmsize[FMT_TYPES] = {
> > +	[VPU_FMT_TYPE_CODEC] = {
> > +		.min_width = W5_MIN_DEC_PIC_WIDTH,
> > +		.max_width = W5_MAX_DEC_PIC_WIDTH,
> > +		.step_width = W5_DEC_CODEC_STEP_WIDTH,
> > +		.min_height = W5_MIN_DEC_PIC_HEIGHT,
> > +		.max_height = W5_MAX_DEC_PIC_HEIGHT,
> > +		.step_height = W5_DEC_CODEC_STEP_HEIGHT,
> > +	},
> > +	[VPU_FMT_TYPE_RAW] = {
> > +		.min_width = W5_MIN_DEC_PIC_WIDTH,
> > +		.max_width = W5_MAX_DEC_PIC_WIDTH,
> > +		.step_width = W5_DEC_RAW_STEP_WIDTH,
> > +		.min_height = W5_MIN_DEC_PIC_HEIGHT,
> > +		.max_height = W5_MAX_DEC_PIC_HEIGHT,
> > +		.step_height = W5_DEC_RAW_STEP_HEIGHT,
> > +	},
> > +};
> > +
> >  /*
> >   * Make sure that the state switch is allowed and add logging for
> debugging
> >   * purposes
> > @@ -234,74 +193,6 @@ static void wave5_handle_src_buffer(struct
> vpu_instance *inst, dma_addr_t rd_ptr
> >  	inst->remaining_consumed_bytes = consumed_bytes;  }
> >
> > -static void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
> unsigned int width,
> > -				 unsigned int height)
> > -{
> > -	switch (pix_mp->pixelformat) {
> > -	case V4L2_PIX_FMT_YUV420:
> > -	case V4L2_PIX_FMT_NV12:
> > -	case V4L2_PIX_FMT_NV21:
> > -		pix_mp->width = round_up(width, 32);
> > -		pix_mp->height = round_up(height, 16);
> > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[0].sizeimage = width * height * 3 / 2;
> > -		break;
> > -	case V4L2_PIX_FMT_YUV422P:
> > -	case V4L2_PIX_FMT_NV16:
> > -	case V4L2_PIX_FMT_NV61:
> > -		pix_mp->width = round_up(width, 32);
> > -		pix_mp->height = round_up(height, 16);
> > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[0].sizeimage = width * height * 2;
> > -		break;
> > -	case V4L2_PIX_FMT_YUV420M:
> > -		pix_mp->width = round_up(width, 32);
> > -		pix_mp->height = round_up(height, 16);
> > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[0].sizeimage = width * height;
> > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
> > -		pix_mp->plane_fmt[1].sizeimage = width * height / 4;
> > -		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
> > -		pix_mp->plane_fmt[2].sizeimage = width * height / 4;
> > -		break;
> > -	case V4L2_PIX_FMT_NV12M:
> > -	case V4L2_PIX_FMT_NV21M:
> > -		pix_mp->width = round_up(width, 32);
> > -		pix_mp->height = round_up(height, 16);
> > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[0].sizeimage = width * height;
> > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[1].sizeimage = width * height / 2;
> > -		break;
> > -	case V4L2_PIX_FMT_YUV422M:
> > -		pix_mp->width = round_up(width, 32);
> > -		pix_mp->height = round_up(height, 16);
> > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[0].sizeimage = width * height;
> > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
> > -		pix_mp->plane_fmt[1].sizeimage = width * height / 2;
> > -		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
> > -		pix_mp->plane_fmt[2].sizeimage = width * height / 2;
> > -		break;
> > -	case V4L2_PIX_FMT_NV16M:
> > -	case V4L2_PIX_FMT_NV61M:
> > -		pix_mp->width = round_up(width, 32);
> > -		pix_mp->height = round_up(height, 16);
> > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[0].sizeimage = width * height;
> > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[1].sizeimage = width * height;
> > -		break;
> > -	default:
> > -		pix_mp->width = width;
> > -		pix_mp->height = height;
> > -		pix_mp->plane_fmt[0].bytesperline = 0;
> > -		pix_mp->plane_fmt[0].sizeimage = max(DEFAULT_SRC_SIZE(width,
> height),
> > -						     pix_mp->plane_fmt[0].sizeimage);
> > -		break;
> > -	}
> > -}
> > -
> >  static int start_decode(struct vpu_instance *inst, u32 *fail_res)  {
> >  	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; @@ -397,9
> > +288,11 @@ static int handle_dynamic_resolution_change(struct vpu_instance
> *inst)
> >  			initial_info->pic_crop_rect.top -
> > initial_info->pic_crop_rect.bottom;
> >
> >  		wave5_update_pix_fmt(&inst->src_fmt, initial_info->pic_width,
> > -				     initial_info->pic_height);
> > +						     initial_info->pic_height,
> > +
> dec_frmsize[VPU_FMT_TYPE_CODEC]);
> >  		wave5_update_pix_fmt(&inst->dst_fmt, initial_info->pic_width,
> > -				     initial_info->pic_height);
> > +						     initial_info->pic_height,
> > +						     dec_frmsize[VPU_FMT_TYPE_RAW]);
> >  	}
> >
> >  	v4l2_event_queue_fh(fh, &vpu_event_src_ch); @@ -548,12 +441,7 @@
> > static int wave5_vpu_dec_enum_framesizes(struct file *f, void *fh, struct
> v4l2_f
> >  	}
> >
> >  	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> > -	fsize->stepwise.min_width = vpu_fmt->min_width;
> > -	fsize->stepwise.max_width = vpu_fmt->max_width;
> > -	fsize->stepwise.step_width = 1;
> > -	fsize->stepwise.min_height = vpu_fmt->min_height;
> > -	fsize->stepwise.max_height = vpu_fmt->max_height;
> > -	fsize->stepwise.step_height = 1;
> > +	fsize->stepwise = dec_frmsize[VPU_FMT_TYPE_CODEC];
> >
> >  	return 0;
> >  }
> > @@ -589,14 +477,10 @@ static int wave5_vpu_dec_try_fmt_cap(struct file
> *file, void *fh, struct v4l2_fo
> >  		width = inst->dst_fmt.width;
> >  		height = inst->dst_fmt.height;
> >  		f->fmt.pix_mp.pixelformat = inst->dst_fmt.pixelformat;
> > -		f->fmt.pix_mp.num_planes = inst->dst_fmt.num_planes;
> >  	} else {
> > -		const struct v4l2_format_info *info = v4l2_format_info(vpu_fmt-
> >v4l2_pix_fmt);
> > -
> > -		width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width, vpu_fmt-
> >max_width);
> > -		height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height,
> vpu_fmt->max_height);
> > +		width = f->fmt.pix_mp.width;
> > +		height = f->fmt.pix_mp.height;
> >  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> > -		f->fmt.pix_mp.num_planes = info->mem_planes;
> >  	}
> >
> >  	if (p_dec_info->initial_info_obtained) { @@ -604,9 +488,9 @@ static
> > int wave5_vpu_dec_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
> >  		height = inst->dst_fmt.height;
> >  	}
> >
> > -	wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
> > -	f->fmt.pix_mp.flags = 0;
> > -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> > +					     height,
> > +					     dec_frmsize[VPU_FMT_TYPE_RAW]);
> >  	f->fmt.pix_mp.colorspace = inst->colorspace;
> >  	f->fmt.pix_mp.ycbcr_enc = inst->ycbcr_enc;
> >  	f->fmt.pix_mp.quantization = inst->quantization; @@ -719,6 +603,7 @@
> > static int wave5_vpu_dec_try_fmt_out(struct file *file, void *fh,
> > struct v4l2_fo  {
> >  	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
> >  	const struct vpu_format *vpu_fmt;
> > +	int width, height;
> >
> >  	dev_dbg(inst->dev->dev,
> >  		"%s: fourcc: %u width: %u height: %u num_planes: %u
> colorspace: %u
> > field: %u\n", @@ -727,20 +612,18 @@ static int
> > wave5_vpu_dec_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
> >
> >  	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat,
> dec_fmt_list[VPU_FMT_TYPE_CODEC]);
> >  	if (!vpu_fmt) {
> > +		width = inst->src_fmt.width;
> > +		height = inst->src_fmt.height;
> >  		f->fmt.pix_mp.pixelformat = inst->src_fmt.pixelformat;
> > -		f->fmt.pix_mp.num_planes = inst->src_fmt.num_planes;
> > -		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->src_fmt.width, inst-
> >src_fmt.height);
> >  	} else {
> > -		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width,
> vpu_fmt->max_width);
> > -		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height,
> vpu_fmt->max_height);
> > -
> > +		width = f->fmt.pix_mp.width;
> > +		height = f->fmt.pix_mp.height;
> >  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> > -		f->fmt.pix_mp.num_planes = 1;
> > -		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
> >  	}
> >
> > -	f->fmt.pix_mp.flags = 0;
> > -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> > +					     height,
> > +					     dec_frmsize[VPU_FMT_TYPE_CODEC]);
> >
> >  	return 0;
> >  }
> > @@ -782,7 +665,9 @@ static int wave5_vpu_dec_s_fmt_out(struct file *file,
> void *fh, struct v4l2_form
> >  	inst->quantization = f->fmt.pix_mp.quantization;
> >  	inst->xfer_func = f->fmt.pix_mp.xfer_func;
> >
> > -	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width, f-
> >fmt.pix_mp.height);
> > +	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width,
> > +					     f->fmt.pix_mp.height,
> > +					     dec_frmsize[VPU_FMT_TYPE_RAW]);
> >
> >  	return 0;
> >  }
> > @@ -1005,6 +890,7 @@ static int wave5_vpu_dec_queue_setup(struct vb2_queue
> *q, unsigned int *num_buff
> >  	struct vpu_instance *inst = vb2_get_drv_priv(q);
> >  	struct v4l2_pix_format_mplane inst_format =
> >  		(q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) ? inst->src_fmt :
> > inst->dst_fmt;
> > +	unsigned int i;
> >
> >  	dev_dbg(inst->dev->dev, "%s: num_buffers: %u | num_planes: %u |
> type: %u\n", __func__,
> >  		*num_buffers, *num_planes, q->type); @@ -1018,31 +904,9 @@
> static
> > int wave5_vpu_dec_queue_setup(struct vb2_queue *q, unsigned int *num_buff
> >  		if (*num_buffers < inst->fbc_buf_count)
> >  			*num_buffers = inst->fbc_buf_count;
> >
> > -		if (*num_planes == 1) {
> > -			if (inst->output_format == FORMAT_422)
> > -				sizes[0] = inst_format.width * inst_format.height
> * 2;
> > -			else
> > -				sizes[0] = inst_format.width * inst_format.height
> * 3 / 2;
> > -			dev_dbg(inst->dev->dev, "%s: size[0]: %u\n", __func__,
> sizes[0]);
> > -		} else if (*num_planes == 2) {
> > -			sizes[0] = inst_format.width * inst_format.height;
> > -			if (inst->output_format == FORMAT_422)
> > -				sizes[1] = inst_format.width * inst_format.height;
> > -			else
> > -				sizes[1] = inst_format.width * inst_format.height
> / 2;
> > -			dev_dbg(inst->dev->dev, "%s: size[0]: %u |
> size[1]: %u\n",
> > -				__func__, sizes[0], sizes[1]);
> > -		} else if (*num_planes == 3) {
> > -			sizes[0] = inst_format.width * inst_format.height;
> > -			if (inst->output_format == FORMAT_422) {
> > -				sizes[1] = inst_format.width * inst_format.height
> / 2;
> > -				sizes[2] = inst_format.width * inst_format.height
> / 2;
> > -			} else {
> > -				sizes[1] = inst_format.width * inst_format.height
> / 4;
> > -				sizes[2] = inst_format.width * inst_format.height
> / 4;
> > -			}
> > -			dev_dbg(inst->dev->dev, "%s: size[0]: %u | size[1]: %u |
> size[2]: %u\n",
> > -				__func__, sizes[0], sizes[1], sizes[2]);
> > +		for (i = 0; i < *num_planes; i++) {
> > +			sizes[i] = inst_format.plane_fmt[i].sizeimage;
> > +			dev_dbg(inst->dev->dev, "%s: size[%u]: %u\n", __func__,
> i,
> > +sizes[i]);
> >  		}
> >  	}
> >
> > @@ -1564,20 +1428,15 @@ static const struct vb2_ops
> > wave5_vpu_dec_vb2_ops = {  static void wave5_set_default_format(struct
> v4l2_pix_format_mplane *src_fmt,
> >  				     struct v4l2_pix_format_mplane *dst_fmt)  {
> > -	unsigned int dst_pix_fmt =
> dec_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> > -	const struct v4l2_format_info *dst_fmt_info =
> v4l2_format_info(dst_pix_fmt);
> > -
> >  	src_fmt->pixelformat =
> dec_fmt_list[VPU_FMT_TYPE_CODEC][0].v4l2_pix_fmt;
> > -	src_fmt->field = V4L2_FIELD_NONE;
> > -	src_fmt->flags = 0;
> > -	src_fmt->num_planes = 1;
> > -	wave5_update_pix_fmt(src_fmt, 720, 480);
> > -
> > -	dst_fmt->pixelformat = dst_pix_fmt;
> > -	dst_fmt->field = V4L2_FIELD_NONE;
> > -	dst_fmt->flags = 0;
> > -	dst_fmt->num_planes = dst_fmt_info->mem_planes;
> > -	wave5_update_pix_fmt(dst_fmt, 736, 480);
> > +	wave5_update_pix_fmt(src_fmt, W5_DEF_DEC_PIC_WIDTH,
> > +				      W5_DEF_DEC_PIC_HEIGHT,
> > +				      dec_frmsize[VPU_FMT_TYPE_CODEC]);
> > +
> > +	dst_fmt->pixelformat = dec_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> > +	wave5_update_pix_fmt(dst_fmt, W5_DEF_DEC_PIC_WIDTH,
> > +				      W5_DEF_DEC_PIC_HEIGHT,
> > +				      dec_frmsize[VPU_FMT_TYPE_RAW]);
> >  }
> >
> >  static int wave5_vpu_dec_queue_init(void *priv, struct vb2_queue
> > *src_vq, struct vb2_queue *dst_vq) diff --git
> > a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > index 013e2bb37fbb..5a32bb138158 100644
> > --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > @@ -15,65 +15,52 @@ static const struct vpu_format
> enc_fmt_list[FMT_TYPES][MAX_FMTS] = {
> >  	[VPU_FMT_TYPE_CODEC] = {
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
> > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_H264,
> > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> >  		},
> >  	},
> >  	[VPU_FMT_TYPE_RAW] = {
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
> > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
> > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
> > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
> > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
> > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> >  		},
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
> > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> >  		},
> >  	}
> >  };
> >
> > +static const struct v4l2_frmsize_stepwise enc_frmsize[FMT_TYPES] = {
> > +	[VPU_FMT_TYPE_CODEC] = {
> > +		.min_width = W5_MIN_ENC_PIC_WIDTH,
> > +		.max_width = W5_MAX_ENC_PIC_WIDTH,
> > +		.step_width = W5_ENC_CODEC_STEP_WIDTH,
> > +		.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > +		.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > +		.step_height = W5_ENC_CODEC_STEP_HEIGHT,
> > +	},
> > +	[VPU_FMT_TYPE_RAW] = {
> > +		.min_width = W5_MIN_ENC_PIC_WIDTH,
> > +		.max_width = W5_MAX_ENC_PIC_WIDTH,
> > +		.step_width = W5_ENC_RAW_STEP_WIDTH,
> > +		.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > +		.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > +		.step_height = W5_ENC_RAW_STEP_HEIGHT,
> > +	},
> > +};
> > +
> >  static int switch_state(struct vpu_instance *inst, enum
> > vpu_instance_state state)  {
> >  	switch (state) {
> > @@ -106,46 +93,6 @@ static int switch_state(struct vpu_instance *inst, enum
> vpu_instance_state state
> >  	return -EINVAL;
> >  }
> >
> > -static void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
> unsigned int width,
> > -				 unsigned int height)
> > -{
> > -	switch (pix_mp->pixelformat) {
> > -	case V4L2_PIX_FMT_YUV420:
> > -	case V4L2_PIX_FMT_NV12:
> > -	case V4L2_PIX_FMT_NV21:
> > -		pix_mp->width = width;
> > -		pix_mp->height = height;
> > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height *
> 3 / 2;
> > -		break;
> > -	case V4L2_PIX_FMT_YUV420M:
> > -		pix_mp->width = width;
> > -		pix_mp->height = height;
> > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height;
> > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
> > -		pix_mp->plane_fmt[1].sizeimage = round_up(width, 32) * height /
> 4;
> > -		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
> > -		pix_mp->plane_fmt[2].sizeimage = round_up(width, 32) * height /
> 4;
> > -		break;
> > -	case V4L2_PIX_FMT_NV12M:
> > -	case V4L2_PIX_FMT_NV21M:
> > -		pix_mp->width = width;
> > -		pix_mp->height = height;
> > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height;
> > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
> > -		pix_mp->plane_fmt[1].sizeimage = round_up(width, 32) * height /
> 2;
> > -		break;
> > -	default:
> > -		pix_mp->width = width;
> > -		pix_mp->height = height;
> > -		pix_mp->plane_fmt[0].bytesperline = 0;
> > -		pix_mp->plane_fmt[0].sizeimage = width * height / 8 * 3;
> > -		break;
> > -	}
> > -}
> > -
> >  static int start_encode(struct vpu_instance *inst, u32 *fail_res)  {
> >  	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; @@ -360,13
> > +307,8 @@ static int wave5_vpu_enc_enum_framesizes(struct file *f, void *fh,
> struct v4l2_f
> >  			return -EINVAL;
> >  	}
> >
> > -	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> > -	fsize->stepwise.min_width = vpu_fmt->min_width;
> > -	fsize->stepwise.max_width = vpu_fmt->max_width;
> > -	fsize->stepwise.step_width = 1;
> > -	fsize->stepwise.min_height = vpu_fmt->min_height;
> > -	fsize->stepwise.max_height = vpu_fmt->max_height;
> > -	fsize->stepwise.step_height = 1;
> > +	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > +	fsize->stepwise = enc_frmsize[VPU_FMT_TYPE_CODEC];
> >
> >  	return 0;
> >  }
> > @@ -392,6 +334,7 @@ static int wave5_vpu_enc_try_fmt_cap(struct file
> > *file, void *fh, struct v4l2_fo  {
> >  	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
> >  	const struct vpu_format *vpu_fmt;
> > +	int width, height;
> >
> >  	dev_dbg(inst->dev->dev, "%s: fourcc: %u width: %u height: %u
> num_planes: %u field: %u\n",
> >  		__func__, f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width,
> > f->fmt.pix_mp.height, @@ -399,20 +342,18 @@ static int
> > wave5_vpu_enc_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
> >
> >  	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat,
> enc_fmt_list[VPU_FMT_TYPE_CODEC]);
> >  	if (!vpu_fmt) {
> > +		width = inst->dst_fmt.width;
> > +		height = inst->dst_fmt.height;
> >  		f->fmt.pix_mp.pixelformat = inst->dst_fmt.pixelformat;
> > -		f->fmt.pix_mp.num_planes = inst->dst_fmt.num_planes;
> > -		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->dst_fmt.width, inst-
> >dst_fmt.height);
> >  	} else {
> > -		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width,
> vpu_fmt->max_width);
> > -		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height,
> vpu_fmt->max_height);
> > -
> > +		width = f->fmt.pix_mp.width;
> > +		height = f->fmt.pix_mp.height;
> >  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> > -		f->fmt.pix_mp.num_planes = 1;
> > -		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
> >  	}
> >
> > -	f->fmt.pix_mp.flags = 0;
> > -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> > +					     height,
> > +					     enc_frmsize[VPU_FMT_TYPE_CODEC]);
> >  	f->fmt.pix_mp.colorspace = inst->colorspace;
> >  	f->fmt.pix_mp.ycbcr_enc = inst->ycbcr_enc;
> >  	f->fmt.pix_mp.quantization = inst->quantization; @@ -500,6 +441,7 @@
> > static int wave5_vpu_enc_try_fmt_out(struct file *file, void *fh,
> > struct v4l2_fo  {
> >  	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
> >  	const struct vpu_format *vpu_fmt;
> > +	int width, height;
> >
> >  	dev_dbg(inst->dev->dev, "%s: fourcc: %u width: %u height: %u
> num_planes: %u field: %u\n",
> >  		__func__, f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width,
> > f->fmt.pix_mp.height, @@ -507,21 +449,18 @@ static int
> > wave5_vpu_enc_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
> >
> >  	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat,
> enc_fmt_list[VPU_FMT_TYPE_RAW]);
> >  	if (!vpu_fmt) {
> > +		width = inst->src_fmt.width;
> > +		height = inst->src_fmt.height;
> >  		f->fmt.pix_mp.pixelformat = inst->src_fmt.pixelformat;
> > -		f->fmt.pix_mp.num_planes = inst->src_fmt.num_planes;
> > -		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->src_fmt.width, inst-
> >src_fmt.height);
> >  	} else {
> > -		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width,
> vpu_fmt->max_width);
> > -		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height,
> vpu_fmt->max_height);
> > -		const struct v4l2_format_info *info = v4l2_format_info(vpu_fmt-
> >v4l2_pix_fmt);
> > -
> > +		width = f->fmt.pix_mp.width;
> > +		height = f->fmt.pix_mp.height;
> >  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> > -		f->fmt.pix_mp.num_planes = info->mem_planes;
> > -		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
> >  	}
> >
> > -	f->fmt.pix_mp.flags = 0;
> > -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> > +					     height,
> > +					     enc_frmsize[VPU_FMT_TYPE_RAW]);
> >
> >  	return 0;
> >  }
> > @@ -568,7 +507,11 @@ static int wave5_vpu_enc_s_fmt_out(struct file *file,
> void *fh, struct v4l2_form
> >  	inst->quantization = f->fmt.pix_mp.quantization;
> >  	inst->xfer_func = f->fmt.pix_mp.xfer_func;
> >
> > -	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width, f-
> >fmt.pix_mp.height);
> > +	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width,
> > +					     f->fmt.pix_mp.height,
> > +					     enc_frmsize[VPU_FMT_TYPE_CODEC]);
> > +	inst->conf_win.width = inst->dst_fmt.width;
> > +	inst->conf_win.height = inst->dst_fmt.height;
> >
> >  	return 0;
> >  }
> > @@ -584,12 +527,17 @@ static int wave5_vpu_enc_g_selection(struct file
> *file, void *fh, struct v4l2_se
> >  	switch (s->target) {
> >  	case V4L2_SEL_TGT_CROP_DEFAULT:
> >  	case V4L2_SEL_TGT_CROP_BOUNDS:
> > -	case V4L2_SEL_TGT_CROP:
> >  		s->r.left = 0;
> >  		s->r.top = 0;
> >  		s->r.width = inst->dst_fmt.width;
> >  		s->r.height = inst->dst_fmt.height;
> >  		break;
> > +	case V4L2_SEL_TGT_CROP:
> > +		s->r.left = 0;
> > +		s->r.top = 0;
> > +		s->r.width = inst->conf_win.width;
> > +		s->r.height = inst->conf_win.height;
> > +		break;
> >  	default:
> >  		return -EINVAL;
> >  	}
> > @@ -612,8 +560,10 @@ static int wave5_vpu_enc_s_selection(struct file
> > *file, void *fh, struct v4l2_se
> >
> >  	s->r.left = 0;
> >  	s->r.top = 0;
> > -	s->r.width = inst->src_fmt.width;
> > -	s->r.height = inst->src_fmt.height;
> > +	s->r.width = min(s->r.width, inst->dst_fmt.width);
> > +	s->r.height = min(s->r.height, inst->dst_fmt.height);
> > +
> > +	inst->conf_win = s->r;
> >
> >  	return 0;
> >  }
> > @@ -1151,8 +1101,8 @@ static void wave5_set_enc_openparam(struct
> enc_open_param *open_param,
> >  	open_param->wave_param.lambda_scaling_enable = 1;
> >
> >  	open_param->line_buf_int_en = true;
> > -	open_param->pic_width = inst->dst_fmt.width;
> > -	open_param->pic_height = inst->dst_fmt.height;
> > +	open_param->pic_width = inst->conf_win.width;
> > +	open_param->pic_height = inst->conf_win.height;
> >  	open_param->frame_rate_info = inst->frame_rate;
> >  	open_param->rc_enable = inst->rc_enable;
> >  	if (inst->rc_enable) {
> > @@ -1456,20 +1406,15 @@ static const struct vb2_ops
> > wave5_vpu_enc_vb2_ops = {  static void wave5_set_default_format(struct
> v4l2_pix_format_mplane *src_fmt,
> >  				     struct v4l2_pix_format_mplane *dst_fmt)  {
> > -	unsigned int src_pix_fmt =
> enc_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> > -	const struct v4l2_format_info *src_fmt_info =
> v4l2_format_info(src_pix_fmt);
> > -
> > -	src_fmt->pixelformat = src_pix_fmt;
> > -	src_fmt->field = V4L2_FIELD_NONE;
> > -	src_fmt->flags = 0;
> > -	src_fmt->num_planes = src_fmt_info->mem_planes;
> > -	wave5_update_pix_fmt(src_fmt, 416, 240);
> > +	src_fmt->pixelformat = enc_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> > +	wave5_update_pix_fmt(src_fmt, W5_DEF_ENC_PIC_WIDTH,
> > +				      W5_DEF_ENC_PIC_HEIGHT,
> > +				      enc_frmsize[VPU_FMT_TYPE_RAW]);
> >
> >  	dst_fmt->pixelformat =
> enc_fmt_list[VPU_FMT_TYPE_CODEC][0].v4l2_pix_fmt;
> > -	dst_fmt->field = V4L2_FIELD_NONE;
> > -	dst_fmt->flags = 0;
> > -	dst_fmt->num_planes = 1;
> > -	wave5_update_pix_fmt(dst_fmt, 416, 240);
> > +	wave5_update_pix_fmt(dst_fmt, W5_DEF_ENC_PIC_WIDTH,
> > +				      W5_DEF_ENC_PIC_HEIGHT,
> > +				      enc_frmsize[VPU_FMT_TYPE_CODEC]);
> >  }
> >
> >  static int wave5_vpu_enc_queue_init(void *priv, struct vb2_queue
> > *src_vq, struct vb2_queue *dst_vq) @@ -1734,6 +1679,8 @@ static int
> wave5_vpu_open_enc(struct file *filp)
> >  	v4l2_ctrl_handler_setup(v4l2_ctrl_hdl);
> >
> >  	wave5_set_default_format(&inst->src_fmt, &inst->dst_fmt);
> > +	inst->conf_win.width = inst->dst_fmt.width;
> > +	inst->conf_win.height = inst->dst_fmt.height;
> >  	inst->colorspace = V4L2_COLORSPACE_REC709;
> >  	inst->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> >  	inst->quantization = V4L2_QUANTIZATION_DEFAULT; diff --git
> > a/drivers/media/platform/chips-media/wave5/wave5-vpu.h
> > b/drivers/media/platform/chips-media/wave5/wave5-vpu.h
> > index 32b7fd3730b5..691d6341fcda 100644
> > --- a/drivers/media/platform/chips-media/wave5/wave5-vpu.h
> > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu.h
> > @@ -38,10 +38,6 @@ enum vpu_fmt_type {
> >
> >  struct vpu_format {
> >  	unsigned int v4l2_pix_fmt;
> > -	unsigned int max_width;
> > -	unsigned int min_width;
> > -	unsigned int max_height;
> > -	unsigned int min_height;
> >  };
> >
> >  static inline struct vpu_instance *wave5_to_vpu_inst(struct v4l2_fh
> > *vfh) diff --git
> > a/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> > b/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> > index d9751eedb0f9..e6a34ae7084e 100644
> > --- a/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> > @@ -30,10 +30,27 @@
> >
> >  #define MAX_NUM_INSTANCE                32
> >
> > -#define W5_MIN_ENC_PIC_WIDTH            256
> > -#define W5_MIN_ENC_PIC_HEIGHT           128
> > -#define W5_MAX_ENC_PIC_WIDTH            8192
> > -#define W5_MAX_ENC_PIC_HEIGHT           8192
> > +#define W5_DEF_DEC_PIC_WIDTH            720U
> > +#define W5_DEF_DEC_PIC_HEIGHT           480U
> > +#define W5_MIN_DEC_PIC_WIDTH            32U
> > +#define W5_MIN_DEC_PIC_HEIGHT           32U
> > +#define W5_MAX_DEC_PIC_WIDTH            8192U
> > +#define W5_MAX_DEC_PIC_HEIGHT           4320U
> > +#define W5_DEC_CODEC_STEP_WIDTH         1U
> > +#define W5_DEC_CODEC_STEP_HEIGHT        1U
> > +#define W5_DEC_RAW_STEP_WIDTH           32U
> > +#define W5_DEC_RAW_STEP_HEIGHT          16U
> > +
> > +#define W5_DEF_ENC_PIC_WIDTH            416U
> > +#define W5_DEF_ENC_PIC_HEIGHT           240U
> > +#define W5_MIN_ENC_PIC_WIDTH            256U
> > +#define W5_MIN_ENC_PIC_HEIGHT           128U
> > +#define W5_MAX_ENC_PIC_WIDTH            8192U
> > +#define W5_MAX_ENC_PIC_HEIGHT           8192U
> > +#define W5_ENC_CODEC_STEP_WIDTH         8U
> > +#define W5_ENC_CODEC_STEP_HEIGHT        8U
> > +#define W5_ENC_RAW_STEP_WIDTH           32U
> > +#define W5_ENC_RAW_STEP_HEIGHT          16U
> >
> >  //  application specific configuration
> >  #define VPU_ENC_TIMEOUT                 60000
> 
> 
> I'll have to give this one a second pass on next revision, but nice work.
> 
> Nicolas

RE: [RESEND PATCH v2 4/4] media: chips-media: wave5: Support YUV422 raw pixel-formats on the encoder.

[jackson.lee]

Hi Nicolas


> -----Original Message-----
> From: Nicolas Dufresne <nicolas@ndufresne.ca>
> Sent: Friday, April 19, 2024 6:06 AM
> To: jackson.lee <jackson.lee@chipsnmedia.com>; mchehab@kernel.org;
> sebastian.fricke@collabora.com
> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>; lafley.kim
> <lafley.kim@chipsnmedia.com>; b-brnich@ti.com
> Subject: Re: [RESEND PATCH v2 4/4] media: chips-media: wave5: Support YUV422
> raw pixel-formats on the encoder.
> 
> Le lundi 11 mars 2024 à 19:56 +0900, jackson.lee a écrit :
> > From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> >
> > Add support for the YUV422P, NV16, NV61, YUV422M, NV16M, NV61M raw pixel-
> formats to the Wave5 encoder.
> > All these formats have a chroma subsampling ratio of 4:2:2 and therefore
> require a new image size calculation as the driver previously only handled a
> ratio of 4:2:0.
> >
> > Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
> > Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
> > ---
> >  .../chips-media/wave5/wave5-vpu-enc.c         | 59 +++++++++++++++++--
> >  1 file changed, 54 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > index 5a32bb138158..77657f63a169 100644
> > --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > @@ -39,6 +39,24 @@ static const struct vpu_format
> enc_fmt_list[FMT_TYPES][MAX_FMTS] = {
> >  		{
> >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
> >  		},
> > +		{
> > +			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422P,
> > +		},
> > +		{
> > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16,
> > +		},
> > +		{
> > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61,
> > +		},
> > +		{
> > +			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422M,
> > +		},
> > +		{
> > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16M,
> > +		},
> > +		{
> > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61M,
> > +		},
> >  	}
> >  };
> >
> > @@ -101,13 +119,30 @@ static int start_encode(struct vpu_instance *inst,
> u32 *fail_res)
> >  	struct vb2_v4l2_buffer *dst_buf;
> >  	struct frame_buffer frame_buf;
> >  	struct enc_param pic_param;
> > -	u32 stride = ALIGN(inst->dst_fmt.width, 32);
> > -	u32 luma_size = (stride * inst->dst_fmt.height);
> > -	u32 chroma_size = ((stride / 2) * (inst->dst_fmt.height / 2));
> > +	u32 stride = inst->src_fmt.plane_fmt[0].bytesperline;
> > +	u32 luma_size = (stride * inst->src_fmt.height);
> > +	u32 chroma_size = 0;
> 
> The helper introduced in previous patch also calculate sizeimage for each
> planes, so no need for this code anymore.

Your comment means the below code? 

	u32 luma_size = inst->src_fmt.plane_fmt[0].sizeimage
	u32 chroma_size = inst->src_fmt.plane_fmt[1].sizeimage

	
> 
> >
> >  	memset(&pic_param, 0, sizeof(struct enc_param));
> >  	memset(&frame_buf, 0, sizeof(struct frame_buffer));
> >

The below code could be removed.

> > +	if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV420 ||
> > +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV420M)
> > +		chroma_size = luma_size / 4;
> > +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12 ||
> > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21 ||
> > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12M ||
> > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21M)
> > +		chroma_size = luma_size / 2;
> > +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422P ||
> > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422M)
> > +		chroma_size = luma_size / 2;
> > +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16 ||
> > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61 ||
> > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16M ||
> > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61M)
> > +		chroma_size = luma_size;
> > +

Is That right?

Thanks
Jackson.



> >  	dst_buf = v4l2_m2m_next_dst_buf(m2m_ctx);
> >  	if (!dst_buf) {
> >  		dev_dbg(inst->dev->dev, "%s: No destination buffer found\n",
> > __func__); @@ -490,11 +525,15 @@ static int wave5_vpu_enc_s_fmt_out(struct
> file *file, void *fh, struct v4l2_form
> >  	}
> >
> >  	if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12 ||
> > -	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12M) {
> > +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12M ||
> > +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16 ||
> > +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16M) {
> >  		inst->cbcr_interleave = true;
> >  		inst->nv21 = false;
> >  	} else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21 ||
> > -		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21M) {
> > +		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21M ||
> > +		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61 ||
> > +		   inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61M) {
> >  		inst->cbcr_interleave = true;
> >  		inst->nv21 = true;
> >  	} else {
> > @@ -1086,6 +1125,16 @@ static void wave5_set_enc_openparam(struct
> enc_open_param *open_param,
> >  	u32 num_ctu_row = ALIGN(inst->dst_fmt.height, 64) / 64;
> >  	u32 num_mb_row = ALIGN(inst->dst_fmt.height, 16) / 16;
> >
> > +	if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422P ||
> > +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16 ||
> > +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61 ||
> > +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422M ||
> > +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16M ||
> > +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61M)
> > +		open_param->src_format = FORMAT_422;
> > +	else
> > +		open_param->src_format = FORMAT_420;
> > +
> >  	open_param->wave_param.gop_preset_idx = PRESET_IDX_IPP_SINGLE;
> >  	open_param->wave_param.hvs_qp_scale = 2;
> >  	open_param->wave_param.hvs_max_delta_qp = 10;

Re: [RESEND PATCH v2 4/4] media: chips-media: wave5: Support YUV422 raw pixel-formats on the encoder.

[Nicolas Dufresne]

Hi Jackson,

Le lundi 22 avril 2024 à 04:30 +0000, jackson.lee a écrit :
> Hi Nicolas
> 
> 
> > -----Original Message-----
> > From: Nicolas Dufresne <nicolas@ndufresne.ca>
> > Sent: Friday, April 19, 2024 6:06 AM
> > To: jackson.lee <jackson.lee@chipsnmedia.com>; mchehab@kernel.org;
> > sebastian.fricke@collabora.com
> > Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> > hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>; lafley.kim
> > <lafley.kim@chipsnmedia.com>; b-brnich@ti.com
> > Subject: Re: [RESEND PATCH v2 4/4] media: chips-media: wave5: Support YUV422
> > raw pixel-formats on the encoder.
> > 
> > Le lundi 11 mars 2024 à 19:56 +0900, jackson.lee a écrit :
> > > From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> > > 
> > > Add support for the YUV422P, NV16, NV61, YUV422M, NV16M, NV61M raw pixel-
> > formats to the Wave5 encoder.
> > > All these formats have a chroma subsampling ratio of 4:2:2 and therefore
> > require a new image size calculation as the driver previously only handled a
> > ratio of 4:2:0.
> > > 
> > > Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
> > > Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
> > > ---
> > >  .../chips-media/wave5/wave5-vpu-enc.c         | 59 +++++++++++++++++--
> > >  1 file changed, 54 insertions(+), 5 deletions(-)
> > > 
> > > diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > index 5a32bb138158..77657f63a169 100644
> > > --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > @@ -39,6 +39,24 @@ static const struct vpu_format
> > enc_fmt_list[FMT_TYPES][MAX_FMTS] = {
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
> > >  		},
> > > +		{
> > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422P,
> > > +		},
> > > +		{
> > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16,
> > > +		},
> > > +		{
> > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61,
> > > +		},
> > > +		{
> > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422M,
> > > +		},
> > > +		{
> > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16M,
> > > +		},
> > > +		{
> > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61M,
> > > +		},
> > >  	}
> > >  };
> > > 
> > > @@ -101,13 +119,30 @@ static int start_encode(struct vpu_instance *inst,
> > u32 *fail_res)
> > >  	struct vb2_v4l2_buffer *dst_buf;
> > >  	struct frame_buffer frame_buf;
> > >  	struct enc_param pic_param;
> > > -	u32 stride = ALIGN(inst->dst_fmt.width, 32);
> > > -	u32 luma_size = (stride * inst->dst_fmt.height);
> > > -	u32 chroma_size = ((stride / 2) * (inst->dst_fmt.height / 2));
> > > +	u32 stride = inst->src_fmt.plane_fmt[0].bytesperline;
> > > +	u32 luma_size = (stride * inst->src_fmt.height);
> > > +	u32 chroma_size = 0;
> > 
> > The helper introduced in previous patch also calculate sizeimage for each
> > planes, so no need for this code anymore.
> 
> Your comment means the below code? 
> 
> 	u32 luma_size = inst->src_fmt.plane_fmt[0].sizeimage
> 	u32 chroma_size = inst->src_fmt.plane_fmt[1].sizeimage
> 
> 	
> > 
> > > 
> > >  	memset(&pic_param, 0, sizeof(struct enc_param));
> > >  	memset(&frame_buf, 0, sizeof(struct frame_buffer));
> > > 
> 
> The below code could be removed.
> 
> > > +	if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV420 ||
> > > +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV420M)
> > > +		chroma_size = luma_size / 4;
> > > +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12 ||
> > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21 ||
> > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12M ||
> > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21M)
> > > +		chroma_size = luma_size / 2;
> > > +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422P ||
> > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422M)
> > > +		chroma_size = luma_size / 2;
> > > +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16 ||
> > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61 ||
> > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16M ||
> > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61M)
> > > +		chroma_size = luma_size;
> > > +
> 
> Is That right?

Yes, using the src_fmt seems accurate for the encoder.

cheers,
Nicolas

RE: [RESEND PATCH v2 3/4] media: chips-media: wave5: Use helpers to calculate bytesperline and sizeimage.

[jackson.lee]

Hi Nicolas

According to your comment, I modified the code as below.
Can you review the change whether it is what you mentioned? 

diff --git a/drivers/media/platform/chips-media/wave5/wave5-helper.c b/drivers/media/platform/chips-media/wave5/wave5-helper.c
index be7e2b2a0..b20ab69cd 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-helper.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-helper.c
@@ -228,14 +228,14 @@ void wave5_return_bufs(struct vb2_queue *q, u32 state)
 }
 
 void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
+			  int pix_fmt_type,
 			  unsigned int width,
 			  unsigned int height,
-			  const struct v4l2_frmsize_stepwise frmsize)
+			  const struct v4l2_frmsize_stepwise *frmsize)
 {
-	v4l2_apply_frmsize_constraints(&width, &height, &frmsize);
+	v4l2_apply_frmsize_constraints(&width, &height, frmsize);
 
-	if (pix_mp->pixelformat == V4L2_PIX_FMT_HEVC ||
-	    pix_mp->pixelformat == V4L2_PIX_FMT_H264) {
+	if (pix_fmt_type == VPU_FMT_TYPE_CODEC) {
 		pix_mp->width = width;
 		pix_mp->height = height;
 		pix_mp->num_planes = 1;
diff --git a/drivers/media/platform/chips-media/wave5/wave5-helper.h b/drivers/media/platform/chips-media/wave5/wave5-helper.h
index a8ecd1920..9937fce55 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-helper.h
+++ b/drivers/media/platform/chips-media/wave5/wave5-helper.h
@@ -29,7 +29,8 @@ const struct vpu_format *wave5_find_vpu_fmt_by_idx(unsigned int idx,
 enum wave_std wave5_to_vpu_std(unsigned int v4l2_pix_fmt, enum vpu_instance_type type);
 void wave5_return_bufs(struct vb2_queue *q, u32 state);
 void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
+			  int pix_fmt_type,
 			  unsigned int width,
 			  unsigned int height,
-			  const struct v4l2_frmsize_stepwise frmsize);
+			  const struct v4l2_frmsize_stepwise *frmsize);
 #endif
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
index 461396ea4..925326734 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
@@ -11,74 +11,100 @@
 #define VPU_DEC_DEV_NAME "C&M Wave5 VPU decoder"
 #define VPU_DEC_DRV_NAME "wave5-dec"
 
+static const struct v4l2_frmsize_stepwise dec_frmsize[FMT_TYPES][MAX_FMTS] = {
+	[VPU_FMT_TYPE_CODEC] = {
+		{
+			.min_width = W5_MIN_DEC_PIC_8_WIDTH,
+			.max_width = W5_MAX_DEC_PIC_WIDTH,
+			.step_width = W5_DEC_CODEC_STEP_WIDTH,
+			.min_height = W5_MIN_DEC_PIC_8_HEIGHT,
+			.max_height = W5_MAX_DEC_PIC_HEIGHT,
+			.step_height = W5_DEC_CODEC_STEP_HEIGHT,
+		},
+		{
+			.min_width = W5_MIN_DEC_PIC_32_WIDTH,
+			.max_width = W5_MAX_DEC_PIC_WIDTH,
+			.step_width = W5_DEC_CODEC_STEP_WIDTH,
+			.min_height = W5_MIN_DEC_PIC_32_HEIGHT,
+			.max_height = W5_MAX_DEC_PIC_HEIGHT,
+			.step_height = W5_DEC_CODEC_STEP_HEIGHT,
+		}
+	},
+	[VPU_FMT_TYPE_RAW] = {
+		{
+			.min_width = W5_MIN_DEC_PIC_8_WIDTH,
+			.max_width = W5_MAX_DEC_PIC_WIDTH,
+			.step_width = W5_DEC_RAW_STEP_WIDTH,
+			.min_height = W5_MIN_DEC_PIC_8_HEIGHT,
+			.max_height = W5_MAX_DEC_PIC_HEIGHT,
+			.step_height = W5_DEC_RAW_STEP_HEIGHT,
+		}
+	},
+};
+
 static const struct vpu_format dec_fmt_list[FMT_TYPES][MAX_FMTS] = {
 	[VPU_FMT_TYPE_CODEC] = {
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_CODEC][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_H264,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_CODEC][1],
 		},
 	},
 	[VPU_FMT_TYPE_RAW] = {
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422P,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422M,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16M,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61M,
+			.v4l2_frmsize = &dec_frmsize[VPU_FMT_TYPE_RAW][0],
 		},
 	}
 };
 
-static const struct v4l2_frmsize_stepwise dec_frmsize[FMT_TYPES] = {
-	[VPU_FMT_TYPE_CODEC] = {
-		.min_width = W5_MIN_DEC_PIC_WIDTH,
-		.max_width = W5_MAX_DEC_PIC_WIDTH,
-		.step_width = W5_DEC_CODEC_STEP_WIDTH,
-		.min_height = W5_MIN_DEC_PIC_HEIGHT,
-		.max_height = W5_MAX_DEC_PIC_HEIGHT,
-		.step_height = W5_DEC_CODEC_STEP_HEIGHT,
-	},
-	[VPU_FMT_TYPE_RAW] = {
-		.min_width = W5_MIN_DEC_PIC_WIDTH,
-		.max_width = W5_MAX_DEC_PIC_WIDTH,
-		.step_width = W5_DEC_RAW_STEP_WIDTH,
-		.min_height = W5_MIN_DEC_PIC_HEIGHT,
-		.max_height = W5_MAX_DEC_PIC_HEIGHT,
-		.step_height = W5_DEC_RAW_STEP_HEIGHT,
-	},
-};
-
 /*
  * Make sure that the state switch is allowed and add logging for debugging
  * purposes
@@ -280,6 +306,8 @@ static int handle_dynamic_resolution_change(struct vpu_instance *inst)
 	}
 
 	if (p_dec_info->initial_info_obtained) {
+		const struct vpu_format *vpu_fmt;
+
 		inst->conf_win.left = initial_info->pic_crop_rect.left;
 		inst->conf_win.top = initial_info->pic_crop_rect.top;
 		inst->conf_win.width = initial_info->pic_width -
@@ -287,12 +315,25 @@ static int handle_dynamic_resolution_change(struct vpu_instance *inst)
 		inst->conf_win.height = initial_info->pic_height -
 			initial_info->pic_crop_rect.top - initial_info->pic_crop_rect.bottom;
 
-		wave5_update_pix_fmt(&inst->src_fmt, initial_info->pic_width,
-						     initial_info->pic_height,
-						     dec_frmsize[VPU_FMT_TYPE_CODEC]);
-		wave5_update_pix_fmt(&inst->dst_fmt, initial_info->pic_width,
-						     initial_info->pic_height,
-						     dec_frmsize[VPU_FMT_TYPE_RAW]);
+		vpu_fmt = wave5_find_vpu_fmt(inst->src_fmt.pixelformat, dec_fmt_list[VPU_FMT_TYPE_CODEC]);
+		if (!vpu_fmt)
+			return -EINVAL;
+
+		wave5_update_pix_fmt(&inst->src_fmt,
+					VPU_FMT_TYPE_CODEC,
+					initial_info->pic_width,
+					initial_info->pic_height,
+					vpu_fmt->v4l2_frmsize);
+
+		vpu_fmt = wave5_find_vpu_fmt(inst->dst_fmt.pixelformat, dec_fmt_list[VPU_FMT_TYPE_RAW]);
+		if (!vpu_fmt)
+			return -EINVAL;
+
+		wave5_update_pix_fmt(&inst->dst_fmt,
+					VPU_FMT_TYPE_RAW,
+					initial_info->pic_width,
+					initial_info->pic_height,
+					vpu_fmt->v4l2_frmsize);
 	}
 
 	v4l2_event_queue_fh(fh, &vpu_event_src_ch);
@@ -441,7 +482,7 @@ static int wave5_vpu_dec_enum_framesizes(struct file *f, void *fh, struct v4l2_f
 	}
 
 	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
-	fsize->stepwise = dec_frmsize[VPU_FMT_TYPE_CODEC];
+	fsize->stepwise = *vpu_fmt->v4l2_frmsize;
 
 	return 0;
 }
@@ -488,9 +529,10 @@ static int wave5_vpu_dec_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
 		height = inst->dst_fmt.height;
 	}
 
-	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
+	wave5_update_pix_fmt(&f->fmt.pix_mp, VPU_FMT_TYPE_RAW,
+					     width,
 					     height,
-					     dec_frmsize[VPU_FMT_TYPE_RAW]);
+					     vpu_fmt->v4l2_frmsize);
 	f->fmt.pix_mp.colorspace = inst->colorspace;
 	f->fmt.pix_mp.ycbcr_enc = inst->ycbcr_enc;
 	f->fmt.pix_mp.quantization = inst->quantization;
@@ -621,9 +663,10 @@ static int wave5_vpu_dec_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
 		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
 	}
 
-	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
+	wave5_update_pix_fmt(&f->fmt.pix_mp, VPU_FMT_TYPE_CODEC,
+					     width,
 					     height,
-					     dec_frmsize[VPU_FMT_TYPE_CODEC]);
+					     vpu_fmt->v4l2_frmsize);
 
 	return 0;
 }
@@ -631,6 +674,7 @@ static int wave5_vpu_dec_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
 static int wave5_vpu_dec_s_fmt_out(struct file *file, void *fh, struct v4l2_format *f)
 {
 	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
+	const struct vpu_format *vpu_fmt;
 	int i, ret;
 
 	dev_dbg(inst->dev->dev,
@@ -665,9 +709,14 @@ static int wave5_vpu_dec_s_fmt_out(struct file *file, void *fh, struct v4l2_form
 	inst->quantization = f->fmt.pix_mp.quantization;
 	inst->xfer_func = f->fmt.pix_mp.xfer_func;
 
-	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width,
+	vpu_fmt = wave5_find_vpu_fmt(inst->dst_fmt.pixelformat, dec_fmt_list[VPU_FMT_TYPE_RAW]);
+	if (!vpu_fmt)
+		return -EINVAL;
+
+	wave5_update_pix_fmt(&inst->dst_fmt, VPU_FMT_TYPE_RAW,
+					     f->fmt.pix_mp.width,
 					     f->fmt.pix_mp.height,
-					     dec_frmsize[VPU_FMT_TYPE_RAW]);
+					     vpu_fmt->v4l2_frmsize);
 
 	return 0;
 }
@@ -1429,14 +1478,16 @@ static void wave5_set_default_format(struct v4l2_pix_format_mplane *src_fmt,
 				     struct v4l2_pix_format_mplane *dst_fmt)
 {
 	src_fmt->pixelformat = dec_fmt_list[VPU_FMT_TYPE_CODEC][0].v4l2_pix_fmt;
-	wave5_update_pix_fmt(src_fmt, W5_DEF_DEC_PIC_WIDTH,
+	wave5_update_pix_fmt(src_fmt, VPU_FMT_TYPE_CODEC,
+				      W5_DEF_DEC_PIC_WIDTH,
 				      W5_DEF_DEC_PIC_HEIGHT,
-				      dec_frmsize[VPU_FMT_TYPE_CODEC]);
+				      &dec_frmsize[VPU_FMT_TYPE_CODEC][0]);
 
 	dst_fmt->pixelformat = dec_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
-	wave5_update_pix_fmt(dst_fmt, W5_DEF_DEC_PIC_WIDTH,
+	wave5_update_pix_fmt(dst_fmt, VPU_FMT_TYPE_RAW,
+				      W5_DEF_DEC_PIC_WIDTH,
 				      W5_DEF_DEC_PIC_HEIGHT,
-				      dec_frmsize[VPU_FMT_TYPE_RAW]);
+				      &dec_frmsize[VPU_FMT_TYPE_RAW][0]);
 }
 
 static int wave5_vpu_dec_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
index 240d1aa14..75d230df4 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
@@ -11,56 +11,64 @@
 #define VPU_ENC_DEV_NAME "C&M Wave5 VPU encoder"
 #define VPU_ENC_DRV_NAME "wave5-enc"
 
+static const struct v4l2_frmsize_stepwise enc_frmsize[FMT_TYPES] = {
+	[VPU_FMT_TYPE_CODEC] = {
+		.min_width = W5_MIN_ENC_PIC_WIDTH,
+		.max_width = W5_MAX_ENC_PIC_WIDTH,
+		.step_width = W5_ENC_CODEC_STEP_WIDTH,
+		.min_height = W5_MIN_ENC_PIC_HEIGHT,
+		.max_height = W5_MAX_ENC_PIC_HEIGHT,
+		.step_height = W5_ENC_CODEC_STEP_HEIGHT,
+	},
+	[VPU_FMT_TYPE_RAW] = {
+		.min_width = W5_MIN_ENC_PIC_WIDTH,
+		.max_width = W5_MAX_ENC_PIC_WIDTH,
+		.step_width = W5_ENC_RAW_STEP_WIDTH,
+		.min_height = W5_MIN_ENC_PIC_HEIGHT,
+		.max_height = W5_MAX_ENC_PIC_HEIGHT,
+		.step_height = W5_ENC_RAW_STEP_HEIGHT,
+	},
+};
+
 static const struct vpu_format enc_fmt_list[FMT_TYPES][MAX_FMTS] = {
 	[VPU_FMT_TYPE_CODEC] = {
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
+			.v4l2_frmsize = &enc_frmsize[VPU_FMT_TYPE_CODEC],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_H264,
+			.v4l2_frmsize = &enc_frmsize[VPU_FMT_TYPE_CODEC],
 		},
 	},
 	[VPU_FMT_TYPE_RAW] = {
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
+			.v4l2_frmsize = &enc_frmsize[VPU_FMT_TYPE_RAW],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
+			.v4l2_frmsize = &enc_frmsize[VPU_FMT_TYPE_RAW],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
+			.v4l2_frmsize = &enc_frmsize[VPU_FMT_TYPE_RAW],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
+			.v4l2_frmsize = &enc_frmsize[VPU_FMT_TYPE_RAW],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
+			.v4l2_frmsize = &enc_frmsize[VPU_FMT_TYPE_RAW],
 		},
 		{
 			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
+			.v4l2_frmsize = &enc_frmsize[VPU_FMT_TYPE_RAW],
 		},
 	}
 };
 
-static const struct v4l2_frmsize_stepwise enc_frmsize[FMT_TYPES] = {
-	[VPU_FMT_TYPE_CODEC] = {
-		.min_width = W5_MIN_ENC_PIC_WIDTH,
-		.max_width = W5_MAX_ENC_PIC_WIDTH,
-		.step_width = W5_ENC_CODEC_STEP_WIDTH,
-		.min_height = W5_MIN_ENC_PIC_HEIGHT,
-		.max_height = W5_MAX_ENC_PIC_HEIGHT,
-		.step_height = W5_ENC_CODEC_STEP_HEIGHT,
-	},
-	[VPU_FMT_TYPE_RAW] = {
-		.min_width = W5_MIN_ENC_PIC_WIDTH,
-		.max_width = W5_MAX_ENC_PIC_WIDTH,
-		.step_width = W5_ENC_RAW_STEP_WIDTH,
-		.min_height = W5_MIN_ENC_PIC_HEIGHT,
-		.max_height = W5_MAX_ENC_PIC_HEIGHT,
-		.step_height = W5_ENC_RAW_STEP_HEIGHT,
-	},
-};
-
 static int switch_state(struct vpu_instance *inst, enum vpu_instance_state state)
 {
 	switch (state) {
@@ -351,9 +359,10 @@ static int wave5_vpu_enc_try_fmt_cap(struct file *file, void *fh, struct v4l2_fo
 		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
 	}
 
-	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
+	wave5_update_pix_fmt(&f->fmt.pix_mp, VPU_FMT_TYPE_CODEC,
+					     width,
 					     height,
-					     enc_frmsize[VPU_FMT_TYPE_CODEC]);
+					     vpu_fmt->v4l2_frmsize);
 	f->fmt.pix_mp.colorspace = inst->colorspace;
 	f->fmt.pix_mp.ycbcr_enc = inst->ycbcr_enc;
 	f->fmt.pix_mp.quantization = inst->quantization;
@@ -458,9 +467,10 @@ static int wave5_vpu_enc_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
 		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
 	}
 
-	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
+	wave5_update_pix_fmt(&f->fmt.pix_mp, VPU_FMT_TYPE_RAW,
+					     width,
 					     height,
-					     enc_frmsize[VPU_FMT_TYPE_RAW]);
+					     vpu_fmt->v4l2_frmsize);
 
 	return 0;
 }
@@ -468,6 +478,7 @@ static int wave5_vpu_enc_try_fmt_out(struct file *file, void *fh, struct v4l2_fo
 static int wave5_vpu_enc_s_fmt_out(struct file *file, void *fh, struct v4l2_format *f)
 {
 	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
+	const struct vpu_format *vpu_fmt;
 	int i, ret;
 
 	dev_dbg(inst->dev->dev, "%s: fourcc: %u width: %u height: %u num_planes: %u field: %u\n",
@@ -507,9 +518,14 @@ static int wave5_vpu_enc_s_fmt_out(struct file *file, void *fh, struct v4l2_form
 	inst->quantization = f->fmt.pix_mp.quantization;
 	inst->xfer_func = f->fmt.pix_mp.xfer_func;
 
-	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width,
+	vpu_fmt = wave5_find_vpu_fmt(inst->dst_fmt.pixelformat, enc_fmt_list[VPU_FMT_TYPE_CODEC]);
+	if (!vpu_fmt)
+		return -EINVAL;
+
+	wave5_update_pix_fmt(&inst->dst_fmt, VPU_FMT_TYPE_CODEC,
+					     f->fmt.pix_mp.width,
 					     f->fmt.pix_mp.height,
-					     enc_frmsize[VPU_FMT_TYPE_CODEC]);
+					     vpu_fmt->v4l2_frmsize);
 	inst->conf_win.width = inst->dst_fmt.width;
 	inst->conf_win.height = inst->dst_fmt.height;
 
@@ -1407,14 +1423,16 @@ static void wave5_set_default_format(struct v4l2_pix_format_mplane *src_fmt,
 				     struct v4l2_pix_format_mplane *dst_fmt)
 {
 	src_fmt->pixelformat = enc_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
-	wave5_update_pix_fmt(src_fmt, W5_DEF_ENC_PIC_WIDTH,
+	wave5_update_pix_fmt(src_fmt, VPU_FMT_TYPE_RAW,
+				      W5_DEF_ENC_PIC_WIDTH,
 				      W5_DEF_ENC_PIC_HEIGHT,
-				      enc_frmsize[VPU_FMT_TYPE_RAW]);
+				      &enc_frmsize[VPU_FMT_TYPE_RAW]);
 
 	dst_fmt->pixelformat = enc_fmt_list[VPU_FMT_TYPE_CODEC][0].v4l2_pix_fmt;
-	wave5_update_pix_fmt(dst_fmt, W5_DEF_ENC_PIC_WIDTH,
+	wave5_update_pix_fmt(dst_fmt, VPU_FMT_TYPE_CODEC,
+				      W5_DEF_ENC_PIC_WIDTH,
 				      W5_DEF_ENC_PIC_HEIGHT,
-				      enc_frmsize[VPU_FMT_TYPE_CODEC]);
+				      &enc_frmsize[VPU_FMT_TYPE_CODEC]);
 }
 
 static int wave5_vpu_enc_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq)
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu.h b/drivers/media/platform/chips-media/wave5/wave5-vpu.h
index 691d6341f..384733255 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu.h
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu.h
@@ -38,6 +38,7 @@ enum vpu_fmt_type {
 
 struct vpu_format {
 	unsigned int v4l2_pix_fmt;
+	const struct v4l2_frmsize_stepwise *v4l2_frmsize;
 };
 
 static inline struct vpu_instance *wave5_to_vpu_inst(struct v4l2_fh *vfh)
diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h b/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
index e6a34ae70..8e11d93ca 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
@@ -32,8 +32,10 @@
 
 #define W5_DEF_DEC_PIC_WIDTH            720U
 #define W5_DEF_DEC_PIC_HEIGHT           480U
-#define W5_MIN_DEC_PIC_WIDTH            32U
-#define W5_MIN_DEC_PIC_HEIGHT           32U
+#define W5_MIN_DEC_PIC_8_WIDTH          8U
+#define W5_MIN_DEC_PIC_8_HEIGHT         8U
+#define W5_MIN_DEC_PIC_32_WIDTH         32U
+#define W5_MIN_DEC_PIC_32_HEIGHT        32U
 #define W5_MAX_DEC_PIC_WIDTH            8192U
 #define W5_MAX_DEC_PIC_HEIGHT           4320U
 #define W5_DEC_CODEC_STEP_WIDTH         1U

> -----Original Message-----
> From: jackson.lee <jackson.lee@chipsnmedia.com>
> Sent: Monday, April 22, 2024 11:18 AM
> To: Nicolas Dufresne <nicolas@ndufresne.ca>; mchehab@kernel.org;
> sebastian.fricke@collabora.com
> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>; lafley.kim
> <lafley.kim@chipsnmedia.com>; b-brnich@ti.com
> Subject: RE: [RESEND PATCH v2 3/4] media: chips-media: wave5: Use helpers to
> calculate bytesperline and sizeimage.
> 
> Hi Nicolas
> 
> > -----Original Message-----
> > From: Nicolas Dufresne <nicolas@ndufresne.ca>
> > Sent: Friday, April 19, 2024 6:03 AM
> > To: jackson.lee <jackson.lee@chipsnmedia.com>; mchehab@kernel.org;
> > sebastian.fricke@collabora.com
> > Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> > hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>; lafley.kim
> > <lafley.kim@chipsnmedia.com>; b-brnich@ti.com
> > Subject: Re: [RESEND PATCH v2 3/4] media: chips-media: wave5: Use
> > helpers to calculate bytesperline and sizeimage.
> >
> > Hi,
> >
> > very nice cleanup.
> >
> > Le lundi 11 mars 2024 à 19:56 +0900, jackson.lee a écrit :
> > > From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> > >
> > > Use v4l2-common helper functions to calculate bytesperline and
> > > sizeimage,
> > instead of calculating in a wave5 driver directly.
> > > In case of raw(YUV) v4l2_pix_format, the wave5 driver updates
> > v4l2_pix_format_mplane struct through v4l2_fill_pixfmt_mp() function.
> > > Encoder and Decoder need same bytesperline and sizeimage values for
> > > same
> > v4l2_pix_format.
> > > So, a wave5_update_pix_fmt is refactored to support both together.
> > >
> > > Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
> > > Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
> > > ---
> > >  .../platform/chips-media/wave5/wave5-helper.c |  24 ++
> > >  .../platform/chips-media/wave5/wave5-helper.h |   4 +
> > >  .../chips-media/wave5/wave5-vpu-dec.c         | 245 ++++--------------
> > >  .../chips-media/wave5/wave5-vpu-enc.c         | 179 +++++--------
> > >  .../platform/chips-media/wave5/wave5-vpu.h    |   4 -
> > >  .../chips-media/wave5/wave5-vpuconfig.h       |  25 +-
> > >  6 files changed, 164 insertions(+), 317 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/chips-media/wave5/wave5-helper.c
> > > b/drivers/media/platform/chips-media/wave5/wave5-helper.c
> > > index 8433ecab230c..53cad4d17aa7 100644
> > > --- a/drivers/media/platform/chips-media/wave5/wave5-helper.c
> > > +++ b/drivers/media/platform/chips-media/wave5/wave5-helper.c
> > > @@ -7,6 +7,8 @@
> > >
> > >  #include "wave5-helper.h"
> > >
> > > +#define DEFAULT_BS_SIZE(width, height) ((width) * (height) / 8 * 3)
> > > +
> > >  const char *state_to_str(enum vpu_instance_state state)  {
> > >  	switch (state) {
> > > @@ -211,3 +213,25 @@ void wave5_return_bufs(struct vb2_queue *q, u32
> state)
> > >  		v4l2_m2m_buf_done(vbuf, state);
> > >  	}
> > >  }
> > > +
> > > +void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
> > > +			  unsigned int width,
> > > +			  unsigned int height,
> > > +			  const struct v4l2_frmsize_stepwise frmsize) {
> > > +	v4l2_apply_frmsize_constraints(&width, &height, &frmsize);
> > > +
> > > +	if (pix_mp->pixelformat == V4L2_PIX_FMT_HEVC ||
> > > +	    pix_mp->pixelformat == V4L2_PIX_FMT_H264) {
> >
> > I notice the caller already index the frmsize between RAW and CODEC.
> > I'd say pass the information as an argument so we don't have to update
> > code later when more codecs are added.
> 
> What you are saying is this ?
> The caller already knows pixel format type if it is codec or raw type.
> So I added a variable to describe a pixel format type.
> 
>      void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp, int
> pix_fmt_type, // add pix_fmt_type
> 			  unsigned int width,
> 			  unsigned int height,
> 			  const struct v4l2_frmsize_stepwise frmsize)
> 
> 		if (pix_fmt_type == VPU_FMT_TYPE_CODEC) {
> 
> 		}
> 		else {
> 
> 		}
> 
> >
> > > +		pix_mp->width = width;
> > > +		pix_mp->height = height;
> > > +		pix_mp->num_planes = 1;
> > > +		pix_mp->plane_fmt[0].bytesperline = 0;
> > > +		pix_mp->plane_fmt[0].sizeimage = max(DEFAULT_BS_SIZE(width,
> > height),
> > > +						     pix_mp->plane_fmt[0].sizeimage);
> > > +	} else {
> > > +		v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat, width, height);
> > > +	}
> > > +	pix_mp->flags = 0;
> > > +	pix_mp->field = V4L2_FIELD_NONE;
> > > +}
> > > diff --git a/drivers/media/platform/chips-media/wave5/wave5-helper.h
> > > b/drivers/media/platform/chips-media/wave5/wave5-helper.h
> > > index 6cee1c14d3ce..a8ecd1920207 100644
> > > --- a/drivers/media/platform/chips-media/wave5/wave5-helper.h
> > > +++ b/drivers/media/platform/chips-media/wave5/wave5-helper.h
> > > @@ -28,4 +28,8 @@ const struct vpu_format
> > *wave5_find_vpu_fmt_by_idx(unsigned int idx,
> > >  						   const struct vpu_format
> > fmt_list[MAX_FMTS]);  enum wave_std
> > > wave5_to_vpu_std(unsigned int v4l2_pix_fmt, enum vpu_instance_type
> > > type);  void wave5_return_bufs(struct vb2_queue *q, u32 state);
> > > +void wave5_update_pix_fmt(struct v4l2_pix_format_mplane *pix_mp,
> > > +			  unsigned int width,
> > > +			  unsigned int height,
> > > +			  const struct v4l2_frmsize_stepwise frmsize);
> > >  #endif
> > > diff --git
> > > a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> > > b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> > > index a199877c643b..43d0a21193dd 100644
> > > --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> > > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> > > @@ -11,115 +11,74 @@
> > >  #define VPU_DEC_DEV_NAME "C&M Wave5 VPU decoder"
> > >  #define VPU_DEC_DRV_NAME "wave5-dec"
> > >
> > > -#define DEFAULT_SRC_SIZE(width, height) ({			\
> > > -	(width) * (height) / 8 * 3;					\
> > > -})
> > > -
> > >  static const struct vpu_format dec_fmt_list[FMT_TYPES][MAX_FMTS] = {
> > >  	[VPU_FMT_TYPE_CODEC] = {
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_H264,
> > > -			.max_width = 8192,
> > > -			.min_width = 32,
> > > -			.max_height = 4320,
> > > -			.min_height = 32,
> > >  		},
> >
> > I notice that min/max was different per codec, but in this patch you
> > actually make them both identical. Is that an error, otherwise the
> > change should be underlined in the commit message.
> >
> > If you really need per codec granularity, you could define the common
> > frmsize array before, and set a pointer here into the static array.
> > This way you won't have to type the same thing over and over, but
> > still have per codec granularity.
> >
> > >  	},
> > >  	[VPU_FMT_TYPE_RAW] = {
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422P,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422M,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16M,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61M,
> > > -			.max_width = 8192,
> > > -			.min_width = 8,
> > > -			.max_height = 4320,
> > > -			.min_height = 8,
> > >  		},
> > >  	}
> > >  };
> > >
> > > +static const struct v4l2_frmsize_stepwise dec_frmsize[FMT_TYPES] = {
> > > +	[VPU_FMT_TYPE_CODEC] = {
> > > +		.min_width = W5_MIN_DEC_PIC_WIDTH,
> > > +		.max_width = W5_MAX_DEC_PIC_WIDTH,
> > > +		.step_width = W5_DEC_CODEC_STEP_WIDTH,
> > > +		.min_height = W5_MIN_DEC_PIC_HEIGHT,
> > > +		.max_height = W5_MAX_DEC_PIC_HEIGHT,
> > > +		.step_height = W5_DEC_CODEC_STEP_HEIGHT,
> > > +	},
> > > +	[VPU_FMT_TYPE_RAW] = {
> > > +		.min_width = W5_MIN_DEC_PIC_WIDTH,
> > > +		.max_width = W5_MAX_DEC_PIC_WIDTH,
> > > +		.step_width = W5_DEC_RAW_STEP_WIDTH,
> > > +		.min_height = W5_MIN_DEC_PIC_HEIGHT,
> > > +		.max_height = W5_MAX_DEC_PIC_HEIGHT,
> > > +		.step_height = W5_DEC_RAW_STEP_HEIGHT,
> > > +	},
> > > +};
> > > +
> > >  /*
> > >   * Make sure that the state switch is allowed and add logging for
> > debugging
> > >   * purposes
> > > @@ -234,74 +193,6 @@ static void wave5_handle_src_buffer(struct
> > vpu_instance *inst, dma_addr_t rd_ptr
> > >  	inst->remaining_consumed_bytes = consumed_bytes;  }
> > >
> > > -static void wave5_update_pix_fmt(struct v4l2_pix_format_mplane
> > > *pix_mp,
> > unsigned int width,
> > > -				 unsigned int height)
> > > -{
> > > -	switch (pix_mp->pixelformat) {
> > > -	case V4L2_PIX_FMT_YUV420:
> > > -	case V4L2_PIX_FMT_NV12:
> > > -	case V4L2_PIX_FMT_NV21:
> > > -		pix_mp->width = round_up(width, 32);
> > > -		pix_mp->height = round_up(height, 16);
> > > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[0].sizeimage = width * height * 3 / 2;
> > > -		break;
> > > -	case V4L2_PIX_FMT_YUV422P:
> > > -	case V4L2_PIX_FMT_NV16:
> > > -	case V4L2_PIX_FMT_NV61:
> > > -		pix_mp->width = round_up(width, 32);
> > > -		pix_mp->height = round_up(height, 16);
> > > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[0].sizeimage = width * height * 2;
> > > -		break;
> > > -	case V4L2_PIX_FMT_YUV420M:
> > > -		pix_mp->width = round_up(width, 32);
> > > -		pix_mp->height = round_up(height, 16);
> > > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[0].sizeimage = width * height;
> > > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
> > > -		pix_mp->plane_fmt[1].sizeimage = width * height / 4;
> > > -		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
> > > -		pix_mp->plane_fmt[2].sizeimage = width * height / 4;
> > > -		break;
> > > -	case V4L2_PIX_FMT_NV12M:
> > > -	case V4L2_PIX_FMT_NV21M:
> > > -		pix_mp->width = round_up(width, 32);
> > > -		pix_mp->height = round_up(height, 16);
> > > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[0].sizeimage = width * height;
> > > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[1].sizeimage = width * height / 2;
> > > -		break;
> > > -	case V4L2_PIX_FMT_YUV422M:
> > > -		pix_mp->width = round_up(width, 32);
> > > -		pix_mp->height = round_up(height, 16);
> > > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[0].sizeimage = width * height;
> > > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
> > > -		pix_mp->plane_fmt[1].sizeimage = width * height / 2;
> > > -		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
> > > -		pix_mp->plane_fmt[2].sizeimage = width * height / 2;
> > > -		break;
> > > -	case V4L2_PIX_FMT_NV16M:
> > > -	case V4L2_PIX_FMT_NV61M:
> > > -		pix_mp->width = round_up(width, 32);
> > > -		pix_mp->height = round_up(height, 16);
> > > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[0].sizeimage = width * height;
> > > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[1].sizeimage = width * height;
> > > -		break;
> > > -	default:
> > > -		pix_mp->width = width;
> > > -		pix_mp->height = height;
> > > -		pix_mp->plane_fmt[0].bytesperline = 0;
> > > -		pix_mp->plane_fmt[0].sizeimage = max(DEFAULT_SRC_SIZE(width,
> > height),
> > > -						     pix_mp->plane_fmt[0].sizeimage);
> > > -		break;
> > > -	}
> > > -}
> > > -
> > >  static int start_decode(struct vpu_instance *inst, u32 *fail_res)  {
> > >  	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; @@ -397,9
> > > +288,11 @@ static int handle_dynamic_resolution_change(struct
> > > +vpu_instance
> > *inst)
> > >  			initial_info->pic_crop_rect.top -
> > > initial_info->pic_crop_rect.bottom;
> > >
> > >  		wave5_update_pix_fmt(&inst->src_fmt, initial_info->pic_width,
> > > -				     initial_info->pic_height);
> > > +						     initial_info->pic_height,
> > > +
> > dec_frmsize[VPU_FMT_TYPE_CODEC]);
> > >  		wave5_update_pix_fmt(&inst->dst_fmt, initial_info->pic_width,
> > > -				     initial_info->pic_height);
> > > +						     initial_info->pic_height,
> > > +						     dec_frmsize[VPU_FMT_TYPE_RAW]);
> > >  	}
> > >
> > >  	v4l2_event_queue_fh(fh, &vpu_event_src_ch); @@ -548,12 +441,7 @@
> > > static int wave5_vpu_dec_enum_framesizes(struct file *f, void *fh,
> > > struct
> > v4l2_f
> > >  	}
> > >
> > >  	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> > > -	fsize->stepwise.min_width = vpu_fmt->min_width;
> > > -	fsize->stepwise.max_width = vpu_fmt->max_width;
> > > -	fsize->stepwise.step_width = 1;
> > > -	fsize->stepwise.min_height = vpu_fmt->min_height;
> > > -	fsize->stepwise.max_height = vpu_fmt->max_height;
> > > -	fsize->stepwise.step_height = 1;
> > > +	fsize->stepwise = dec_frmsize[VPU_FMT_TYPE_CODEC];
> > >
> > >  	return 0;
> > >  }
> > > @@ -589,14 +477,10 @@ static int wave5_vpu_dec_try_fmt_cap(struct
> > > file
> > *file, void *fh, struct v4l2_fo
> > >  		width = inst->dst_fmt.width;
> > >  		height = inst->dst_fmt.height;
> > >  		f->fmt.pix_mp.pixelformat = inst->dst_fmt.pixelformat;
> > > -		f->fmt.pix_mp.num_planes = inst->dst_fmt.num_planes;
> > >  	} else {
> > > -		const struct v4l2_format_info *info = v4l2_format_info(vpu_fmt-
> > >v4l2_pix_fmt);
> > > -
> > > -		width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width, vpu_fmt-
> > >max_width);
> > > -		height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height,
> > vpu_fmt->max_height);
> > > +		width = f->fmt.pix_mp.width;
> > > +		height = f->fmt.pix_mp.height;
> > >  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> > > -		f->fmt.pix_mp.num_planes = info->mem_planes;
> > >  	}
> > >
> > >  	if (p_dec_info->initial_info_obtained) { @@ -604,9 +488,9 @@
> > > static int wave5_vpu_dec_try_fmt_cap(struct file *file, void *fh, struct
> v4l2_fo
> > >  		height = inst->dst_fmt.height;
> > >  	}
> > >
> > > -	wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
> > > -	f->fmt.pix_mp.flags = 0;
> > > -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > > +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> > > +					     height,
> > > +					     dec_frmsize[VPU_FMT_TYPE_RAW]);
> > >  	f->fmt.pix_mp.colorspace = inst->colorspace;
> > >  	f->fmt.pix_mp.ycbcr_enc = inst->ycbcr_enc;
> > >  	f->fmt.pix_mp.quantization = inst->quantization; @@ -719,6 +603,7
> > > @@ static int wave5_vpu_dec_try_fmt_out(struct file *file, void *fh,
> > > struct v4l2_fo  {
> > >  	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
> > >  	const struct vpu_format *vpu_fmt;
> > > +	int width, height;
> > >
> > >  	dev_dbg(inst->dev->dev,
> > >  		"%s: fourcc: %u width: %u height: %u num_planes: %u
> > colorspace: %u
> > > field: %u\n", @@ -727,20 +612,18 @@ static int
> > > wave5_vpu_dec_try_fmt_out(struct file *file, void *fh, struct
> > > v4l2_fo
> > >
> > >  	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat,
> > dec_fmt_list[VPU_FMT_TYPE_CODEC]);
> > >  	if (!vpu_fmt) {
> > > +		width = inst->src_fmt.width;
> > > +		height = inst->src_fmt.height;
> > >  		f->fmt.pix_mp.pixelformat = inst->src_fmt.pixelformat;
> > > -		f->fmt.pix_mp.num_planes = inst->src_fmt.num_planes;
> > > -		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->src_fmt.width, inst-
> > >src_fmt.height);
> > >  	} else {
> > > -		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width,
> > vpu_fmt->max_width);
> > > -		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height,
> > vpu_fmt->max_height);
> > > -
> > > +		width = f->fmt.pix_mp.width;
> > > +		height = f->fmt.pix_mp.height;
> > >  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> > > -		f->fmt.pix_mp.num_planes = 1;
> > > -		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
> > >  	}
> > >
> > > -	f->fmt.pix_mp.flags = 0;
> > > -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > > +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> > > +					     height,
> > > +					     dec_frmsize[VPU_FMT_TYPE_CODEC]);
> > >
> > >  	return 0;
> > >  }
> > > @@ -782,7 +665,9 @@ static int wave5_vpu_dec_s_fmt_out(struct file
> > > *file,
> > void *fh, struct v4l2_form
> > >  	inst->quantization = f->fmt.pix_mp.quantization;
> > >  	inst->xfer_func = f->fmt.pix_mp.xfer_func;
> > >
> > > -	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width, f-
> > >fmt.pix_mp.height);
> > > +	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width,
> > > +					     f->fmt.pix_mp.height,
> > > +					     dec_frmsize[VPU_FMT_TYPE_RAW]);
> > >
> > >  	return 0;
> > >  }
> > > @@ -1005,6 +890,7 @@ static int wave5_vpu_dec_queue_setup(struct
> > > vb2_queue
> > *q, unsigned int *num_buff
> > >  	struct vpu_instance *inst = vb2_get_drv_priv(q);
> > >  	struct v4l2_pix_format_mplane inst_format =
> > >  		(q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) ? inst->src_fmt :
> > > inst->dst_fmt;
> > > +	unsigned int i;
> > >
> > >  	dev_dbg(inst->dev->dev, "%s: num_buffers: %u | num_planes: %u |
> > type: %u\n", __func__,
> > >  		*num_buffers, *num_planes, q->type); @@ -1018,31 +904,9 @@
> > static
> > > int wave5_vpu_dec_queue_setup(struct vb2_queue *q, unsigned int *num_buff
> > >  		if (*num_buffers < inst->fbc_buf_count)
> > >  			*num_buffers = inst->fbc_buf_count;
> > >
> > > -		if (*num_planes == 1) {
> > > -			if (inst->output_format == FORMAT_422)
> > > -				sizes[0] = inst_format.width * inst_format.height
> > * 2;
> > > -			else
> > > -				sizes[0] = inst_format.width * inst_format.height
> > * 3 / 2;
> > > -			dev_dbg(inst->dev->dev, "%s: size[0]: %u\n", __func__,
> > sizes[0]);
> > > -		} else if (*num_planes == 2) {
> > > -			sizes[0] = inst_format.width * inst_format.height;
> > > -			if (inst->output_format == FORMAT_422)
> > > -				sizes[1] = inst_format.width * inst_format.height;
> > > -			else
> > > -				sizes[1] = inst_format.width * inst_format.height
> > / 2;
> > > -			dev_dbg(inst->dev->dev, "%s: size[0]: %u |
> > size[1]: %u\n",
> > > -				__func__, sizes[0], sizes[1]);
> > > -		} else if (*num_planes == 3) {
> > > -			sizes[0] = inst_format.width * inst_format.height;
> > > -			if (inst->output_format == FORMAT_422) {
> > > -				sizes[1] = inst_format.width * inst_format.height
> > / 2;
> > > -				sizes[2] = inst_format.width * inst_format.height
> > / 2;
> > > -			} else {
> > > -				sizes[1] = inst_format.width * inst_format.height
> > / 4;
> > > -				sizes[2] = inst_format.width * inst_format.height
> > / 4;
> > > -			}
> > > -			dev_dbg(inst->dev->dev, "%s: size[0]: %u | size[1]: %u |
> > size[2]: %u\n",
> > > -				__func__, sizes[0], sizes[1], sizes[2]);
> > > +		for (i = 0; i < *num_planes; i++) {
> > > +			sizes[i] = inst_format.plane_fmt[i].sizeimage;
> > > +			dev_dbg(inst->dev->dev, "%s: size[%u]: %u\n", __func__,
> > i,
> > > +sizes[i]);
> > >  		}
> > >  	}
> > >
> > > @@ -1564,20 +1428,15 @@ static const struct vb2_ops
> > > wave5_vpu_dec_vb2_ops = {  static void
> > > wave5_set_default_format(struct
> > v4l2_pix_format_mplane *src_fmt,
> > >  				     struct v4l2_pix_format_mplane *dst_fmt)  {
> > > -	unsigned int dst_pix_fmt =
> > dec_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> > > -	const struct v4l2_format_info *dst_fmt_info =
> > v4l2_format_info(dst_pix_fmt);
> > > -
> > >  	src_fmt->pixelformat =
> > dec_fmt_list[VPU_FMT_TYPE_CODEC][0].v4l2_pix_fmt;
> > > -	src_fmt->field = V4L2_FIELD_NONE;
> > > -	src_fmt->flags = 0;
> > > -	src_fmt->num_planes = 1;
> > > -	wave5_update_pix_fmt(src_fmt, 720, 480);
> > > -
> > > -	dst_fmt->pixelformat = dst_pix_fmt;
> > > -	dst_fmt->field = V4L2_FIELD_NONE;
> > > -	dst_fmt->flags = 0;
> > > -	dst_fmt->num_planes = dst_fmt_info->mem_planes;
> > > -	wave5_update_pix_fmt(dst_fmt, 736, 480);
> > > +	wave5_update_pix_fmt(src_fmt, W5_DEF_DEC_PIC_WIDTH,
> > > +				      W5_DEF_DEC_PIC_HEIGHT,
> > > +				      dec_frmsize[VPU_FMT_TYPE_CODEC]);
> > > +
> > > +	dst_fmt->pixelformat = dec_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> > > +	wave5_update_pix_fmt(dst_fmt, W5_DEF_DEC_PIC_WIDTH,
> > > +				      W5_DEF_DEC_PIC_HEIGHT,
> > > +				      dec_frmsize[VPU_FMT_TYPE_RAW]);
> > >  }
> > >
> > >  static int wave5_vpu_dec_queue_init(void *priv, struct vb2_queue
> > > *src_vq, struct vb2_queue *dst_vq) diff --git
> > > a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > index 013e2bb37fbb..5a32bb138158 100644
> > > --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > @@ -15,65 +15,52 @@ static const struct vpu_format
> > enc_fmt_list[FMT_TYPES][MAX_FMTS] = {
> > >  	[VPU_FMT_TYPE_CODEC] = {
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_HEVC,
> > > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_H264,
> > > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > >  		},
> > >  	},
> > >  	[VPU_FMT_TYPE_RAW] = {
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420,
> > > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12,
> > > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21,
> > > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV420M,
> > > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV12M,
> > > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > >  		},
> > >  		{
> > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
> > > -			.max_width = W5_MAX_ENC_PIC_WIDTH,
> > > -			.min_width = W5_MIN_ENC_PIC_WIDTH,
> > > -			.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > > -			.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > >  		},
> > >  	}
> > >  };
> > >
> > > +static const struct v4l2_frmsize_stepwise enc_frmsize[FMT_TYPES] = {
> > > +	[VPU_FMT_TYPE_CODEC] = {
> > > +		.min_width = W5_MIN_ENC_PIC_WIDTH,
> > > +		.max_width = W5_MAX_ENC_PIC_WIDTH,
> > > +		.step_width = W5_ENC_CODEC_STEP_WIDTH,
> > > +		.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > > +		.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > > +		.step_height = W5_ENC_CODEC_STEP_HEIGHT,
> > > +	},
> > > +	[VPU_FMT_TYPE_RAW] = {
> > > +		.min_width = W5_MIN_ENC_PIC_WIDTH,
> > > +		.max_width = W5_MAX_ENC_PIC_WIDTH,
> > > +		.step_width = W5_ENC_RAW_STEP_WIDTH,
> > > +		.min_height = W5_MIN_ENC_PIC_HEIGHT,
> > > +		.max_height = W5_MAX_ENC_PIC_HEIGHT,
> > > +		.step_height = W5_ENC_RAW_STEP_HEIGHT,
> > > +	},
> > > +};
> > > +
> > >  static int switch_state(struct vpu_instance *inst, enum
> > > vpu_instance_state state)  {
> > >  	switch (state) {
> > > @@ -106,46 +93,6 @@ static int switch_state(struct vpu_instance
> > > *inst, enum
> > vpu_instance_state state
> > >  	return -EINVAL;
> > >  }
> > >
> > > -static void wave5_update_pix_fmt(struct v4l2_pix_format_mplane
> > > *pix_mp,
> > unsigned int width,
> > > -				 unsigned int height)
> > > -{
> > > -	switch (pix_mp->pixelformat) {
> > > -	case V4L2_PIX_FMT_YUV420:
> > > -	case V4L2_PIX_FMT_NV12:
> > > -	case V4L2_PIX_FMT_NV21:
> > > -		pix_mp->width = width;
> > > -		pix_mp->height = height;
> > > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height *
> > 3 / 2;
> > > -		break;
> > > -	case V4L2_PIX_FMT_YUV420M:
> > > -		pix_mp->width = width;
> > > -		pix_mp->height = height;
> > > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height;
> > > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32) / 2;
> > > -		pix_mp->plane_fmt[1].sizeimage = round_up(width, 32) * height /
> > 4;
> > > -		pix_mp->plane_fmt[2].bytesperline = round_up(width, 32) / 2;
> > > -		pix_mp->plane_fmt[2].sizeimage = round_up(width, 32) * height /
> > 4;
> > > -		break;
> > > -	case V4L2_PIX_FMT_NV12M:
> > > -	case V4L2_PIX_FMT_NV21M:
> > > -		pix_mp->width = width;
> > > -		pix_mp->height = height;
> > > -		pix_mp->plane_fmt[0].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[0].sizeimage = round_up(width, 32) * height;
> > > -		pix_mp->plane_fmt[1].bytesperline = round_up(width, 32);
> > > -		pix_mp->plane_fmt[1].sizeimage = round_up(width, 32) * height /
> > 2;
> > > -		break;
> > > -	default:
> > > -		pix_mp->width = width;
> > > -		pix_mp->height = height;
> > > -		pix_mp->plane_fmt[0].bytesperline = 0;
> > > -		pix_mp->plane_fmt[0].sizeimage = width * height / 8 * 3;
> > > -		break;
> > > -	}
> > > -}
> > > -
> > >  static int start_encode(struct vpu_instance *inst, u32 *fail_res)  {
> > >  	struct v4l2_m2m_ctx *m2m_ctx = inst->v4l2_fh.m2m_ctx; @@ -360,13
> > > +307,8 @@ static int wave5_vpu_enc_enum_framesizes(struct file *f,
> > > +void *fh,
> > struct v4l2_f
> > >  			return -EINVAL;
> > >  	}
> > >
> > > -	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> > > -	fsize->stepwise.min_width = vpu_fmt->min_width;
> > > -	fsize->stepwise.max_width = vpu_fmt->max_width;
> > > -	fsize->stepwise.step_width = 1;
> > > -	fsize->stepwise.min_height = vpu_fmt->min_height;
> > > -	fsize->stepwise.max_height = vpu_fmt->max_height;
> > > -	fsize->stepwise.step_height = 1;
> > > +	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > > +	fsize->stepwise = enc_frmsize[VPU_FMT_TYPE_CODEC];
> > >
> > >  	return 0;
> > >  }
> > > @@ -392,6 +334,7 @@ static int wave5_vpu_enc_try_fmt_cap(struct file
> > > *file, void *fh, struct v4l2_fo  {
> > >  	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
> > >  	const struct vpu_format *vpu_fmt;
> > > +	int width, height;
> > >
> > >  	dev_dbg(inst->dev->dev, "%s: fourcc: %u width: %u height: %u
> > num_planes: %u field: %u\n",
> > >  		__func__, f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width,
> > > f->fmt.pix_mp.height, @@ -399,20 +342,18 @@ static int
> > > wave5_vpu_enc_try_fmt_cap(struct file *file, void *fh, struct
> > > v4l2_fo
> > >
> > >  	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat,
> > enc_fmt_list[VPU_FMT_TYPE_CODEC]);
> > >  	if (!vpu_fmt) {
> > > +		width = inst->dst_fmt.width;
> > > +		height = inst->dst_fmt.height;
> > >  		f->fmt.pix_mp.pixelformat = inst->dst_fmt.pixelformat;
> > > -		f->fmt.pix_mp.num_planes = inst->dst_fmt.num_planes;
> > > -		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->dst_fmt.width, inst-
> > >dst_fmt.height);
> > >  	} else {
> > > -		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width,
> > vpu_fmt->max_width);
> > > -		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height,
> > vpu_fmt->max_height);
> > > -
> > > +		width = f->fmt.pix_mp.width;
> > > +		height = f->fmt.pix_mp.height;
> > >  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> > > -		f->fmt.pix_mp.num_planes = 1;
> > > -		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
> > >  	}
> > >
> > > -	f->fmt.pix_mp.flags = 0;
> > > -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > > +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> > > +					     height,
> > > +					     enc_frmsize[VPU_FMT_TYPE_CODEC]);
> > >  	f->fmt.pix_mp.colorspace = inst->colorspace;
> > >  	f->fmt.pix_mp.ycbcr_enc = inst->ycbcr_enc;
> > >  	f->fmt.pix_mp.quantization = inst->quantization; @@ -500,6 +441,7
> > > @@ static int wave5_vpu_enc_try_fmt_out(struct file *file, void *fh,
> > > struct v4l2_fo  {
> > >  	struct vpu_instance *inst = wave5_to_vpu_inst(fh);
> > >  	const struct vpu_format *vpu_fmt;
> > > +	int width, height;
> > >
> > >  	dev_dbg(inst->dev->dev, "%s: fourcc: %u width: %u height: %u
> > num_planes: %u field: %u\n",
> > >  		__func__, f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width,
> > > f->fmt.pix_mp.height, @@ -507,21 +449,18 @@ static int
> > > wave5_vpu_enc_try_fmt_out(struct file *file, void *fh, struct
> > > v4l2_fo
> > >
> > >  	vpu_fmt = wave5_find_vpu_fmt(f->fmt.pix_mp.pixelformat,
> > enc_fmt_list[VPU_FMT_TYPE_RAW]);
> > >  	if (!vpu_fmt) {
> > > +		width = inst->src_fmt.width;
> > > +		height = inst->src_fmt.height;
> > >  		f->fmt.pix_mp.pixelformat = inst->src_fmt.pixelformat;
> > > -		f->fmt.pix_mp.num_planes = inst->src_fmt.num_planes;
> > > -		wave5_update_pix_fmt(&f->fmt.pix_mp, inst->src_fmt.width, inst-
> > >src_fmt.height);
> > >  	} else {
> > > -		int width = clamp(f->fmt.pix_mp.width, vpu_fmt->min_width,
> > vpu_fmt->max_width);
> > > -		int height = clamp(f->fmt.pix_mp.height, vpu_fmt->min_height,
> > vpu_fmt->max_height);
> > > -		const struct v4l2_format_info *info = v4l2_format_info(vpu_fmt-
> > >v4l2_pix_fmt);
> > > -
> > > +		width = f->fmt.pix_mp.width;
> > > +		height = f->fmt.pix_mp.height;
> > >  		f->fmt.pix_mp.pixelformat = vpu_fmt->v4l2_pix_fmt;
> > > -		f->fmt.pix_mp.num_planes = info->mem_planes;
> > > -		wave5_update_pix_fmt(&f->fmt.pix_mp, width, height);
> > >  	}
> > >
> > > -	f->fmt.pix_mp.flags = 0;
> > > -	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
> > > +	wave5_update_pix_fmt(&f->fmt.pix_mp, width,
> > > +					     height,
> > > +					     enc_frmsize[VPU_FMT_TYPE_RAW]);
> > >
> > >  	return 0;
> > >  }
> > > @@ -568,7 +507,11 @@ static int wave5_vpu_enc_s_fmt_out(struct file
> > > *file,
> > void *fh, struct v4l2_form
> > >  	inst->quantization = f->fmt.pix_mp.quantization;
> > >  	inst->xfer_func = f->fmt.pix_mp.xfer_func;
> > >
> > > -	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width, f-
> > >fmt.pix_mp.height);
> > > +	wave5_update_pix_fmt(&inst->dst_fmt, f->fmt.pix_mp.width,
> > > +					     f->fmt.pix_mp.height,
> > > +					     enc_frmsize[VPU_FMT_TYPE_CODEC]);
> > > +	inst->conf_win.width = inst->dst_fmt.width;
> > > +	inst->conf_win.height = inst->dst_fmt.height;
> > >
> > >  	return 0;
> > >  }
> > > @@ -584,12 +527,17 @@ static int wave5_vpu_enc_g_selection(struct
> > > file
> > *file, void *fh, struct v4l2_se
> > >  	switch (s->target) {
> > >  	case V4L2_SEL_TGT_CROP_DEFAULT:
> > >  	case V4L2_SEL_TGT_CROP_BOUNDS:
> > > -	case V4L2_SEL_TGT_CROP:
> > >  		s->r.left = 0;
> > >  		s->r.top = 0;
> > >  		s->r.width = inst->dst_fmt.width;
> > >  		s->r.height = inst->dst_fmt.height;
> > >  		break;
> > > +	case V4L2_SEL_TGT_CROP:
> > > +		s->r.left = 0;
> > > +		s->r.top = 0;
> > > +		s->r.width = inst->conf_win.width;
> > > +		s->r.height = inst->conf_win.height;
> > > +		break;
> > >  	default:
> > >  		return -EINVAL;
> > >  	}
> > > @@ -612,8 +560,10 @@ static int wave5_vpu_enc_s_selection(struct
> > > file *file, void *fh, struct v4l2_se
> > >
> > >  	s->r.left = 0;
> > >  	s->r.top = 0;
> > > -	s->r.width = inst->src_fmt.width;
> > > -	s->r.height = inst->src_fmt.height;
> > > +	s->r.width = min(s->r.width, inst->dst_fmt.width);
> > > +	s->r.height = min(s->r.height, inst->dst_fmt.height);
> > > +
> > > +	inst->conf_win = s->r;
> > >
> > >  	return 0;
> > >  }
> > > @@ -1151,8 +1101,8 @@ static void wave5_set_enc_openparam(struct
> > enc_open_param *open_param,
> > >  	open_param->wave_param.lambda_scaling_enable = 1;
> > >
> > >  	open_param->line_buf_int_en = true;
> > > -	open_param->pic_width = inst->dst_fmt.width;
> > > -	open_param->pic_height = inst->dst_fmt.height;
> > > +	open_param->pic_width = inst->conf_win.width;
> > > +	open_param->pic_height = inst->conf_win.height;
> > >  	open_param->frame_rate_info = inst->frame_rate;
> > >  	open_param->rc_enable = inst->rc_enable;
> > >  	if (inst->rc_enable) {
> > > @@ -1456,20 +1406,15 @@ static const struct vb2_ops
> > > wave5_vpu_enc_vb2_ops = {  static void
> > > wave5_set_default_format(struct
> > v4l2_pix_format_mplane *src_fmt,
> > >  				     struct v4l2_pix_format_mplane *dst_fmt)  {
> > > -	unsigned int src_pix_fmt =
> > enc_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> > > -	const struct v4l2_format_info *src_fmt_info =
> > v4l2_format_info(src_pix_fmt);
> > > -
> > > -	src_fmt->pixelformat = src_pix_fmt;
> > > -	src_fmt->field = V4L2_FIELD_NONE;
> > > -	src_fmt->flags = 0;
> > > -	src_fmt->num_planes = src_fmt_info->mem_planes;
> > > -	wave5_update_pix_fmt(src_fmt, 416, 240);
> > > +	src_fmt->pixelformat = enc_fmt_list[VPU_FMT_TYPE_RAW][0].v4l2_pix_fmt;
> > > +	wave5_update_pix_fmt(src_fmt, W5_DEF_ENC_PIC_WIDTH,
> > > +				      W5_DEF_ENC_PIC_HEIGHT,
> > > +				      enc_frmsize[VPU_FMT_TYPE_RAW]);
> > >
> > >  	dst_fmt->pixelformat =
> > enc_fmt_list[VPU_FMT_TYPE_CODEC][0].v4l2_pix_fmt;
> > > -	dst_fmt->field = V4L2_FIELD_NONE;
> > > -	dst_fmt->flags = 0;
> > > -	dst_fmt->num_planes = 1;
> > > -	wave5_update_pix_fmt(dst_fmt, 416, 240);
> > > +	wave5_update_pix_fmt(dst_fmt, W5_DEF_ENC_PIC_WIDTH,
> > > +				      W5_DEF_ENC_PIC_HEIGHT,
> > > +				      enc_frmsize[VPU_FMT_TYPE_CODEC]);
> > >  }
> > >
> > >  static int wave5_vpu_enc_queue_init(void *priv, struct vb2_queue
> > > *src_vq, struct vb2_queue *dst_vq) @@ -1734,6 +1679,8 @@ static int
> > wave5_vpu_open_enc(struct file *filp)
> > >  	v4l2_ctrl_handler_setup(v4l2_ctrl_hdl);
> > >
> > >  	wave5_set_default_format(&inst->src_fmt, &inst->dst_fmt);
> > > +	inst->conf_win.width = inst->dst_fmt.width;
> > > +	inst->conf_win.height = inst->dst_fmt.height;
> > >  	inst->colorspace = V4L2_COLORSPACE_REC709;
> > >  	inst->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> > >  	inst->quantization = V4L2_QUANTIZATION_DEFAULT; diff --git
> > > a/drivers/media/platform/chips-media/wave5/wave5-vpu.h
> > > b/drivers/media/platform/chips-media/wave5/wave5-vpu.h
> > > index 32b7fd3730b5..691d6341fcda 100644
> > > --- a/drivers/media/platform/chips-media/wave5/wave5-vpu.h
> > > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu.h
> > > @@ -38,10 +38,6 @@ enum vpu_fmt_type {
> > >
> > >  struct vpu_format {
> > >  	unsigned int v4l2_pix_fmt;
> > > -	unsigned int max_width;
> > > -	unsigned int min_width;
> > > -	unsigned int max_height;
> > > -	unsigned int min_height;
> > >  };
> > >
> > >  static inline struct vpu_instance *wave5_to_vpu_inst(struct v4l2_fh
> > > *vfh) diff --git
> > > a/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> > > b/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> > > index d9751eedb0f9..e6a34ae7084e 100644
> > > --- a/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> > > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> > > @@ -30,10 +30,27 @@
> > >
> > >  #define MAX_NUM_INSTANCE                32
> > >
> > > -#define W5_MIN_ENC_PIC_WIDTH            256
> > > -#define W5_MIN_ENC_PIC_HEIGHT           128
> > > -#define W5_MAX_ENC_PIC_WIDTH            8192
> > > -#define W5_MAX_ENC_PIC_HEIGHT           8192
> > > +#define W5_DEF_DEC_PIC_WIDTH            720U
> > > +#define W5_DEF_DEC_PIC_HEIGHT           480U
> > > +#define W5_MIN_DEC_PIC_WIDTH            32U
> > > +#define W5_MIN_DEC_PIC_HEIGHT           32U
> > > +#define W5_MAX_DEC_PIC_WIDTH            8192U
> > > +#define W5_MAX_DEC_PIC_HEIGHT           4320U
> > > +#define W5_DEC_CODEC_STEP_WIDTH         1U
> > > +#define W5_DEC_CODEC_STEP_HEIGHT        1U
> > > +#define W5_DEC_RAW_STEP_WIDTH           32U
> > > +#define W5_DEC_RAW_STEP_HEIGHT          16U
> > > +
> > > +#define W5_DEF_ENC_PIC_WIDTH            416U
> > > +#define W5_DEF_ENC_PIC_HEIGHT           240U
> > > +#define W5_MIN_ENC_PIC_WIDTH            256U
> > > +#define W5_MIN_ENC_PIC_HEIGHT           128U
> > > +#define W5_MAX_ENC_PIC_WIDTH            8192U
> > > +#define W5_MAX_ENC_PIC_HEIGHT           8192U
> > > +#define W5_ENC_CODEC_STEP_WIDTH         8U
> > > +#define W5_ENC_CODEC_STEP_HEIGHT        8U
> > > +#define W5_ENC_RAW_STEP_WIDTH           32U
> > > +#define W5_ENC_RAW_STEP_HEIGHT          16U
> > >
> > >  //  application specific configuration
> > >  #define VPU_ENC_TIMEOUT                 60000
> >
> >
> > I'll have to give this one a second pass on next revision, but nice work.
> >
> > Nicolas

RE: [RESEND PATCH v2 4/4] media: chips-media: wave5: Support YUV422 raw pixel-formats on the encoder.

[jackson.lee]

Hi Nicolas

> -----Original Message-----
> From: Nicolas Dufresne <nicolas@ndufresne.ca>
> Sent: Tuesday, April 23, 2024 12:49 AM
> To: jackson.lee <jackson.lee@chipsnmedia.com>; mchehab@kernel.org;
> sebastian.fricke@collabora.com
> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>; lafley.kim
> <lafley.kim@chipsnmedia.com>; b-brnich@ti.com
> Subject: Re: [RESEND PATCH v2 4/4] media: chips-media: wave5: Support YUV422
> raw pixel-formats on the encoder.
> 
> Hi Jackson,
> 
> Le lundi 22 avril 2024 à 04:30 +0000, jackson.lee a écrit :
> > Hi Nicolas
> >
> >
> > > -----Original Message-----
> > > From: Nicolas Dufresne <nicolas@ndufresne.ca>
> > > Sent: Friday, April 19, 2024 6:06 AM
> > > To: jackson.lee <jackson.lee@chipsnmedia.com>; mchehab@kernel.org;
> > > sebastian.fricke@collabora.com
> > > Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> > > hverkuil@xs4all.nl; Nas Chung <nas.chung@chipsnmedia.com>;
> > > lafley.kim <lafley.kim@chipsnmedia.com>; b-brnich@ti.com
> > > Subject: Re: [RESEND PATCH v2 4/4] media: chips-media: wave5:
> > > Support YUV422 raw pixel-formats on the encoder.
> > >
> > > Le lundi 11 mars 2024 à 19:56 +0900, jackson.lee a écrit :
> > > > From: "Jackson.lee" <jackson.lee@chipsnmedia.com>
> > > >
> > > > Add support for the YUV422P, NV16, NV61, YUV422M, NV16M, NV61M raw
> > > > pixel-
> > > formats to the Wave5 encoder.
> > > > All these formats have a chroma subsampling ratio of 4:2:2 and
> > > > therefore
> > > require a new image size calculation as the driver previously only
> > > handled a ratio of 4:2:0.
> > > >
> > > > Signed-off-by: Jackson.lee <jackson.lee@chipsnmedia.com>
> > > > Signed-off-by: Nas Chung <nas.chung@chipsnmedia.com>
> > > > ---
> > > >  .../chips-media/wave5/wave5-vpu-enc.c         | 59 +++++++++++++++++--
> > > >  1 file changed, 54 insertions(+), 5 deletions(-)
> > > >
> > > > diff --git
> > > > a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > > b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > > index 5a32bb138158..77657f63a169 100644
> > > > --- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > > +++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > > > @@ -39,6 +39,24 @@ static const struct vpu_format
> > > enc_fmt_list[FMT_TYPES][MAX_FMTS] = {
> > > >  		{
> > > >  			.v4l2_pix_fmt = V4L2_PIX_FMT_NV21M,
> > > >  		},
> > > > +		{
> > > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422P,
> > > > +		},
> > > > +		{
> > > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16,
> > > > +		},
> > > > +		{
> > > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61,
> > > > +		},
> > > > +		{
> > > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_YUV422M,
> > > > +		},
> > > > +		{
> > > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV16M,
> > > > +		},
> > > > +		{
> > > > +			.v4l2_pix_fmt = V4L2_PIX_FMT_NV61M,
> > > > +		},
> > > >  	}
> > > >  };
> > > >
> > > > @@ -101,13 +119,30 @@ static int start_encode(struct vpu_instance
> > > > *inst,
> > > u32 *fail_res)
> > > >  	struct vb2_v4l2_buffer *dst_buf;
> > > >  	struct frame_buffer frame_buf;
> > > >  	struct enc_param pic_param;
> > > > -	u32 stride = ALIGN(inst->dst_fmt.width, 32);
> > > > -	u32 luma_size = (stride * inst->dst_fmt.height);
> > > > -	u32 chroma_size = ((stride / 2) * (inst->dst_fmt.height / 2));
> > > > +	u32 stride = inst->src_fmt.plane_fmt[0].bytesperline;
> > > > +	u32 luma_size = (stride * inst->src_fmt.height);

I reviewed the above code, again. The code would be proper.
Because in case of NV12 color space, it has one planar, therefor the YUV value is in the same linear memory.
To calculate luma_size, we need the above code.


> > > > +	u32 chroma_size = 0;
> > >
> > > The helper introduced in previous patch also calculate sizeimage for
> > > each planes, so no need for this code anymore.
> >
> > Your comment means the below code?
> >
> > 	u32 luma_size = inst->src_fmt.plane_fmt[0].sizeimage
> > 	u32 chroma_size = inst->src_fmt.plane_fmt[1].sizeimage
> >
> >
> > >
> > > >
> > > >  	memset(&pic_param, 0, sizeof(struct enc_param));
> > > >  	memset(&frame_buf, 0, sizeof(struct frame_buffer));
> > > >
> >
> > The below code could be removed.
> >
> > > > +	if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV420 ||
> > > > +	    inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV420M)
> > > > +		chroma_size = luma_size / 4;
> > > > +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12 ||
> > > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21 ||
> > > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV12M ||
> > > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV21M)
> > > > +		chroma_size = luma_size / 2;
> > > > +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422P ||
> > > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_YUV422M)
> > > > +		chroma_size = luma_size / 2;
> > > > +	else if (inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16 ||
> > > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61 ||
> > > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV16M ||
> > > > +		 inst->src_fmt.pixelformat == V4L2_PIX_FMT_NV61M)
> > > > +		chroma_size = luma_size;
> > > > +
> >
> > Is That right?
> 
> Yes, using the src_fmt seems accurate for the encoder.
> 
> cheers,
> Nicolas