[PATCH v2 00/29] Venus updates

[PATCH v2 00/29] Venus updates

[Stanimir Varbanov]

Hello,

Here is v2 with following comments addressed:

* reworked venus suspend 3xx and reuse it for 4xx.
* drop 10/28 patch from v1, i.e. call of session_continue when
  buffer requirements are not sufficient.
* fixed kbuild test robot warning in 11/28 by allocating instance
  variable from heap.
* spelling typo in 15/28.
* added Reviewed-by for DT changes.
* extended 28/28 HEVC support for encoder, now the profile and
  level are selected properly.

Comments are welcome!

regards,
Stan

Stanimir Varbanov (29):
  venus: hfi_msgs: correct pointer increment
  venus: hfi: preparation to support venus 4xx
  venus: hfi: update sequence event to handle more properties
  venus: hfi_cmds: add set_properties for 4xx version
  venus: hfi: support session continue for 4xx version
  venus: hfi: handle buffer output2 type as well
  venus: hfi_venus: add halt AXI support for Venus 4xx
  venus: hfi_venus: fix suspend function for venus 3xx versions
  venus: hfi_venus: move set of default properties to core init
  venus: hfi_venus: add suspend functionality for Venus 4xx
  venus: venc,vdec: adds clocks needed for venus 4xx
  venus: add common capability parser
  venus: helpers: make a commmon function for power_enable
  venus: core: delete not used flag for buffer mode
  venus: helpers: rename a helper function and use buffer mode from caps
  venus: add a helper function to set dynamic buffer mode
  venus: add helper function to set actual buffer size
  venus: delete no longer used bufmode flag from instance
  venus: helpers: add buffer type argument to a helper
  venus: helpers: add a new helper to set raw format
  venus: helpers,vdec,venc: add helpers to set work mode and core usage
  venus: helpers: extend set_num_bufs helper with one more argument
  venus: helpers: add a helper to return opb buffer sizes
  venus: vdec: get required input buffers as well
  venus: vdec: new function for output configuration
  venus: move frame size calculations in common place
  venus: implementing multi-stream support
  venus: add sdm845 compatible and resource data
  venus: add HEVC codec support

 .../devicetree/bindings/media/qcom,venus.txt       |   1 +
 drivers/media/platform/qcom/venus/Makefile         |   3 +-
 drivers/media/platform/qcom/venus/core.c           | 107 ++++
 drivers/media/platform/qcom/venus/core.h           |  93 ++--
 drivers/media/platform/qcom/venus/helpers.c        | 558 +++++++++++++++++++--
 drivers/media/platform/qcom/venus/helpers.h        |  23 +-
 drivers/media/platform/qcom/venus/hfi.c            |  12 +-
 drivers/media/platform/qcom/venus/hfi.h            |   9 +
 drivers/media/platform/qcom/venus/hfi_cmds.c       |  64 ++-
 drivers/media/platform/qcom/venus/hfi_helper.h     | 112 ++++-
 drivers/media/platform/qcom/venus/hfi_msgs.c       | 401 +++------------
 drivers/media/platform/qcom/venus/hfi_parser.c     | 291 +++++++++++
 drivers/media/platform/qcom/venus/hfi_parser.h     |  45 ++
 drivers/media/platform/qcom/venus/hfi_venus.c      |  95 +++-
 drivers/media/platform/qcom/venus/hfi_venus_io.h   |  25 +
 drivers/media/platform/qcom/venus/vdec.c           | 316 +++++++-----
 drivers/media/platform/qcom/venus/venc.c           | 211 ++++----
 17 files changed, 1689 insertions(+), 677 deletions(-)
 create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.c
 create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.h

-- 
2.14.1

[PATCH v2 01/29] venus: hfi_msgs: correct pointer increment

[Stanimir Varbanov]

Data pointer should be incremented by size of the structure not
the size of a pointer, correct the mistake.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi_msgs.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/venus/hfi_msgs.c
index 90c93d9603dc..589e1a6b36a9 100644
--- a/drivers/media/platform/qcom/venus/hfi_msgs.c
+++ b/drivers/media/platform/qcom/venus/hfi_msgs.c
@@ -60,14 +60,14 @@ static void event_seq_changed(struct venus_core *core, struct venus_inst *inst,
 			frame_sz = (struct hfi_framesize *)data_ptr;
 			event.width = frame_sz->width;
 			event.height = frame_sz->height;
-			data_ptr += sizeof(frame_sz);
+			data_ptr += sizeof(*frame_sz);
 			break;
 		case HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT:
 			data_ptr += sizeof(u32);
 			profile_level = (struct hfi_profile_level *)data_ptr;
 			event.profile = profile_level->profile;
 			event.level = profile_level->level;
-			data_ptr += sizeof(profile_level);
+			data_ptr += sizeof(*profile_level);
 			break;
 		default:
 			break;
-- 
2.14.1

Re: [PATCH v2 01/29] venus: hfi_msgs: correct pointer increment

[Tomasz Figa]

On Tue, May 15, 2018 at 5:13 PM Stanimir Varbanov <
stanimir.varbanov@linaro.org> wrote:

> Data pointer should be incremented by size of the structure not
> the size of a pointer, correct the mistake.

> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>   drivers/media/platform/qcom/venus/hfi_msgs.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)

> diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c
b/drivers/media/platform/qcom/venus/hfi_msgs.c
> index 90c93d9603dc..589e1a6b36a9 100644
> --- a/drivers/media/platform/qcom/venus/hfi_msgs.c
> +++ b/drivers/media/platform/qcom/venus/hfi_msgs.c
> @@ -60,14 +60,14 @@ static void event_seq_changed(struct venus_core
*core, struct venus_inst *inst,
>                          frame_sz = (struct hfi_framesize *)data_ptr;
>                          event.width = frame_sz->width;
>                          event.height = frame_sz->height;
> -                       data_ptr += sizeof(frame_sz);
> +                       data_ptr += sizeof(*frame_sz);
>                          break;
>                  case HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT:
>                          data_ptr += sizeof(u32);
>                          profile_level = (struct hfi_profile_level
*)data_ptr;
>                          event.profile = profile_level->profile;
>                          event.level = profile_level->level;
> -                       data_ptr += sizeof(profile_level);
> +                       data_ptr += sizeof(*profile_level);
>                          break;
>                  default:
>                          break;
> --
> 2.14.1


(Posted to v1 originally, since I missed v2.)

Reviewed-by: Tomasz Figa <tfiga@chromium.org>

Best regards,
Tomasz

[PATCH v2 02/29] venus: hfi: preparation to support venus 4xx

[Stanimir Varbanov]

This covers the differences between 1xx,3xx and 4xx.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h         |  4 ++
 drivers/media/platform/qcom/venus/helpers.c      | 37 +++++++----
 drivers/media/platform/qcom/venus/hfi_helper.h   | 84 ++++++++++++++++++++++--
 drivers/media/platform/qcom/venus/hfi_venus_io.h | 24 +++++++
 drivers/media/platform/qcom/venus/vdec.c         |  5 +-
 drivers/media/platform/qcom/venus/venc.c         |  5 +-
 6 files changed, 137 insertions(+), 22 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 0360d295f4c8..8d3e150800c9 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -305,6 +305,10 @@ struct venus_inst {
 	struct hfi_buffer_requirements bufreq[HFI_BUFFER_TYPE_MAX];
 };
 
+#define IS_V1(core)	((core)->res->hfi_version == HFI_VERSION_1XX)
+#define IS_V3(core)	((core)->res->hfi_version == HFI_VERSION_3XX)
+#define IS_V4(core)	((core)->res->hfi_version == HFI_VERSION_4XX)
+
 #define ctrl_to_inst(ctrl)	\
 	container_of((ctrl)->handler, struct venus_inst, ctrl_handler)
 
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 0ce9559a2924..d9065cc8a7d3 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -166,21 +166,37 @@ static int intbufs_unset_buffers(struct venus_inst *inst)
 	return ret;
 }
 
-static const unsigned int intbuf_types[] = {
-	HFI_BUFFER_INTERNAL_SCRATCH,
-	HFI_BUFFER_INTERNAL_SCRATCH_1,
-	HFI_BUFFER_INTERNAL_SCRATCH_2,
+static const unsigned int intbuf_types_1xx[] = {
+	HFI_BUFFER_INTERNAL_SCRATCH(HFI_VERSION_1XX),
+	HFI_BUFFER_INTERNAL_SCRATCH_1(HFI_VERSION_1XX),
+	HFI_BUFFER_INTERNAL_SCRATCH_2(HFI_VERSION_1XX),
+	HFI_BUFFER_INTERNAL_PERSIST,
+	HFI_BUFFER_INTERNAL_PERSIST_1,
+};
+
+static const unsigned int intbuf_types_4xx[] = {
+	HFI_BUFFER_INTERNAL_SCRATCH(HFI_VERSION_4XX),
+	HFI_BUFFER_INTERNAL_SCRATCH_1(HFI_VERSION_4XX),
+	HFI_BUFFER_INTERNAL_SCRATCH_2(HFI_VERSION_4XX),
 	HFI_BUFFER_INTERNAL_PERSIST,
 	HFI_BUFFER_INTERNAL_PERSIST_1,
 };
 
 static int intbufs_alloc(struct venus_inst *inst)
 {
-	unsigned int i;
+	size_t arr_sz;
+	size_t i;
 	int ret;
 
-	for (i = 0; i < ARRAY_SIZE(intbuf_types); i++) {
-		ret = intbufs_set_buffer(inst, intbuf_types[i]);
+	if (IS_V4(inst->core))
+		arr_sz = ARRAY_SIZE(intbuf_types_4xx);
+	else
+		arr_sz = ARRAY_SIZE(intbuf_types_1xx);
+
+	for (i = 0; i < arr_sz; i++) {
+		ret = intbufs_set_buffer(inst,
+			    IS_V4(inst->core) ? intbuf_types_4xx[i] :
+						intbuf_types_1xx[i]);
 		if (ret)
 			goto error;
 	}
@@ -257,12 +273,11 @@ static int load_scale_clocks(struct venus_core *core)
 
 set_freq:
 
-	if (core->res->hfi_version == HFI_VERSION_3XX) {
-		ret = clk_set_rate(clk, freq);
+	ret = clk_set_rate(clk, freq);
+
+	if (IS_V3(core) || IS_V4(core)) {
 		ret |= clk_set_rate(core->core0_clk, freq);
 		ret |= clk_set_rate(core->core1_clk, freq);
-	} else {
-		ret = clk_set_rate(clk, freq);
 	}
 
 	if (ret) {
diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/venus/hfi_helper.h
index 55d8eb21403a..1bc5aab1ce6b 100644
--- a/drivers/media/platform/qcom/venus/hfi_helper.h
+++ b/drivers/media/platform/qcom/venus/hfi_helper.h
@@ -121,6 +121,7 @@
 #define HFI_EXTRADATA_METADATA_FILLER			0x7fe00002
 
 #define HFI_INDEX_EXTRADATA_INPUT_CROP			0x0700000e
+#define HFI_INDEX_EXTRADATA_OUTPUT_CROP			0x0700000f
 #define HFI_INDEX_EXTRADATA_DIGITAL_ZOOM		0x07000010
 #define HFI_INDEX_EXTRADATA_ASPECT_RATIO		0x7f100003
 
@@ -376,13 +377,18 @@
 #define HFI_BUFFER_OUTPUT2			0x3
 #define HFI_BUFFER_INTERNAL_PERSIST		0x4
 #define HFI_BUFFER_INTERNAL_PERSIST_1		0x5
-#define HFI_BUFFER_INTERNAL_SCRATCH		0x1000001
-#define HFI_BUFFER_EXTRADATA_INPUT		0x1000002
-#define HFI_BUFFER_EXTRADATA_OUTPUT		0x1000003
-#define HFI_BUFFER_EXTRADATA_OUTPUT2		0x1000004
-#define HFI_BUFFER_INTERNAL_SCRATCH_1		0x1000005
-#define HFI_BUFFER_INTERNAL_SCRATCH_2		0x1000006
-
+#define HFI_BUFFER_INTERNAL_SCRATCH(ver)	\
+	(((ver) == HFI_VERSION_4XX) ? 0x6 : 0x1000001)
+#define HFI_BUFFER_INTERNAL_SCRATCH_1(ver)	\
+	(((ver) == HFI_VERSION_4XX) ? 0x7 : 0x1000005)
+#define HFI_BUFFER_INTERNAL_SCRATCH_2(ver)	\
+	(((ver) == HFI_VERSION_4XX) ? 0x8 : 0x1000006)
+#define HFI_BUFFER_EXTRADATA_INPUT(ver)		\
+	(((ver) == HFI_VERSION_4XX) ? 0xc : 0x1000002)
+#define HFI_BUFFER_EXTRADATA_OUTPUT(ver)	\
+	(((ver) == HFI_VERSION_4XX) ? 0xa : 0x1000003)
+#define HFI_BUFFER_EXTRADATA_OUTPUT2(ver)	\
+	(((ver) == HFI_VERSION_4XX) ? 0xb : 0x1000004)
 #define HFI_BUFFER_TYPE_MAX			11
 
 #define HFI_BUFFER_MODE_STATIC			0x1000001
@@ -424,12 +430,14 @@
 #define HFI_PROPERTY_PARAM_CODEC_MASK_SUPPORTED			0x100e
 #define HFI_PROPERTY_PARAM_MVC_BUFFER_LAYOUT			0x100f
 #define HFI_PROPERTY_PARAM_MAX_SESSIONS_SUPPORTED		0x1010
+#define HFI_PROPERTY_PARAM_WORK_MODE				0x1015
 
 /*
  * HFI_PROPERTY_CONFIG_COMMON_START
  * HFI_DOMAIN_BASE_COMMON + HFI_ARCH_COMMON_OFFSET + 0x2000
  */
 #define HFI_PROPERTY_CONFIG_FRAME_RATE				0x2001
+#define HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE			0x2002
 
 /*
  * HFI_PROPERTY_PARAM_VDEC_COMMON_START
@@ -438,6 +446,9 @@
 #define HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM			0x1003001
 #define HFI_PROPERTY_PARAM_VDEC_CONCEAL_COLOR			0x1003002
 #define HFI_PROPERTY_PARAM_VDEC_NONCP_OUTPUT2			0x1003003
+#define HFI_PROPERTY_PARAM_VDEC_PIXEL_BITDEPTH			0x1003007
+#define HFI_PROPERTY_PARAM_VDEC_PIC_STRUCT			0x1003009
+#define HFI_PROPERTY_PARAM_VDEC_COLOUR_SPACE			0x100300a
 
 /*
  * HFI_PROPERTY_CONFIG_VDEC_COMMON_START
@@ -518,6 +529,7 @@
 enum hfi_version {
 	HFI_VERSION_1XX,
 	HFI_VERSION_3XX,
+	HFI_VERSION_4XX
 };
 
 struct hfi_buffer_info {
@@ -767,12 +779,56 @@ struct hfi_framesize {
 	u32 height;
 };
 
+#define VIDC_CORE_ID_DEFAULT	0
+#define VIDC_CORE_ID_1		1
+#define VIDC_CORE_ID_2		2
+#define VIDC_CORE_ID_3		3
+
+struct hfi_videocores_usage_type {
+	u32 video_core_enable_mask;
+};
+
+#define VIDC_WORK_MODE_1	1
+#define VIDC_WORK_MODE_2	2
+
+struct hfi_video_work_mode {
+	u32 video_work_mode;
+};
+
 struct hfi_h264_vui_timing_info {
 	u32 enable;
 	u32 fixed_framerate;
 	u32 time_scale;
 };
 
+struct hfi_bit_depth {
+	u32 buffer_type;
+	u32 bit_depth;
+};
+
+struct hfi_picture_type {
+	u32 is_sync_frame;
+	u32 picture_type;
+};
+
+struct hfi_pic_struct {
+	u32 progressive_only;
+};
+
+struct hfi_colour_space {
+	u32 colour_space;
+};
+
+struct hfi_extradata_input_crop {
+	u32 size;
+	u32 version;
+	u32 port_index;
+	u32 left;
+	u32 top;
+	u32 width;
+	u32 height;
+};
+
 #define HFI_COLOR_FORMAT_MONOCHROME		0x01
 #define HFI_COLOR_FORMAT_NV12			0x02
 #define HFI_COLOR_FORMAT_NV21			0x03
@@ -961,6 +1017,12 @@ struct hfi_buffer_count_actual {
 	u32 count_actual;
 };
 
+struct hfi_buffer_count_actual_4xx {
+	u32 type;
+	u32 count_actual;
+	u32 count_min_host;
+};
+
 struct hfi_buffer_size_actual {
 	u32 type;
 	u32 size;
@@ -971,6 +1033,14 @@ struct hfi_buffer_display_hold_count_actual {
 	u32 hold_count;
 };
 
+/* HFI 4XX reorder the fields, use these macros */
+#define HFI_BUFREQ_HOLD_COUNT(bufreq, ver)	\
+	((ver) == HFI_VERSION_4XX ? 0 : (bufreq)->hold_count)
+#define HFI_BUFREQ_COUNT_MIN(bufreq, ver)	\
+	((ver) == HFI_VERSION_4XX ? (bufreq)->hold_count : (bufreq)->count_min)
+#define HFI_BUFREQ_COUNT_MIN_HOST(bufreq, ver)	\
+	((ver) == HFI_VERSION_4XX ? (bufreq)->count_min : 0)
+
 struct hfi_buffer_requirements {
 	u32 type;
 	u32 size;
diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h b/drivers/media/platform/qcom/venus/hfi_venus_io.h
index 98cc350113ab..76f47936d0fa 100644
--- a/drivers/media/platform/qcom/venus/hfi_venus_io.h
+++ b/drivers/media/platform/qcom/venus/hfi_venus_io.h
@@ -110,4 +110,28 @@
 #define WRAPPER_CPU_STATUS			(WRAPPER_BASE + 0x2014)
 #define WRAPPER_SW_RESET			(WRAPPER_BASE + 0x3000)
 
+/* Venus 4xx */
+#define WRAPPER_VCODEC0_MMCC_POWER_STATUS	(WRAPPER_BASE + 0x90)
+#define WRAPPER_VCODEC0_MMCC_POWER_CONTROL	(WRAPPER_BASE + 0x94)
+
+#define WRAPPER_VCODEC1_MMCC_POWER_STATUS	(WRAPPER_BASE + 0x110)
+#define WRAPPER_VCODEC1_MMCC_POWER_CONTROL	(WRAPPER_BASE + 0x114)
+
+/* vcodec noc error log registers */
+#define VCODEC_CORE0_VIDEO_NOC_BASE_OFFS		0x4000
+#define VCODEC_CORE1_VIDEO_NOC_BASE_OFFS		0xc000
+#define VCODEC_COREX_VIDEO_NOC_ERR_SWID_LOW_OFFS	0x500
+#define VCODEC_COREX_VIDEO_NOC_ERR_SWID_HIGH_OFFS	0x504
+#define VCODEC_COREX_VIDEO_NOC_ERR_MAINCTL_LOW_OFFS	0x508
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRVLD_LOW_OFFS	0x510
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRCLR_LOW_OFFS	0x518
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG0_LOW_OFFS	0x520
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG0_HIGH_OFFS	0x524
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG1_LOW_OFFS	0x528
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG1_HIGH_OFFS	0x52c
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG2_LOW_OFFS	0x530
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG2_HIGH_OFFS	0x534
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG3_LOW_OFFS	0x538
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG3_HIGH_OFFS	0x53c
+
 #endif
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 49bbd1861d3a..261a51adeef2 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -689,6 +689,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
 
 static int vdec_verify_conf(struct venus_inst *inst)
 {
+	enum hfi_version ver = inst->core->res->hfi_version;
 	struct hfi_buffer_requirements bufreq;
 	int ret;
 
@@ -700,14 +701,14 @@ static int vdec_verify_conf(struct venus_inst *inst)
 		return ret;
 
 	if (inst->num_output_bufs < bufreq.count_actual ||
-	    inst->num_output_bufs < bufreq.count_min)
+	    inst->num_output_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
 		return -EINVAL;
 
 	ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);
 	if (ret)
 		return ret;
 
-	if (inst->num_input_bufs < bufreq.count_min)
+	if (inst->num_input_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
 		return -EINVAL;
 
 	return 0;
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 6b2ce479584e..947001170a77 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -892,6 +892,7 @@ static int venc_queue_setup(struct vb2_queue *q,
 
 static int venc_verify_conf(struct venus_inst *inst)
 {
+	enum hfi_version ver = inst->core->res->hfi_version;
 	struct hfi_buffer_requirements bufreq;
 	int ret;
 
@@ -903,7 +904,7 @@ static int venc_verify_conf(struct venus_inst *inst)
 		return ret;
 
 	if (inst->num_output_bufs < bufreq.count_actual ||
-	    inst->num_output_bufs < bufreq.count_min)
+	    inst->num_output_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
 		return -EINVAL;
 
 	ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);
@@ -911,7 +912,7 @@ static int venc_verify_conf(struct venus_inst *inst)
 		return ret;
 
 	if (inst->num_input_bufs < bufreq.count_actual ||
-	    inst->num_input_bufs < bufreq.count_min)
+	    inst->num_input_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
 		return -EINVAL;
 
 	return 0;
-- 
2.14.1

Re: [PATCH v2 02/29] venus: hfi: preparation to support venus 4xx

[Tomasz Figa]

Hi Stanimir,

On Tue, May 15, 2018 at 5:14 PM Stanimir Varbanov <
stanimir.varbanov@linaro.org> wrote:

> This covers the differences between 1xx,3xx and 4xx.

> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>    drivers/media/platform/qcom/venus/core.h         |  4 ++
>    drivers/media/platform/qcom/venus/helpers.c      | 37 +++++++----
>    drivers/media/platform/qcom/venus/hfi_helper.h   | 84
++++++++++++++++++++++--
>    drivers/media/platform/qcom/venus/hfi_venus_io.h | 24 +++++++
>    drivers/media/platform/qcom/venus/vdec.c         |  5 +-
>    drivers/media/platform/qcom/venus/venc.c         |  5 +-
>    6 files changed, 137 insertions(+), 22 deletions(-)

Please see my comments inline.

[snip]

> @@ -257,12 +273,11 @@ static int load_scale_clocks(struct venus_core
*core)

>    set_freq:

> -       if (core->res->hfi_version == HFI_VERSION_3XX) {
> -               ret = clk_set_rate(clk, freq);
> +       ret = clk_set_rate(clk, freq);
> +
> +       if (IS_V3(core) || IS_V4(core)) {
>                   ret |= clk_set_rate(core->core0_clk, freq);
>                   ret |= clk_set_rate(core->core1_clk, freq);
> -       } else {
> -               ret = clk_set_rate(clk, freq);
>           }

nit: The clock API defines NULL clock as a special no-op value and so
clk_set_rate(NULL, ...) would return 0 instantly. Maybe it would just make
sense to have core0_clk and core1_clk set to NULL for V1 and remove the
condition here?

>           if (ret) {
> diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h
b/drivers/media/platform/qcom/venus/hfi_helper.h
> index 55d8eb21403a..1bc5aab1ce6b 100644
> --- a/drivers/media/platform/qcom/venus/hfi_helper.h
> +++ b/drivers/media/platform/qcom/venus/hfi_helper.h
> @@ -121,6 +121,7 @@
>    #define HFI_EXTRADATA_METADATA_FILLER                  0x7fe00002

>    #define HFI_INDEX_EXTRADATA_INPUT_CROP                 0x0700000e
> +#define HFI_INDEX_EXTRADATA_OUTPUT_CROP                        0x0700000f

nit: Would it make sense to suffix this _4XX, so that reader could know
that it was introduced in this version? Similarly for other newly added
definitions.

>    #define HFI_INDEX_EXTRADATA_DIGITAL_ZOOM               0x07000010
>    #define HFI_INDEX_EXTRADATA_ASPECT_RATIO               0x7f100003

> @@ -376,13 +377,18 @@
>    #define HFI_BUFFER_OUTPUT2                     0x3
>    #define HFI_BUFFER_INTERNAL_PERSIST            0x4
>    #define HFI_BUFFER_INTERNAL_PERSIST_1          0x5
> -#define HFI_BUFFER_INTERNAL_SCRATCH            0x1000001
> -#define HFI_BUFFER_EXTRADATA_INPUT             0x1000002
> -#define HFI_BUFFER_EXTRADATA_OUTPUT            0x1000003
> -#define HFI_BUFFER_EXTRADATA_OUTPUT2           0x1000004
> -#define HFI_BUFFER_INTERNAL_SCRATCH_1          0x1000005
> -#define HFI_BUFFER_INTERNAL_SCRATCH_2          0x1000006
> -
> +#define HFI_BUFFER_INTERNAL_SCRATCH(ver)       \
> +       (((ver) == HFI_VERSION_4XX) ? 0x6 : 0x1000001)
> +#define HFI_BUFFER_INTERNAL_SCRATCH_1(ver)     \
> +       (((ver) == HFI_VERSION_4XX) ? 0x7 : 0x1000005)
> +#define HFI_BUFFER_INTERNAL_SCRATCH_2(ver)     \
> +       (((ver) == HFI_VERSION_4XX) ? 0x8 : 0x1000006)
> +#define HFI_BUFFER_EXTRADATA_INPUT(ver)                \
> +       (((ver) == HFI_VERSION_4XX) ? 0xc : 0x1000002)
> +#define HFI_BUFFER_EXTRADATA_OUTPUT(ver)       \
> +       (((ver) == HFI_VERSION_4XX) ? 0xa : 0x1000003)
> +#define HFI_BUFFER_EXTRADATA_OUTPUT2(ver)      \
> +       (((ver) == HFI_VERSION_4XX) ? 0xb : 0x1000004)

nit: Does it make sense to add an argument, rather than simply defining
separate HFI_BUFFER_INTERNAL_SCRATCH_1XX and
HFI_BUFFER_INTERNAL_SCRATCH_4XX? In my subjective opinion, the argument
just makes it harder to read, as it's not clear how it is used inside the
macro from reading just the call to it. Also it would get messy when adding
further variants in future.

[snip]

> +/* HFI 4XX reorder the fields, use these macros */
> +#define HFI_BUFREQ_HOLD_COUNT(bufreq, ver)     \
> +       ((ver) == HFI_VERSION_4XX ? 0 : (bufreq)->hold_count)
> +#define HFI_BUFREQ_COUNT_MIN(bufreq, ver)      \
> +       ((ver) == HFI_VERSION_4XX ? (bufreq)->hold_count :
(bufreq)->count_min)
> +#define HFI_BUFREQ_COUNT_MIN_HOST(bufreq, ver) \
> +       ((ver) == HFI_VERSION_4XX ? (bufreq)->count_min : 0)
> +

Hmm, this is a bit messy. The macro is supposed to return count_min, but it
returns hold_count. Shouldn't we define a separate
hfi_buffer_requirements_4xx struct for 4XX?

Even though this seems to simplify the code eventually, I think it might be
quite confusing for anyone working with the driver in the future.

Also HFI_BUFREQ_HOLD_COUNT and HFI_BUFREQ_COUNT_MIN_HOST don't seem to be
used anywhere.

[snip]

> +/* vcodec noc error log registers */
> +#define VCODEC_CORE0_VIDEO_NOC_BASE_OFFS               0x4000
> +#define VCODEC_CORE1_VIDEO_NOC_BASE_OFFS               0xc000
> +#define VCODEC_COREX_VIDEO_NOC_ERR_SWID_LOW_OFFS       0x500
> +#define VCODEC_COREX_VIDEO_NOC_ERR_SWID_HIGH_OFFS      0x504
> +#define VCODEC_COREX_VIDEO_NOC_ERR_MAINCTL_LOW_OFFS    0x508
> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRVLD_LOW_OFFS     0x510
> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRCLR_LOW_OFFS     0x518
> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG0_LOW_OFFS    0x520
> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG0_HIGH_OFFS   0x524
> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG1_LOW_OFFS    0x528
> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG1_HIGH_OFFS   0x52c
> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG2_LOW_OFFS    0x530
> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG2_HIGH_OFFS   0x534
> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG3_LOW_OFFS    0x538
> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG3_HIGH_OFFS   0x53c

What are these offsets from?

nit: Other registers seem to be defined as (xxx_BASE + offset). Is there
any reason to define these in another way?

> +
>    #endif
> diff --git a/drivers/media/platform/qcom/venus/vdec.c
b/drivers/media/platform/qcom/venus/vdec.c
> index 49bbd1861d3a..261a51adeef2 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -689,6 +689,7 @@ static int vdec_queue_setup(struct vb2_queue *q,

>    static int vdec_verify_conf(struct venus_inst *inst)
>    {
> +       enum hfi_version ver = inst->core->res->hfi_version;
>           struct hfi_buffer_requirements bufreq;
>           int ret;

> @@ -700,14 +701,14 @@ static int vdec_verify_conf(struct venus_inst *inst)
>                   return ret;

>           if (inst->num_output_bufs < bufreq.count_actual ||
> -           inst->num_output_bufs < bufreq.count_min)
> +           inst->num_output_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
>                   return -EINVAL;

>           ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);
>           if (ret)
>                   return ret;

> -       if (inst->num_input_bufs < bufreq.count_min)
> +       if (inst->num_input_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
>                   return -EINVAL;

Back to the point I raised above, maybe we could make
venus_helper_get_bufreq() untangle the order of fields for 4XX? It seems to
do a memcpy anyway, so doing it field by field for such small struct
shouldn't really matter.

Best regards,
Tomasz

Re: [PATCH v2 02/29] venus: hfi: preparation to support venus 4xx

[Stanimir Varbanov]

Hi Tomasz,

Thanks for the comments!

On 05/18/2018 12:44 PM, Tomasz Figa wrote:
> Hi Stanimir,
> 
> On Tue, May 15, 2018 at 5:14 PM Stanimir Varbanov <
> stanimir.varbanov@linaro.org> wrote:
> 
>> This covers the differences between 1xx,3xx and 4xx.
> 
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>    drivers/media/platform/qcom/venus/core.h         |  4 ++
>>    drivers/media/platform/qcom/venus/helpers.c      | 37 +++++++----
>>    drivers/media/platform/qcom/venus/hfi_helper.h   | 84
> ++++++++++++++++++++++--
>>    drivers/media/platform/qcom/venus/hfi_venus_io.h | 24 +++++++
>>    drivers/media/platform/qcom/venus/vdec.c         |  5 +-
>>    drivers/media/platform/qcom/venus/venc.c         |  5 +-
>>    6 files changed, 137 insertions(+), 22 deletions(-)
> 
> Please see my comments inline.
> 
> [snip]
> 
>> @@ -257,12 +273,11 @@ static int load_scale_clocks(struct venus_core
> *core)
> 
>>    set_freq:
> 
>> -       if (core->res->hfi_version == HFI_VERSION_3XX) {
>> -               ret = clk_set_rate(clk, freq);
>> +       ret = clk_set_rate(clk, freq);
>> +
>> +       if (IS_V3(core) || IS_V4(core)) {
>>                   ret |= clk_set_rate(core->core0_clk, freq);
>>                   ret |= clk_set_rate(core->core1_clk, freq);
>> -       } else {
>> -               ret = clk_set_rate(clk, freq);
>>           }
> 
> nit: The clock API defines NULL clock as a special no-op value and so
> clk_set_rate(NULL, ...) would return 0 instantly. Maybe it would just make
> sense to have core0_clk and core1_clk set to NULL for V1 and remove the
> condition here?

OK, we could avoid the condition but I'll add a comment that those
clocks exist from v3 onwards.

> 
>>           if (ret) {
>> diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h
> b/drivers/media/platform/qcom/venus/hfi_helper.h
>> index 55d8eb21403a..1bc5aab1ce6b 100644
>> --- a/drivers/media/platform/qcom/venus/hfi_helper.h
>> +++ b/drivers/media/platform/qcom/venus/hfi_helper.h
>> @@ -121,6 +121,7 @@
>>    #define HFI_EXTRADATA_METADATA_FILLER                  0x7fe00002
> 
>>    #define HFI_INDEX_EXTRADATA_INPUT_CROP                 0x0700000e
>> +#define HFI_INDEX_EXTRADATA_OUTPUT_CROP                        0x0700000f
> 
> nit: Would it make sense to suffix this _4XX, so that reader could know
> that it was introduced in this version? Similarly for other newly added
> definitions.

I personally don't like the version suffix on defines. Also seems this
is not used in the code, so I'll drop it for now.

> 
>>    #define HFI_INDEX_EXTRADATA_DIGITAL_ZOOM               0x07000010
>>    #define HFI_INDEX_EXTRADATA_ASPECT_RATIO               0x7f100003
> 
>> @@ -376,13 +377,18 @@
>>    #define HFI_BUFFER_OUTPUT2                     0x3
>>    #define HFI_BUFFER_INTERNAL_PERSIST            0x4
>>    #define HFI_BUFFER_INTERNAL_PERSIST_1          0x5
>> -#define HFI_BUFFER_INTERNAL_SCRATCH            0x1000001
>> -#define HFI_BUFFER_EXTRADATA_INPUT             0x1000002
>> -#define HFI_BUFFER_EXTRADATA_OUTPUT            0x1000003
>> -#define HFI_BUFFER_EXTRADATA_OUTPUT2           0x1000004
>> -#define HFI_BUFFER_INTERNAL_SCRATCH_1          0x1000005
>> -#define HFI_BUFFER_INTERNAL_SCRATCH_2          0x1000006
>> -
>> +#define HFI_BUFFER_INTERNAL_SCRATCH(ver)       \
>> +       (((ver) == HFI_VERSION_4XX) ? 0x6 : 0x1000001)
>> +#define HFI_BUFFER_INTERNAL_SCRATCH_1(ver)     \
>> +       (((ver) == HFI_VERSION_4XX) ? 0x7 : 0x1000005)
>> +#define HFI_BUFFER_INTERNAL_SCRATCH_2(ver)     \
>> +       (((ver) == HFI_VERSION_4XX) ? 0x8 : 0x1000006)
>> +#define HFI_BUFFER_EXTRADATA_INPUT(ver)                \
>> +       (((ver) == HFI_VERSION_4XX) ? 0xc : 0x1000002)
>> +#define HFI_BUFFER_EXTRADATA_OUTPUT(ver)       \
>> +       (((ver) == HFI_VERSION_4XX) ? 0xa : 0x1000003)
>> +#define HFI_BUFFER_EXTRADATA_OUTPUT2(ver)      \
>> +       (((ver) == HFI_VERSION_4XX) ? 0xb : 0x1000004)
> 
> nit: Does it make sense to add an argument, rather than simply defining
> separate HFI_BUFFER_INTERNAL_SCRATCH_1XX and
> HFI_BUFFER_INTERNAL_SCRATCH_4XX? In my subjective opinion, the argument

I'd like to keep the name of the define version agnostic.

> just makes it harder to read, as it's not clear how it is used inside the
> macro from reading just the call to it. Also it would get messy when adding
> further variants in future.
> 
> [snip]
> 
>> +/* HFI 4XX reorder the fields, use these macros */
>> +#define HFI_BUFREQ_HOLD_COUNT(bufreq, ver)     \
>> +       ((ver) == HFI_VERSION_4XX ? 0 : (bufreq)->hold_count)
>> +#define HFI_BUFREQ_COUNT_MIN(bufreq, ver)      \
>> +       ((ver) == HFI_VERSION_4XX ? (bufreq)->hold_count :
> (bufreq)->count_min)
>> +#define HFI_BUFREQ_COUNT_MIN_HOST(bufreq, ver) \
>> +       ((ver) == HFI_VERSION_4XX ? (bufreq)->count_min : 0)
>> +
> 
> Hmm, this is a bit messy. The macro is supposed to return count_min, but it
> returns hold_count. Shouldn't we define a separate

yep, that was the purpose of the macro, i.e. to swap the fields
depending on the version.

> hfi_buffer_requirements_4xx struct for 4XX?

With above macros I wanted to avoid that. We already have few structures
with 3x prefix and I want to stop that growing.

> 
> Even though this seems to simplify the code eventually, I think it might be
> quite confusing for anyone working with the driver in the future.

It is a matter of taste in the end of the day.

> 
> Also HFI_BUFREQ_HOLD_COUNT and HFI_BUFREQ_COUNT_MIN_HOST don't seem to be
> used anywhere.

yes, they are here for completeness ;)

> 
> [snip]
> 
>> +/* vcodec noc error log registers */
>> +#define VCODEC_CORE0_VIDEO_NOC_BASE_OFFS               0x4000
>> +#define VCODEC_CORE1_VIDEO_NOC_BASE_OFFS               0xc000
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_SWID_LOW_OFFS       0x500
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_SWID_HIGH_OFFS      0x504
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_MAINCTL_LOW_OFFS    0x508
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRVLD_LOW_OFFS     0x510
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRCLR_LOW_OFFS     0x518
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG0_LOW_OFFS    0x520
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG0_HIGH_OFFS   0x524
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG1_LOW_OFFS    0x528
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG1_HIGH_OFFS   0x52c
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG2_LOW_OFFS    0x530
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG2_HIGH_OFFS   0x534
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG3_LOW_OFFS    0x538
>> +#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG3_HIGH_OFFS   0x53c
> 
> What are these offsets from?

Those registers are used in donwstream driver for debug purpose. In case
iommu fault is triggered dumping these registers could give us a clue
what goes wrong.

> 
> nit: Other registers seem to be defined as (xxx_BASE + offset). Is there
> any reason to define these in another way?

For example the first register in the list is a BASE address for core0.
Next _COREX_ are offsets from the base.

Looks these registers are not used in this patchset I'll drop them for now.

> 
>> +
>>    #endif
>> diff --git a/drivers/media/platform/qcom/venus/vdec.c
> b/drivers/media/platform/qcom/venus/vdec.c
>> index 49bbd1861d3a..261a51adeef2 100644
>> --- a/drivers/media/platform/qcom/venus/vdec.c
>> +++ b/drivers/media/platform/qcom/venus/vdec.c
>> @@ -689,6 +689,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
> 
>>    static int vdec_verify_conf(struct venus_inst *inst)
>>    {
>> +       enum hfi_version ver = inst->core->res->hfi_version;
>>           struct hfi_buffer_requirements bufreq;
>>           int ret;
> 
>> @@ -700,14 +701,14 @@ static int vdec_verify_conf(struct venus_inst *inst)
>>                   return ret;
> 
>>           if (inst->num_output_bufs < bufreq.count_actual ||
>> -           inst->num_output_bufs < bufreq.count_min)
>> +           inst->num_output_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
>>                   return -EINVAL;
> 
>>           ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);
>>           if (ret)
>>                   return ret;
> 
>> -       if (inst->num_input_bufs < bufreq.count_min)
>> +       if (inst->num_input_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
>>                   return -EINVAL;
> 
> Back to the point I raised above, maybe we could make
> venus_helper_get_bufreq() untangle the order of fields for 4XX? It seems to
> do a memcpy anyway, so doing it field by field for such small struct
> shouldn't really matter.

That couldn't happen because the structure fields are not only reordered
but also renamed.

1xx and 3xx	vs.	4xx

hold_count		count_min
count_min		count_min_host
count_actual		count_actual

-- 
regards,
Stan

[PATCH v2 03/29] venus: hfi: update sequence event to handle more properties

[Stanimir Varbanov]

HFI version 4xx can pass more properties in the sequence change
event, extend the event structure with them.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi.h      |  9 ++++++
 drivers/media/platform/qcom/venus/hfi_msgs.c | 46 ++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/venus/hfi.h
index 5466b7d60dd0..21376d93170f 100644
--- a/drivers/media/platform/qcom/venus/hfi.h
+++ b/drivers/media/platform/qcom/venus/hfi.h
@@ -74,6 +74,15 @@ struct hfi_event_data {
 	u32 tag;
 	u32 profile;
 	u32 level;
+	u32 bit_depth;
+	u32 pic_struct;
+	u32 colour_space;
+	u32 entropy_mode;
+	u32 buf_count;
+	struct {
+		u32 left, top;
+		u32 width, height;
+	} input_crop;
 };
 
 /* define core states */
diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/venus/hfi_msgs.c
index 589e1a6b36a9..5970e9b1716b 100644
--- a/drivers/media/platform/qcom/venus/hfi_msgs.c
+++ b/drivers/media/platform/qcom/venus/hfi_msgs.c
@@ -25,10 +25,17 @@
 static void event_seq_changed(struct venus_core *core, struct venus_inst *inst,
 			      struct hfi_msg_event_notify_pkt *pkt)
 {
+	enum hfi_version ver = core->res->hfi_version;
 	struct hfi_event_data event = {0};
 	int num_properties_changed;
 	struct hfi_framesize *frame_sz;
 	struct hfi_profile_level *profile_level;
+	struct hfi_bit_depth *pixel_depth;
+	struct hfi_pic_struct *pic_struct;
+	struct hfi_colour_space *colour_info;
+	struct hfi_buffer_requirements *bufreq;
+	struct hfi_extradata_input_crop *crop;
+	u32 entropy_mode = 0;
 	u8 *data_ptr;
 	u32 ptype;
 
@@ -69,6 +76,45 @@ static void event_seq_changed(struct venus_core *core, struct venus_inst *inst,
 			event.level = profile_level->level;
 			data_ptr += sizeof(*profile_level);
 			break;
+		case HFI_PROPERTY_PARAM_VDEC_PIXEL_BITDEPTH:
+			data_ptr += sizeof(u32);
+			pixel_depth = (struct hfi_bit_depth *)data_ptr;
+			event.bit_depth = pixel_depth->bit_depth;
+			data_ptr += sizeof(*pixel_depth);
+			break;
+		case HFI_PROPERTY_PARAM_VDEC_PIC_STRUCT:
+			data_ptr += sizeof(u32);
+			pic_struct = (struct hfi_pic_struct *)data_ptr;
+			event.pic_struct = pic_struct->progressive_only;
+			data_ptr += sizeof(*pic_struct);
+			break;
+		case HFI_PROPERTY_PARAM_VDEC_COLOUR_SPACE:
+			data_ptr += sizeof(u32);
+			colour_info = (struct hfi_colour_space *)data_ptr;
+			event.colour_space = colour_info->colour_space;
+			data_ptr += sizeof(*colour_info);
+			break;
+		case HFI_PROPERTY_CONFIG_VDEC_ENTROPY:
+			data_ptr += sizeof(u32);
+			entropy_mode = *(u32 *)data_ptr;
+			event.entropy_mode = entropy_mode;
+			data_ptr += sizeof(u32);
+			break;
+		case HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS:
+			data_ptr += sizeof(u32);
+			bufreq = (struct hfi_buffer_requirements *)data_ptr;
+			event.buf_count = HFI_BUFREQ_COUNT_MIN(bufreq, ver);
+			data_ptr += sizeof(*bufreq);
+			break;
+		case HFI_INDEX_EXTRADATA_INPUT_CROP:
+			data_ptr += sizeof(u32);
+			crop = (struct hfi_extradata_input_crop *)data_ptr;
+			event.input_crop.left = crop->left;
+			event.input_crop.top = crop->top;
+			event.input_crop.width = crop->width;
+			event.input_crop.height = crop->height;
+			data_ptr += sizeof(*crop);
+			break;
 		default:
 			break;
 		}
-- 
2.14.1

Re: [PATCH v2 03/29] venus: hfi: update sequence event to handle more properties

[Tomasz Figa]

On Tue, May 15, 2018 at 5:14 PM Stanimir Varbanov <
stanimir.varbanov@linaro.org> wrote:

> HFI version 4xx can pass more properties in the sequence change
> event, extend the event structure with them.

> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>   drivers/media/platform/qcom/venus/hfi.h      |  9 ++++++
>   drivers/media/platform/qcom/venus/hfi_msgs.c | 46
++++++++++++++++++++++++++++
>   2 files changed, 55 insertions(+)

> diff --git a/drivers/media/platform/qcom/venus/hfi.h
b/drivers/media/platform/qcom/venus/hfi.h
> index 5466b7d60dd0..21376d93170f 100644
> --- a/drivers/media/platform/qcom/venus/hfi.h
> +++ b/drivers/media/platform/qcom/venus/hfi.h
> @@ -74,6 +74,15 @@ struct hfi_event_data {
>          u32 tag;
>          u32 profile;
>          u32 level;

nit; Could we add a comment saying that it showed in 4xx?

[snip]

> +               case HFI_PROPERTY_CONFIG_VDEC_ENTROPY:
> +                       data_ptr += sizeof(u32);
> +                       entropy_mode = *(u32 *)data_ptr;
> +                       event.entropy_mode = entropy_mode;

Is the |entropy_mode| local variable necessary?

Best regards,
Tomasz

Re: [PATCH v2 03/29] venus: hfi: update sequence event to handle more properties

[Stanimir Varbanov]

Hi Tomasz,

On 05/18/2018 04:53 PM, Tomasz Figa wrote:
> On Tue, May 15, 2018 at 5:14 PM Stanimir Varbanov <
> stanimir.varbanov@linaro.org> wrote:
> 
>> HFI version 4xx can pass more properties in the sequence change
>> event, extend the event structure with them.
> 
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>   drivers/media/platform/qcom/venus/hfi.h      |  9 ++++++
>>   drivers/media/platform/qcom/venus/hfi_msgs.c | 46
> ++++++++++++++++++++++++++++
>>   2 files changed, 55 insertions(+)
> 
>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
> b/drivers/media/platform/qcom/venus/hfi.h
>> index 5466b7d60dd0..21376d93170f 100644
>> --- a/drivers/media/platform/qcom/venus/hfi.h
>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>> @@ -74,6 +74,15 @@ struct hfi_event_data {
>>          u32 tag;
>>          u32 profile;
>>          u32 level;
> 
> nit; Could we add a comment saying that it showed in 4xx?

Sure, I can add a comment.

> 
> [snip]
> 
>> +               case HFI_PROPERTY_CONFIG_VDEC_ENTROPY:
>> +                       data_ptr += sizeof(u32);
>> +                       entropy_mode = *(u32 *)data_ptr;
>> +                       event.entropy_mode = entropy_mode;
> 
> Is the |entropy_mode| local variable necessary?

Isn't GCC smart enough ;) Sure, I can drop entropy_mode local variable.

-- 
regards,
Stan

[PATCH v2 04/29] venus: hfi_cmds: add set_properties for 4xx version

[Stanimir Varbanov]

Adds set_properties method to handle newer 4xx properties and
fall-back to 3xx for the rest.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi_cmds.c | 64 +++++++++++++++++++++++++++-
 1 file changed, 63 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c b/drivers/media/platform/qcom/venus/hfi_cmds.c
index 1cfeb7743041..6bd287154796 100644
--- a/drivers/media/platform/qcom/venus/hfi_cmds.c
+++ b/drivers/media/platform/qcom/venus/hfi_cmds.c
@@ -1166,6 +1166,65 @@ pkt_session_set_property_3xx(struct hfi_session_set_property_pkt *pkt,
 	return ret;
 }
 
+static int
+pkt_session_set_property_4xx(struct hfi_session_set_property_pkt *pkt,
+			     void *cookie, u32 ptype, void *pdata)
+{
+	void *prop_data;
+	int ret = 0;
+
+	if (!pkt || !cookie || !pdata)
+		return -EINVAL;
+
+	prop_data = &pkt->data[1];
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_SET_PROPERTY;
+	pkt->shdr.session_id = hash32_ptr(cookie);
+	pkt->num_properties = 1;
+	pkt->data[0] = ptype;
+
+	/*
+	 * Any session set property which is different in 3XX packetization
+	 * should be added as a new case below. All unchanged session set
+	 * properties will be handled in the default case.
+	 */
+	switch (ptype) {
+	case HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL: {
+		struct hfi_buffer_count_actual *in = pdata;
+		struct hfi_buffer_count_actual_4xx *count = prop_data;
+
+		count->count_actual = in->count_actual;
+		count->type = in->type;
+		count->count_min_host = in->count_actual;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*count);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_WORK_MODE: {
+		struct hfi_video_work_mode *in = pdata, *wm = prop_data;
+
+		wm->video_work_mode = in->video_work_mode;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*wm);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE: {
+		struct hfi_videocores_usage_type *in = pdata, *cu = prop_data;
+
+		cu->video_core_enable_mask = in->video_core_enable_mask;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*cu);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VENC_MAX_BITRATE:
+		/* not implemented on Venus 4xx */
+		break;
+	default:
+		ret = pkt_session_set_property_3xx(pkt, cookie, ptype, pdata);
+		break;
+	}
+
+	return ret;
+}
+
 int pkt_session_get_property(struct hfi_session_get_property_pkt *pkt,
 			     void *cookie, u32 ptype)
 {
@@ -1181,7 +1240,10 @@ int pkt_session_set_property(struct hfi_session_set_property_pkt *pkt,
 	if (hfi_ver == HFI_VERSION_1XX)
 		return pkt_session_set_property_1x(pkt, cookie, ptype, pdata);
 
-	return pkt_session_set_property_3xx(pkt, cookie, ptype, pdata);
+	if (hfi_ver == HFI_VERSION_3XX)
+		return pkt_session_set_property_3xx(pkt, cookie, ptype, pdata);
+
+	return pkt_session_set_property_4xx(pkt, cookie, ptype, pdata);
 }
 
 void pkt_set_version(enum hfi_version version)
-- 
2.14.1

Re: [PATCH v2 04/29] venus: hfi_cmds: add set_properties for 4xx version

[Tomasz Figa]

On Tue, May 15, 2018 at 5:13 PM Stanimir Varbanov <
stanimir.varbanov@linaro.org> wrote:

> Adds set_properties method to handle newer 4xx properties and
> fall-back to 3xx for the rest.

> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>   drivers/media/platform/qcom/venus/hfi_cmds.c | 64
+++++++++++++++++++++++++++-
>   1 file changed, 63 insertions(+), 1 deletion(-)

> diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c
b/drivers/media/platform/qcom/venus/hfi_cmds.c
> index 1cfeb7743041..6bd287154796 100644
> --- a/drivers/media/platform/qcom/venus/hfi_cmds.c
> +++ b/drivers/media/platform/qcom/venus/hfi_cmds.c

[snip]

> +       case HFI_PROPERTY_CONFIG_VENC_MAX_BITRATE:
> +               /* not implemented on Venus 4xx */

Shouldn't return -EINVAL here, similar to what
pkt_session_set_property_1x() does for unknown property?

> +               break;
> +       default:
> +               ret = pkt_session_set_property_3xx(pkt, cookie, ptype,
pdata);
> +               break;

nit: How about simply return pkt_session_set_property_3xx(pkt, cookie,
ptype, pdata); and removing the |ret| variable completely, since the return
below the switch can just return 0 all the time?

> +       }
> +
> +       return ret;
> +}
> +
>   int pkt_session_get_property(struct hfi_session_get_property_pkt *pkt,
>                               void *cookie, u32 ptype)
>   {
> @@ -1181,7 +1240,10 @@ int pkt_session_set_property(struct
hfi_session_set_property_pkt *pkt,
>          if (hfi_ver == HFI_VERSION_1XX)
>                  return pkt_session_set_property_1x(pkt, cookie, ptype,
pdata);

> -       return pkt_session_set_property_3xx(pkt, cookie, ptype, pdata);
> +       if (hfi_ver == HFI_VERSION_3XX)
> +               return pkt_session_set_property_3xx(pkt, cookie, ptype,
pdata);
> +
> +       return pkt_session_set_property_4xx(pkt, cookie, ptype, pdata);

nit: Since we're adding third variant, I'd consider using function pointers
here, but no strong opinion.

Best regards,
Tomasz

Re: [PATCH v2 04/29] venus: hfi_cmds: add set_properties for 4xx version

[Stanimir Varbanov]

Hi Tomasz,

On 05/18/2018 05:16 PM, Tomasz Figa wrote:
> On Tue, May 15, 2018 at 5:13 PM Stanimir Varbanov <
> stanimir.varbanov@linaro.org> wrote:
> 
>> Adds set_properties method to handle newer 4xx properties and
>> fall-back to 3xx for the rest.
> 
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>   drivers/media/platform/qcom/venus/hfi_cmds.c | 64
> +++++++++++++++++++++++++++-
>>   1 file changed, 63 insertions(+), 1 deletion(-)
> 
>> diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c
> b/drivers/media/platform/qcom/venus/hfi_cmds.c
>> index 1cfeb7743041..6bd287154796 100644
>> --- a/drivers/media/platform/qcom/venus/hfi_cmds.c
>> +++ b/drivers/media/platform/qcom/venus/hfi_cmds.c
> 
> [snip]
> 
>> +       case HFI_PROPERTY_CONFIG_VENC_MAX_BITRATE:
>> +               /* not implemented on Venus 4xx */
> 
> Shouldn't return -EINVAL here, similar to what
> pkt_session_set_property_1x() does for unknown property?

Probably the right error code should be ENOTSUPP, but I kind of
following the rule to silently not return the error to simplify the
callers of set_property (otherwise I have to have a version conditional
code in the callers).

> 
>> +               break;
>> +       default:
>> +               ret = pkt_session_set_property_3xx(pkt, cookie, ptype,
> pdata);
>> +               break;
> 
> nit: How about simply return pkt_session_set_property_3xx(pkt, cookie,
> ptype, pdata); and removing the |ret| variable completely, since the return
> below the switch can just return 0 all the time?

OK, I will do that way.

> 
>> +       }
>> +
>> +       return ret;
>> +}
>> +
>>   int pkt_session_get_property(struct hfi_session_get_property_pkt *pkt,
>>                               void *cookie, u32 ptype)
>>   {
>> @@ -1181,7 +1240,10 @@ int pkt_session_set_property(struct
> hfi_session_set_property_pkt *pkt,
>>          if (hfi_ver == HFI_VERSION_1XX)
>>                  return pkt_session_set_property_1x(pkt, cookie, ptype,
> pdata);
> 
>> -       return pkt_session_set_property_3xx(pkt, cookie, ptype, pdata);
>> +       if (hfi_ver == HFI_VERSION_3XX)
>> +               return pkt_session_set_property_3xx(pkt, cookie, ptype,
> pdata);
>> +
>> +       return pkt_session_set_property_4xx(pkt, cookie, ptype, pdata);
> 
> nit: Since we're adding third variant, I'd consider using function pointers
> here, but no strong opinion.

Let's keep that for future improvements.

-- 
regards,
Stan

[PATCH v2 05/29] venus: hfi: support session continue for 4xx version

[Stanimir Varbanov]

This makes possible to handle session_continue for 4xx as well.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
index bca894a00c07..cbc6fad05e47 100644
--- a/drivers/media/platform/qcom/venus/hfi.c
+++ b/drivers/media/platform/qcom/venus/hfi.c
@@ -312,7 +312,7 @@ int hfi_session_continue(struct venus_inst *inst)
 {
 	struct venus_core *core = inst->core;
 
-	if (core->res->hfi_version != HFI_VERSION_3XX)
+	if (core->res->hfi_version == HFI_VERSION_1XX)
 		return 0;
 
 	return core->ops->session_continue(inst);
-- 
2.14.1

[PATCH v2 06/29] venus: hfi: handle buffer output2 type as well

[Stanimir Varbanov]

This adds handling of buffers of type OUTPUT2 which is needed to
support Venus 4xx version.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi.c      | 3 ++-
 drivers/media/platform/qcom/venus/hfi_msgs.c | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
index cbc6fad05e47..a570fdad0de0 100644
--- a/drivers/media/platform/qcom/venus/hfi.c
+++ b/drivers/media/platform/qcom/venus/hfi.c
@@ -473,7 +473,8 @@ int hfi_session_process_buf(struct venus_inst *inst, struct hfi_frame_data *fd)
 
 	if (fd->buffer_type == HFI_BUFFER_INPUT)
 		return ops->session_etb(inst, fd);
-	else if (fd->buffer_type == HFI_BUFFER_OUTPUT)
+	else if (fd->buffer_type == HFI_BUFFER_OUTPUT ||
+		 fd->buffer_type == HFI_BUFFER_OUTPUT2)
 		return ops->session_ftb(inst, fd);
 
 	return -EINVAL;
diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/venus/hfi_msgs.c
index 5970e9b1716b..023802e62833 100644
--- a/drivers/media/platform/qcom/venus/hfi_msgs.c
+++ b/drivers/media/platform/qcom/venus/hfi_msgs.c
@@ -825,7 +825,8 @@ static void hfi_session_ftb_done(struct venus_core *core,
 		error = HFI_ERR_SESSION_INVALID_PARAMETER;
 	}
 
-	if (buffer_type != HFI_BUFFER_OUTPUT)
+	if (buffer_type != HFI_BUFFER_OUTPUT &&
+	    buffer_type != HFI_BUFFER_OUTPUT2)
 		goto done;
 
 	if (hfi_flags & HFI_BUFFERFLAG_EOS)
-- 
2.14.1

[PATCH v2 07/29] venus: hfi_venus: add halt AXI support for Venus 4xx

[Stanimir Varbanov]

Add AXI halt support for version 4xx by using venus wrapper
registers.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi_venus.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/venus/hfi_venus.c
index 734ce11b0ed0..53546174aab8 100644
--- a/drivers/media/platform/qcom/venus/hfi_venus.c
+++ b/drivers/media/platform/qcom/venus/hfi_venus.c
@@ -532,6 +532,23 @@ static int venus_halt_axi(struct venus_hfi_device *hdev)
 	u32 val;
 	int ret;
 
+	if (hdev->core->res->hfi_version == HFI_VERSION_4XX) {
+		val = venus_readl(hdev, WRAPPER_CPU_AXI_HALT);
+		val |= BIT(16);
+		venus_writel(hdev, WRAPPER_CPU_AXI_HALT, val);
+
+		ret = readl_poll_timeout(base + WRAPPER_CPU_AXI_HALT_STATUS,
+					 val, val & BIT(24),
+					 POLL_INTERVAL_US,
+					 VBIF_AXI_HALT_ACK_TIMEOUT_US);
+		if (ret) {
+			dev_err(dev, "AXI bus port halt timeout\n");
+			return ret;
+		}
+
+		return 0;
+	}
+
 	/* Halt AXI and AXI IMEM VBIF Access */
 	val = venus_readl(hdev, VBIF_AXI_HALT_CTRL0);
 	val |= VBIF_AXI_HALT_CTRL0_HALT_REQ;
-- 
2.14.1

Re: [PATCH v2 07/29] venus: hfi_venus: add halt AXI support for Venus 4xx

[Tomasz Figa]

On Tue, May 15, 2018 at 5:12 PM Stanimir Varbanov <
stanimir.varbanov@linaro.org> wrote:

> Add AXI halt support for version 4xx by using venus wrapper
> registers.

> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>   drivers/media/platform/qcom/venus/hfi_venus.c | 17 +++++++++++++++++
>   1 file changed, 17 insertions(+)

> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c
b/drivers/media/platform/qcom/venus/hfi_venus.c
> index 734ce11b0ed0..53546174aab8 100644
> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
> +++ b/drivers/media/platform/qcom/venus/hfi_venus.c
> @@ -532,6 +532,23 @@ static int venus_halt_axi(struct venus_hfi_device
*hdev)
>          u32 val;
>          int ret;

> +       if (hdev->core->res->hfi_version == HFI_VERSION_4XX) {
> +               val = venus_readl(hdev, WRAPPER_CPU_AXI_HALT);
> +               val |= BIT(16);

Can we have the bit defined?

> +               venus_writel(hdev, WRAPPER_CPU_AXI_HALT, val);
> +
> +               ret = readl_poll_timeout(base +
WRAPPER_CPU_AXI_HALT_STATUS,
> +                                        val, val & BIT(24),

Ditto.

Best regards,
Tomasz

Re: [PATCH v2 07/29] venus: hfi_venus: add halt AXI support for Venus 4xx

[Stanimir Varbanov]

Hi Tomasz,

On 05/18/2018 05:23 PM, Tomasz Figa wrote:
> On Tue, May 15, 2018 at 5:12 PM Stanimir Varbanov <
> stanimir.varbanov@linaro.org> wrote:
> 
>> Add AXI halt support for version 4xx by using venus wrapper
>> registers.
> 
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>   drivers/media/platform/qcom/venus/hfi_venus.c | 17 +++++++++++++++++
>>   1 file changed, 17 insertions(+)
> 
>> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c
> b/drivers/media/platform/qcom/venus/hfi_venus.c
>> index 734ce11b0ed0..53546174aab8 100644
>> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
>> +++ b/drivers/media/platform/qcom/venus/hfi_venus.c
>> @@ -532,6 +532,23 @@ static int venus_halt_axi(struct venus_hfi_device
> *hdev)
>>          u32 val;
>>          int ret;
> 
>> +       if (hdev->core->res->hfi_version == HFI_VERSION_4XX) {
>> +               val = venus_readl(hdev, WRAPPER_CPU_AXI_HALT);
>> +               val |= BIT(16);
> 
> Can we have the bit defined?
> 
>> +               venus_writel(hdev, WRAPPER_CPU_AXI_HALT, val);
>> +
>> +               ret = readl_poll_timeout(base +
> WRAPPER_CPU_AXI_HALT_STATUS,
>> +                                        val, val & BIT(24),
> 
> Ditto.

Sure will add defines.

-- 
regards,
Stan

[PATCH v2 08/29] venus: hfi_venus: fix suspend function for venus 3xx versions

[Stanimir Varbanov]

This fixes the suspend function for Venus 3xx versions by
add a check for WFI (wait for interrupt) bit. This bit
is on when the ARM9 is idle and entered in low power mode.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi_venus.c    | 59 ++++++++++++++++--------
 drivers/media/platform/qcom/venus/hfi_venus_io.h |  1 +
 2 files changed, 41 insertions(+), 19 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/venus/hfi_venus.c
index 53546174aab8..aac351f699a0 100644
--- a/drivers/media/platform/qcom/venus/hfi_venus.c
+++ b/drivers/media/platform/qcom/venus/hfi_venus.c
@@ -1447,7 +1447,7 @@ static int venus_suspend_3xx(struct venus_core *core)
 {
 	struct venus_hfi_device *hdev = to_hfi_priv(core);
 	struct device *dev = core->dev;
-	u32 ctrl_status, wfi_status;
+	u32 ctrl_status, cpu_status;
 	int ret;
 	int cnt = 100;
 
@@ -1463,29 +1463,50 @@ static int venus_suspend_3xx(struct venus_core *core)
 		return -EINVAL;
 	}
 
-	ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
-	if (!(ctrl_status & CPU_CS_SCIACMDARG0_PC_READY)) {
-		wfi_status = venus_readl(hdev, WRAPPER_CPU_STATUS);
+	/*
+	 * Power collapse sequence for Venus 3xx and 4xx versions:
+	 * 1. Check for ARM9 and video core to be idle by checking WFI bit
+	 *    (bit 0) in CPU status register and by checking Idle (bit 30) in
+	 *    Control status register for video core.
+	 * 2. Send a command to prepare for power collapse.
+	 * 3. Check for WFI and PC_READY bits.
+	 */
+
+	while (--cnt) {
+		cpu_status = venus_readl(hdev, WRAPPER_CPU_STATUS);
 		ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
 
-		ret = venus_prepare_power_collapse(hdev, false);
-		if (ret) {
-			dev_err(dev, "prepare for power collapse fail (%d)\n",
-				ret);
-			return ret;
-		}
+		if (cpu_status & WRAPPER_CPU_STATUS_WFI &&
+		    ctrl_status & CPU_CS_SCIACMDARG0_INIT_IDLE_MSG_MASK)
+			break;
 
-		cnt = 100;
-		while (cnt--) {
-			wfi_status = venus_readl(hdev, WRAPPER_CPU_STATUS);
-			ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
-			if (ctrl_status & CPU_CS_SCIACMDARG0_PC_READY &&
-			    wfi_status & BIT(0))
-				break;
-			usleep_range(1000, 1500);
-		}
+		usleep_range(1000, 1500);
 	}
 
+	if (!cnt)
+		return -ETIMEDOUT;
+
+	ret = venus_prepare_power_collapse(hdev, false);
+	if (ret) {
+		dev_err(dev, "prepare for power collapse fail (%d)\n", ret);
+		return ret;
+	}
+
+	cnt = 100;
+	while (--cnt) {
+		cpu_status = venus_readl(hdev, WRAPPER_CPU_STATUS);
+		ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
+
+		if (cpu_status & WRAPPER_CPU_STATUS_WFI &&
+		    ctrl_status & CPU_CS_SCIACMDARG0_PC_READY)
+			break;
+
+		usleep_range(1000, 1500);
+	}
+
+	if (!cnt)
+		return -ETIMEDOUT;
+
 	mutex_lock(&hdev->lock);
 
 	ret = venus_power_off(hdev);
diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h b/drivers/media/platform/qcom/venus/hfi_venus_io.h
index 76f47936d0fa..12e3d33a3d82 100644
--- a/drivers/media/platform/qcom/venus/hfi_venus_io.h
+++ b/drivers/media/platform/qcom/venus/hfi_venus_io.h
@@ -108,6 +108,7 @@
 
 #define WRAPPER_CPU_CGC_DIS			(WRAPPER_BASE + 0x2010)
 #define WRAPPER_CPU_STATUS			(WRAPPER_BASE + 0x2014)
+#define WRAPPER_CPU_STATUS_WFI			BIT(0)
 #define WRAPPER_SW_RESET			(WRAPPER_BASE + 0x3000)
 
 /* Venus 4xx */
-- 
2.14.1

Re: [PATCH v2 08/29] venus: hfi_venus: fix suspend function for venus 3xx versions

[Tomasz Figa]

On Tue, May 15, 2018 at 5:11 PM Stanimir Varbanov <
stanimir.varbanov@linaro.org> wrote:

> This fixes the suspend function for Venus 3xx versions by
> add a check for WFI (wait for interrupt) bit. This bit
> is on when the ARM9 is idle and entered in low power mode.

> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>   drivers/media/platform/qcom/venus/hfi_venus.c    | 59
++++++++++++++++--------
>   drivers/media/platform/qcom/venus/hfi_venus_io.h |  1 +
>   2 files changed, 41 insertions(+), 19 deletions(-)

> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c
b/drivers/media/platform/qcom/venus/hfi_venus.c
> index 53546174aab8..aac351f699a0 100644
> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
> +++ b/drivers/media/platform/qcom/venus/hfi_venus.c
> @@ -1447,7 +1447,7 @@ static int venus_suspend_3xx(struct venus_core
*core)
>   {
>          struct venus_hfi_device *hdev = to_hfi_priv(core);
>          struct device *dev = core->dev;
> -       u32 ctrl_status, wfi_status;
> +       u32 ctrl_status, cpu_status;
>          int ret;
>          int cnt = 100;

> @@ -1463,29 +1463,50 @@ static int venus_suspend_3xx(struct venus_core
*core)
>                  return -EINVAL;
>          }

> -       ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
> -       if (!(ctrl_status & CPU_CS_SCIACMDARG0_PC_READY)) {
> -               wfi_status = venus_readl(hdev, WRAPPER_CPU_STATUS);
> +       /*
> +        * Power collapse sequence for Venus 3xx and 4xx versions:
> +        * 1. Check for ARM9 and video core to be idle by checking WFI bit
> +        *    (bit 0) in CPU status register and by checking Idle (bit
30) in
> +        *    Control status register for video core.
> +        * 2. Send a command to prepare for power collapse.
> +        * 3. Check for WFI and PC_READY bits.
> +        */
> +
> +       while (--cnt) {
> +               cpu_status = venus_readl(hdev, WRAPPER_CPU_STATUS);
>                  ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);

> -               ret = venus_prepare_power_collapse(hdev, false);
> -               if (ret) {
> -                       dev_err(dev, "prepare for power collapse fail
(%d)\n",
> -                               ret);
> -                       return ret;
> -               }
> +               if (cpu_status & WRAPPER_CPU_STATUS_WFI &&
> +                   ctrl_status & CPU_CS_SCIACMDARG0_INIT_IDLE_MSG_MASK)
> +                       break;

> -               cnt = 100;
> -               while (cnt--) {
> -                       wfi_status = venus_readl(hdev,
WRAPPER_CPU_STATUS);
> -                       ctrl_status = venus_readl(hdev,
CPU_CS_SCIACMDARG0);
> -                       if (ctrl_status & CPU_CS_SCIACMDARG0_PC_READY &&
> -                           wfi_status & BIT(0))
> -                               break;
> -                       usleep_range(1000, 1500);
> -               }
> +               usleep_range(1000, 1500);
>          }

To avoid opencoding the polling, I'd suggest doing a readx_poll_timeout()
trick:

static bool venus_arm9_and_video_core_idle(struct venus_hfi_device *hdev)
{
         // Read both registers and return true if both have the right bits
set
}

static int venus_suspend_3xx(struct venus_core *core)
{
         bool val;
         int ret;
         // ...
         ret = readx_poll_timeout(venus_arm9_and_video_core_idle, hdev, val,
val, 1500, 100 * 1500);
         if (ret)
                 return ret;
         // ...
}


> +       if (!cnt)
> +               return -ETIMEDOUT;
> +
> +       ret = venus_prepare_power_collapse(hdev, false);
> +       if (ret) {
> +               dev_err(dev, "prepare for power collapse fail (%d)\n",
ret);
> +               return ret;
> +       }
> +
> +       cnt = 100;
> +       while (--cnt) {
> +               cpu_status = venus_readl(hdev, WRAPPER_CPU_STATUS);
> +               ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
> +
> +               if (cpu_status & WRAPPER_CPU_STATUS_WFI &&
> +                   ctrl_status & CPU_CS_SCIACMDARG0_PC_READY)
> +                       break;
> +
> +               usleep_range(1000, 1500);
> +       }
> +
> +       if (!cnt)
> +               return -ETIMEDOUT;

Same readx_poll_timeout() trick can be used here, with different helper
function, e.g. venus_arm9_idle_and_pc_ready().

Best regards,
Tomasz

Re: [PATCH v2 08/29] venus: hfi_venus: fix suspend function for venus 3xx versions

[Stanimir Varbanov]

Hi Tomasz,

On 05/18/2018 06:14 PM, Tomasz Figa wrote:
> On Tue, May 15, 2018 at 5:11 PM Stanimir Varbanov <
> stanimir.varbanov@linaro.org> wrote:
> 
>> This fixes the suspend function for Venus 3xx versions by
>> add a check for WFI (wait for interrupt) bit. This bit
>> is on when the ARM9 is idle and entered in low power mode.
> 
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>   drivers/media/platform/qcom/venus/hfi_venus.c    | 59
> ++++++++++++++++--------
>>   drivers/media/platform/qcom/venus/hfi_venus_io.h |  1 +
>>   2 files changed, 41 insertions(+), 19 deletions(-)
> 
>> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c
> b/drivers/media/platform/qcom/venus/hfi_venus.c
>> index 53546174aab8..aac351f699a0 100644
>> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
>> +++ b/drivers/media/platform/qcom/venus/hfi_venus.c
>> @@ -1447,7 +1447,7 @@ static int venus_suspend_3xx(struct venus_core
> *core)
>>   {
>>          struct venus_hfi_device *hdev = to_hfi_priv(core);
>>          struct device *dev = core->dev;
>> -       u32 ctrl_status, wfi_status;
>> +       u32 ctrl_status, cpu_status;
>>          int ret;
>>          int cnt = 100;
> 
>> @@ -1463,29 +1463,50 @@ static int venus_suspend_3xx(struct venus_core
> *core)
>>                  return -EINVAL;
>>          }
> 
>> -       ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
>> -       if (!(ctrl_status & CPU_CS_SCIACMDARG0_PC_READY)) {
>> -               wfi_status = venus_readl(hdev, WRAPPER_CPU_STATUS);
>> +       /*
>> +        * Power collapse sequence for Venus 3xx and 4xx versions:
>> +        * 1. Check for ARM9 and video core to be idle by checking WFI bit
>> +        *    (bit 0) in CPU status register and by checking Idle (bit
> 30) in
>> +        *    Control status register for video core.
>> +        * 2. Send a command to prepare for power collapse.
>> +        * 3. Check for WFI and PC_READY bits.
>> +        */
>> +
>> +       while (--cnt) {
>> +               cpu_status = venus_readl(hdev, WRAPPER_CPU_STATUS);
>>                  ctrl_status = venus_readl(hdev, CPU_CS_SCIACMDARG0);
> 
>> -               ret = venus_prepare_power_collapse(hdev, false);
>> -               if (ret) {
>> -                       dev_err(dev, "prepare for power collapse fail
> (%d)\n",
>> -                               ret);
>> -                       return ret;
>> -               }
>> +               if (cpu_status & WRAPPER_CPU_STATUS_WFI &&
>> +                   ctrl_status & CPU_CS_SCIACMDARG0_INIT_IDLE_MSG_MASK)
>> +                       break;
> 
>> -               cnt = 100;
>> -               while (cnt--) {
>> -                       wfi_status = venus_readl(hdev,
> WRAPPER_CPU_STATUS);
>> -                       ctrl_status = venus_readl(hdev,
> CPU_CS_SCIACMDARG0);
>> -                       if (ctrl_status & CPU_CS_SCIACMDARG0_PC_READY &&
>> -                           wfi_status & BIT(0))
>> -                               break;
>> -                       usleep_range(1000, 1500);
>> -               }
>> +               usleep_range(1000, 1500);
>>          }
> 
> To avoid opencoding the polling, I'd suggest doing a readx_poll_timeout()
> trick:

I like the idea, will try to rework that and use readx_poll_timeout.

<snip>

-- 
regards,
Stan

[PATCH v2 09/29] venus: hfi_venus: move set of default properties to core init

[Stanimir Varbanov]

This moves setting of default properties (firmware debug, idle
indicator and low power mode) from session init to core init.
All of those properties are need to be enabled/disabled early
so that they could be used before the clients are even initialized.

The other reason is to set idle indicator property early before
we enter into venus_suspend function where we need to check for
ARM9 WFI.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi_venus.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/venus/hfi_venus.c
index aac351f699a0..284da69eb81b 100644
--- a/drivers/media/platform/qcom/venus/hfi_venus.c
+++ b/drivers/media/platform/qcom/venus/hfi_venus.c
@@ -1090,6 +1090,10 @@ static int venus_core_init(struct venus_core *core)
 	if (ret)
 		dev_warn(dev, "failed to send image version pkt to fw\n");
 
+	ret = venus_sys_set_default_properties(hdev);
+	if (ret)
+		return ret;
+
 	return 0;
 }
 
@@ -1134,10 +1138,6 @@ static int venus_session_init(struct venus_inst *inst, u32 session_type,
 	struct hfi_session_init_pkt pkt;
 	int ret;
 
-	ret = venus_sys_set_default_properties(hdev);
-	if (ret)
-		return ret;
-
 	ret = pkt_session_init(&pkt, inst, session_type, codec);
 	if (ret)
 		goto err;
-- 
2.14.1

[PATCH v2 10/29] venus: hfi_venus: add suspend functionality for Venus 4xx

[Stanimir Varbanov]

This adds suspend (power collapse) functionality by reusing
the suspend function for Venus 3xx and also enables idle indicator
property for Venus 4xx (where it is disabled by default).

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi_venus.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/venus/hfi_venus.c
index 284da69eb81b..109116e1545d 100644
--- a/drivers/media/platform/qcom/venus/hfi_venus.c
+++ b/drivers/media/platform/qcom/venus/hfi_venus.c
@@ -878,6 +878,14 @@ static int venus_sys_set_default_properties(struct venus_hfi_device *hdev)
 	if (ret)
 		dev_warn(dev, "setting fw debug msg ON failed (%d)\n", ret);
 
+	/*
+	 * Idle indicator is disabled by default on some 4xx firmware versions,
+	 * enable it explicitly in order to make suspend functional by checking
+	 * WFI (wait-for-interrupt) bit.
+	 */
+	if (IS_V4(hdev->core))
+		venus_sys_idle_indicator = true;
+
 	ret = venus_sys_set_idle_message(hdev, venus_sys_idle_indicator);
 	if (ret)
 		dev_warn(dev, "setting idle response ON failed (%d)\n", ret);
@@ -1525,7 +1533,8 @@ static int venus_suspend_3xx(struct venus_core *core)
 
 static int venus_suspend(struct venus_core *core)
 {
-	if (core->res->hfi_version == HFI_VERSION_3XX)
+	if (core->res->hfi_version == HFI_VERSION_3XX ||
+	    core->res->hfi_version == HFI_VERSION_4XX)
 		return venus_suspend_3xx(core);
 
 	return venus_suspend_1xx(core);
-- 
2.14.1

[PATCH v2 11/29] venus: venc,vdec: adds clocks needed for venus 4xx

[Stanimir Varbanov]

This extends the clocks number to support suspend and resume
on Venus version 4xx.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h |  4 +--
 drivers/media/platform/qcom/venus/vdec.c | 42 ++++++++++++++++++++++++++------
 drivers/media/platform/qcom/venus/venc.c | 42 ++++++++++++++++++++++++++------
 3 files changed, 72 insertions(+), 16 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 8d3e150800c9..b5b9a84e9155 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -92,8 +92,8 @@ struct venus_core {
 	void __iomem *base;
 	int irq;
 	struct clk *clks[VIDC_CLKS_NUM_MAX];
-	struct clk *core0_clk;
-	struct clk *core1_clk;
+	struct clk *core0_clk, *core0_bus_clk;
+	struct clk *core1_clk, *core1_bus_clk;
 	struct video_device *vdev_dec;
 	struct video_device *vdev_enc;
 	struct v4l2_device v4l2_dev;
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 261a51adeef2..c45452634e7e 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -1081,12 +1081,18 @@ static int vdec_probe(struct platform_device *pdev)
 	if (!core)
 		return -EPROBE_DEFER;
 
-	if (core->res->hfi_version == HFI_VERSION_3XX) {
+	if (IS_V3(core) || IS_V4(core)) {
 		core->core0_clk = devm_clk_get(dev, "core");
 		if (IS_ERR(core->core0_clk))
 			return PTR_ERR(core->core0_clk);
 	}
 
+	if (IS_V4(core)) {
+		core->core0_bus_clk = devm_clk_get(dev, "bus");
+		if (IS_ERR(core->core0_bus_clk))
+			return PTR_ERR(core->core0_bus_clk);
+	}
+
 	platform_set_drvdata(pdev, core);
 
 	vdev = video_device_alloc();
@@ -1132,12 +1138,23 @@ static __maybe_unused int vdec_runtime_suspend(struct device *dev)
 {
 	struct venus_core *core = dev_get_drvdata(dev);
 
-	if (core->res->hfi_version == HFI_VERSION_1XX)
+	if (IS_V1(core))
 		return 0;
 
-	writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	if (IS_V3(core))
+		writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(0, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+
+	if (IS_V4(core))
+		clk_disable_unprepare(core->core0_bus_clk);
+
 	clk_disable_unprepare(core->core0_clk);
-	writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+
+	if (IS_V3(core))
+		writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(1, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
 
 	return 0;
 }
@@ -1147,12 +1164,23 @@ static __maybe_unused int vdec_runtime_resume(struct device *dev)
 	struct venus_core *core = dev_get_drvdata(dev);
 	int ret;
 
-	if (core->res->hfi_version == HFI_VERSION_1XX)
+	if (IS_V1(core))
 		return 0;
 
-	writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	if (IS_V3(core))
+		writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(0, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+
 	ret = clk_prepare_enable(core->core0_clk);
-	writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+
+	if (IS_V4(core))
+		ret |= clk_prepare_enable(core->core0_bus_clk);
+
+	if (IS_V3(core))
+		writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(1, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
 
 	return ret;
 }
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 947001170a77..bc8c2e7a8d2c 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -1225,12 +1225,18 @@ static int venc_probe(struct platform_device *pdev)
 	if (!core)
 		return -EPROBE_DEFER;
 
-	if (core->res->hfi_version == HFI_VERSION_3XX) {
+	if (IS_V3(core) || IS_V4(core)) {
 		core->core1_clk = devm_clk_get(dev, "core");
 		if (IS_ERR(core->core1_clk))
 			return PTR_ERR(core->core1_clk);
 	}
 
+	if (IS_V4(core)) {
+		core->core1_bus_clk = devm_clk_get(dev, "bus");
+		if (IS_ERR(core->core1_bus_clk))
+			return PTR_ERR(core->core1_bus_clk);
+	}
+
 	platform_set_drvdata(pdev, core);
 
 	vdev = video_device_alloc();
@@ -1276,12 +1282,23 @@ static __maybe_unused int venc_runtime_suspend(struct device *dev)
 {
 	struct venus_core *core = dev_get_drvdata(dev);
 
-	if (core->res->hfi_version == HFI_VERSION_1XX)
+	if (IS_V1(core))
 		return 0;
 
-	writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	if (IS_V3(core))
+		writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(0, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+
+	if (IS_V4(core))
+		clk_disable_unprepare(core->core1_bus_clk);
+
 	clk_disable_unprepare(core->core1_clk);
-	writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+
+	if (IS_V3(core))
+		writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(1, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
 
 	return 0;
 }
@@ -1291,12 +1308,23 @@ static __maybe_unused int venc_runtime_resume(struct device *dev)
 	struct venus_core *core = dev_get_drvdata(dev);
 	int ret;
 
-	if (core->res->hfi_version == HFI_VERSION_1XX)
+	if (IS_V1(core))
 		return 0;
 
-	writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	if (IS_V3(core))
+		writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(0, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+
 	ret = clk_prepare_enable(core->core1_clk);
-	writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+
+	if (IS_V4(core))
+		ret |= clk_prepare_enable(core->core1_bus_clk);
+
+	if (IS_V3(core))
+		writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(1, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
 
 	return ret;
 }
-- 
2.14.1

Re: [PATCH v2 11/29] venus: venc,vdec: adds clocks needed for venus 4xx

[Tomasz Figa]

Hi Stanimir,

On Tue, May 15, 2018 at 5:10 PM Stanimir Varbanov <
stanimir.varbanov@linaro.org> wrote:

> This extends the clocks number to support suspend and resume
> on Venus version 4xx.

> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>   drivers/media/platform/qcom/venus/core.h |  4 +--
>   drivers/media/platform/qcom/venus/vdec.c | 42
++++++++++++++++++++++++++------
>   drivers/media/platform/qcom/venus/venc.c | 42
++++++++++++++++++++++++++------
>   3 files changed, 72 insertions(+), 16 deletions(-)

> diff --git a/drivers/media/platform/qcom/venus/core.h
b/drivers/media/platform/qcom/venus/core.h
> index 8d3e150800c9..b5b9a84e9155 100644
> --- a/drivers/media/platform/qcom/venus/core.h
> +++ b/drivers/media/platform/qcom/venus/core.h
> @@ -92,8 +92,8 @@ struct venus_core {
>          void __iomem *base;
>          int irq;
>          struct clk *clks[VIDC_CLKS_NUM_MAX];
> -       struct clk *core0_clk;
> -       struct clk *core1_clk;
> +       struct clk *core0_clk, *core0_bus_clk;
> +       struct clk *core1_clk, *core1_bus_clk;
>          struct video_device *vdev_dec;
>          struct video_device *vdev_enc;
>          struct v4l2_device v4l2_dev;
> diff --git a/drivers/media/platform/qcom/venus/vdec.c
b/drivers/media/platform/qcom/venus/vdec.c
> index 261a51adeef2..c45452634e7e 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -1081,12 +1081,18 @@ static int vdec_probe(struct platform_device
*pdev)
>          if (!core)
>                  return -EPROBE_DEFER;

> -       if (core->res->hfi_version == HFI_VERSION_3XX) {
> +       if (IS_V3(core) || IS_V4(core)) {
>                  core->core0_clk = devm_clk_get(dev, "core");
>                  if (IS_ERR(core->core0_clk))
>                          return PTR_ERR(core->core0_clk);
>          }

> +       if (IS_V4(core)) {
> +               core->core0_bus_clk = devm_clk_get(dev, "bus");
> +               if (IS_ERR(core->core0_bus_clk))
> +                       return PTR_ERR(core->core0_bus_clk);
> +       }
> +

Rather than doing this conditional dance, wouldn't it make more sense to
just list all the clocks in variant data struct and use clk_bulk_get()?

>          platform_set_drvdata(pdev, core);

>          vdev = video_device_alloc();
> @@ -1132,12 +1138,23 @@ static __maybe_unused int
vdec_runtime_suspend(struct device *dev)
>   {
>          struct venus_core *core = dev_get_drvdata(dev);

> -       if (core->res->hfi_version == HFI_VERSION_1XX)
> +       if (IS_V1(core))
>                  return 0;

> -       writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> +       if (IS_V3(core))
> +               writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> +       else if (IS_V4(core))
> +               writel(0, core->base +
WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
> +
> +       if (IS_V4(core))
> +               clk_disable_unprepare(core->core0_bus_clk);
> +
>          clk_disable_unprepare(core->core0_clk);
> -       writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> +
> +       if (IS_V3(core))
> +               writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> +       else if (IS_V4(core))
> +               writel(1, core->base +
WRAPPER_VCODEC0_MMCC_POWER_CONTROL);

Almost every step here differs between version. I'd suggest splitting this
into separate functions for both versions.


>          return 0;
>   }
> @@ -1147,12 +1164,23 @@ static __maybe_unused int
vdec_runtime_resume(struct device *dev)
>          struct venus_core *core = dev_get_drvdata(dev);
>          int ret;

> -       if (core->res->hfi_version == HFI_VERSION_1XX)
> +       if (IS_V1(core))
>                  return 0;

> -       writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> +       if (IS_V3(core))
> +               writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> +       else if (IS_V4(core))
> +               writel(0, core->base +
WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
> +
>          ret = clk_prepare_enable(core->core0_clk);
> -       writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> +
> +       if (IS_V4(core))
> +               ret |= clk_prepare_enable(core->core0_bus_clk);

Please don't OR error codes. If both calls end up with different errors,
the resulting value would be meaningless.

> +
> +       if (IS_V3(core))
> +               writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> +       else if (IS_V4(core))
> +               writel(1, core->base +
WRAPPER_VCODEC0_MMCC_POWER_CONTROL);

>          return ret;
>   }

Same for this function. I'd see separate function for each version.

> diff --git a/drivers/media/platform/qcom/venus/venc.c
b/drivers/media/platform/qcom/venus/venc.c
> index 947001170a77..bc8c2e7a8d2c 100644
> --- a/drivers/media/platform/qcom/venus/venc.c
> +++ b/drivers/media/platform/qcom/venus/venc.c
> @@ -1225,12 +1225,18 @@ static int venc_probe(struct platform_device
*pdev)
>          if (!core)
>                  return -EPROBE_DEFER;

> -       if (core->res->hfi_version == HFI_VERSION_3XX) {
> +       if (IS_V3(core) || IS_V4(core)) {
>                  core->core1_clk = devm_clk_get(dev, "core");
>                  if (IS_ERR(core->core1_clk))
>                          return PTR_ERR(core->core1_clk);
>          }

> +       if (IS_V4(core)) {
> +               core->core1_bus_clk = devm_clk_get(dev, "bus");
> +               if (IS_ERR(core->core1_bus_clk))
> +                       return PTR_ERR(core->core1_bus_clk);
> +       }
> +

Same as for corresponding function of vdec.

>          platform_set_drvdata(pdev, core);

>          vdev = video_device_alloc();
> @@ -1276,12 +1282,23 @@ static __maybe_unused int
venc_runtime_suspend(struct device *dev)
>   {
>          struct venus_core *core = dev_get_drvdata(dev);

> -       if (core->res->hfi_version == HFI_VERSION_1XX)
> +       if (IS_V1(core))
>                  return 0;

> -       writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
> +       if (IS_V3(core))
> +               writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
> +       else if (IS_V4(core))
> +               writel(0, core->base +
WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
> +
> +       if (IS_V4(core))
> +               clk_disable_unprepare(core->core1_bus_clk);
> +
>          clk_disable_unprepare(core->core1_clk);
> -       writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
> +
> +       if (IS_V3(core))
> +               writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
> +       else if (IS_V4(core))
> +               writel(1, core->base +
WRAPPER_VCODEC1_MMCC_POWER_CONTROL);

>          return 0;
>   }

Same as for vdec.

> @@ -1291,12 +1308,23 @@ static __maybe_unused int
venc_runtime_resume(struct device *dev)
>          struct venus_core *core = dev_get_drvdata(dev);
>          int ret;

> -       if (core->res->hfi_version == HFI_VERSION_1XX)
> +       if (IS_V1(core))
>                  return 0;

> -       writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
> +       if (IS_V3(core))
> +               writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
> +       else if (IS_V4(core))
> +               writel(0, core->base +
WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
> +
>          ret = clk_prepare_enable(core->core1_clk);
> -       writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
> +
> +       if (IS_V4(core))
> +               ret |= clk_prepare_enable(core->core1_bus_clk);
> +
> +       if (IS_V3(core))
> +               writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
> +       else if (IS_V4(core))
> +               writel(1, core->base +
WRAPPER_VCODEC1_MMCC_POWER_CONTROL);

>          return ret;
>   }

Same as for vdec.

Best regards,
Tomasz

Re: [PATCH v2 11/29] venus: venc,vdec: adds clocks needed for venus 4xx

[Stanimir Varbanov]

Hi Tomasz,

On 05/24/2018 09:11 AM, Tomasz Figa wrote:
> Hi Stanimir,
> 
> On Tue, May 15, 2018 at 5:10 PM Stanimir Varbanov <
> stanimir.varbanov@linaro.org> wrote:
> 
>> This extends the clocks number to support suspend and resume
>> on Venus version 4xx.
> 
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>   drivers/media/platform/qcom/venus/core.h |  4 +--
>>   drivers/media/platform/qcom/venus/vdec.c | 42
> ++++++++++++++++++++++++++------
>>   drivers/media/platform/qcom/venus/venc.c | 42
> ++++++++++++++++++++++++++------
>>   3 files changed, 72 insertions(+), 16 deletions(-)
> 
>> diff --git a/drivers/media/platform/qcom/venus/core.h
> b/drivers/media/platform/qcom/venus/core.h
>> index 8d3e150800c9..b5b9a84e9155 100644
>> --- a/drivers/media/platform/qcom/venus/core.h
>> +++ b/drivers/media/platform/qcom/venus/core.h
>> @@ -92,8 +92,8 @@ struct venus_core {
>>          void __iomem *base;
>>          int irq;
>>          struct clk *clks[VIDC_CLKS_NUM_MAX];
>> -       struct clk *core0_clk;
>> -       struct clk *core1_clk;
>> +       struct clk *core0_clk, *core0_bus_clk;
>> +       struct clk *core1_clk, *core1_bus_clk;
>>          struct video_device *vdev_dec;
>>          struct video_device *vdev_enc;
>>          struct v4l2_device v4l2_dev;
>> diff --git a/drivers/media/platform/qcom/venus/vdec.c
> b/drivers/media/platform/qcom/venus/vdec.c
>> index 261a51adeef2..c45452634e7e 100644
>> --- a/drivers/media/platform/qcom/venus/vdec.c
>> +++ b/drivers/media/platform/qcom/venus/vdec.c
>> @@ -1081,12 +1081,18 @@ static int vdec_probe(struct platform_device
> *pdev)
>>          if (!core)
>>                  return -EPROBE_DEFER;
> 
>> -       if (core->res->hfi_version == HFI_VERSION_3XX) {
>> +       if (IS_V3(core) || IS_V4(core)) {
>>                  core->core0_clk = devm_clk_get(dev, "core");
>>                  if (IS_ERR(core->core0_clk))
>>                          return PTR_ERR(core->core0_clk);
>>          }
> 
>> +       if (IS_V4(core)) {
>> +               core->core0_bus_clk = devm_clk_get(dev, "bus");
>> +               if (IS_ERR(core->core0_bus_clk))
>> +                       return PTR_ERR(core->core0_bus_clk);
>> +       }
>> +
> 
> Rather than doing this conditional dance, wouldn't it make more sense to
> just list all the clocks in variant data struct and use clk_bulk_get()?

Do you mean the same as it is done for venus/core.c ?

> 
>>          platform_set_drvdata(pdev, core);
> 
>>          vdev = video_device_alloc();
>> @@ -1132,12 +1138,23 @@ static __maybe_unused int
> vdec_runtime_suspend(struct device *dev)
>>   {
>>          struct venus_core *core = dev_get_drvdata(dev);
> 
>> -       if (core->res->hfi_version == HFI_VERSION_1XX)
>> +       if (IS_V1(core))
>>                  return 0;
> 
>> -       writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
>> +       if (IS_V3(core))
>> +               writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
>> +       else if (IS_V4(core))
>> +               writel(0, core->base +
> WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
>> +
>> +       if (IS_V4(core))
>> +               clk_disable_unprepare(core->core0_bus_clk);
>> +
>>          clk_disable_unprepare(core->core0_clk);
>> -       writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
>> +
>> +       if (IS_V3(core))
>> +               writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
>> +       else if (IS_V4(core))
>> +               writel(1, core->base +
> WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
> 
> Almost every step here differs between version. I'd suggest splitting this
> into separate functions for both versions.

I think it will be better to squash this patch with 13/29.


-- 
regards,
Stan

Re: [PATCH v2 11/29] venus: venc,vdec: adds clocks needed for venus 4xx

[Tomasz Figa]

On Mon, May 28, 2018 at 5:47 PM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> Hi Tomasz,
>
> On 05/24/2018 09:11 AM, Tomasz Figa wrote:
> > Hi Stanimir,
> >
> > On Tue, May 15, 2018 at 5:10 PM Stanimir Varbanov <
> > stanimir.varbanov@linaro.org> wrote:
> >
> >> This extends the clocks number to support suspend and resume
> >> on Venus version 4xx.
> >
> >> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> >> ---
> >>   drivers/media/platform/qcom/venus/core.h |  4 +--
> >>   drivers/media/platform/qcom/venus/vdec.c | 42
> > ++++++++++++++++++++++++++------
> >>   drivers/media/platform/qcom/venus/venc.c | 42
> > ++++++++++++++++++++++++++------
> >>   3 files changed, 72 insertions(+), 16 deletions(-)
> >
> >> diff --git a/drivers/media/platform/qcom/venus/core.h
> > b/drivers/media/platform/qcom/venus/core.h
> >> index 8d3e150800c9..b5b9a84e9155 100644
> >> --- a/drivers/media/platform/qcom/venus/core.h
> >> +++ b/drivers/media/platform/qcom/venus/core.h
> >> @@ -92,8 +92,8 @@ struct venus_core {
> >>          void __iomem *base;
> >>          int irq;
> >>          struct clk *clks[VIDC_CLKS_NUM_MAX];
> >> -       struct clk *core0_clk;
> >> -       struct clk *core1_clk;
> >> +       struct clk *core0_clk, *core0_bus_clk;
> >> +       struct clk *core1_clk, *core1_bus_clk;
> >>          struct video_device *vdev_dec;
> >>          struct video_device *vdev_enc;
> >>          struct v4l2_device v4l2_dev;
> >> diff --git a/drivers/media/platform/qcom/venus/vdec.c
> > b/drivers/media/platform/qcom/venus/vdec.c
> >> index 261a51adeef2..c45452634e7e 100644
> >> --- a/drivers/media/platform/qcom/venus/vdec.c
> >> +++ b/drivers/media/platform/qcom/venus/vdec.c
> >> @@ -1081,12 +1081,18 @@ static int vdec_probe(struct platform_device
> > *pdev)
> >>          if (!core)
> >>                  return -EPROBE_DEFER;
> >
> >> -       if (core->res->hfi_version == HFI_VERSION_3XX) {
> >> +       if (IS_V3(core) || IS_V4(core)) {
> >>                  core->core0_clk = devm_clk_get(dev, "core");
> >>                  if (IS_ERR(core->core0_clk))
> >>                          return PTR_ERR(core->core0_clk);
> >>          }
> >
> >> +       if (IS_V4(core)) {
> >> +               core->core0_bus_clk = devm_clk_get(dev, "bus");
> >> +               if (IS_ERR(core->core0_bus_clk))
> >> +                       return PTR_ERR(core->core0_bus_clk);
> >> +       }
> >> +
> >
> > Rather than doing this conditional dance, wouldn't it make more sense to
> > just list all the clocks in variant data struct and use clk_bulk_get()?
>
> Do you mean the same as it is done for venus/core.c ?

I mean clk_bulk_get() as in drivers/clk/clk-bulk.c. I guess
venus/core.c would also benefit from switching to these helpers.

>
> >
> >>          platform_set_drvdata(pdev, core);
> >
> >>          vdev = video_device_alloc();
> >> @@ -1132,12 +1138,23 @@ static __maybe_unused int
> > vdec_runtime_suspend(struct device *dev)
> >>   {
> >>          struct venus_core *core = dev_get_drvdata(dev);
> >
> >> -       if (core->res->hfi_version == HFI_VERSION_1XX)
> >> +       if (IS_V1(core))
> >>                  return 0;
> >
> >> -       writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> >> +       if (IS_V3(core))
> >> +               writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> >> +       else if (IS_V4(core))
> >> +               writel(0, core->base +
> > WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
> >> +
> >> +       if (IS_V4(core))
> >> +               clk_disable_unprepare(core->core0_bus_clk);
> >> +
> >>          clk_disable_unprepare(core->core0_clk);
> >> -       writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> >> +
> >> +       if (IS_V3(core))
> >> +               writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> >> +       else if (IS_V4(core))
> >> +               writel(1, core->base +
> > WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
> >
> > Almost every step here differs between version. I'd suggest splitting this
> > into separate functions for both versions.
>
> I think it will be better to squash this patch with 13/29.

I see. Let me review patch 13 first then.

Best regards,
Tomasz

[PATCH v2 12/29] venus: add common capability parser

[Stanimir Varbanov]

This adds common capability parser for all supported Venus
versions. Having it will help to enumerate better the supported
raw formars and codecs and also the capabilities for every
codec like max/min width/height, framerate, bitrate and so on.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/Makefile     |   3 +-
 drivers/media/platform/qcom/venus/core.c       |  85 ++++++
 drivers/media/platform/qcom/venus/core.h       |  68 +++--
 drivers/media/platform/qcom/venus/hfi.c        |   5 +-
 drivers/media/platform/qcom/venus/hfi_helper.h |  28 +-
 drivers/media/platform/qcom/venus/hfi_msgs.c   | 348 ++-----------------------
 drivers/media/platform/qcom/venus/hfi_parser.c | 295 +++++++++++++++++++++
 drivers/media/platform/qcom/venus/hfi_parser.h |  45 ++++
 drivers/media/platform/qcom/venus/vdec.c       |  38 +--
 drivers/media/platform/qcom/venus/venc.c       |  52 ++--
 10 files changed, 530 insertions(+), 437 deletions(-)
 create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.c
 create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.h

diff --git a/drivers/media/platform/qcom/venus/Makefile b/drivers/media/platform/qcom/venus/Makefile
index bfd4edf7c83f..b44b11b03e12 100644
--- a/drivers/media/platform/qcom/venus/Makefile
+++ b/drivers/media/platform/qcom/venus/Makefile
@@ -2,7 +2,8 @@
 # Makefile for Qualcomm Venus driver
 
 venus-core-objs += core.o helpers.o firmware.o \
-		   hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o
+		   hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
+		   hfi_parser.o
 
 venus-dec-objs += vdec.o vdec_ctrls.o
 venus-enc-objs += venc.o venc_ctrls.o
diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/venus/core.c
index 41eef376eb2d..381bfdd688db 100644
--- a/drivers/media/platform/qcom/venus/core.c
+++ b/drivers/media/platform/qcom/venus/core.c
@@ -152,6 +152,83 @@ static void venus_clks_disable(struct venus_core *core)
 		clk_disable_unprepare(core->clks[i]);
 }
 
+static u32 to_v4l2_codec_type(u32 codec)
+{
+	switch (codec) {
+	case HFI_VIDEO_CODEC_H264:
+		return V4L2_PIX_FMT_H264;
+	case HFI_VIDEO_CODEC_H263:
+		return V4L2_PIX_FMT_H263;
+	case HFI_VIDEO_CODEC_MPEG1:
+		return V4L2_PIX_FMT_MPEG1;
+	case HFI_VIDEO_CODEC_MPEG2:
+		return V4L2_PIX_FMT_MPEG2;
+	case HFI_VIDEO_CODEC_MPEG4:
+		return V4L2_PIX_FMT_MPEG4;
+	case HFI_VIDEO_CODEC_VC1:
+		return V4L2_PIX_FMT_VC1_ANNEX_G;
+	case HFI_VIDEO_CODEC_VP8:
+		return V4L2_PIX_FMT_VP8;
+	case HFI_VIDEO_CODEC_VP9:
+		return V4L2_PIX_FMT_VP9;
+	case HFI_VIDEO_CODEC_DIVX:
+	case HFI_VIDEO_CODEC_DIVX_311:
+		return V4L2_PIX_FMT_XVID;
+	default:
+		return 0;
+	}
+}
+
+static int venus_enumerate_codecs(struct venus_core *core, u32 type)
+{
+	const struct hfi_inst_ops dummy_ops = {};
+	struct venus_inst *inst;
+	u32 codec, codecs;
+	unsigned int i;
+	int ret;
+
+	if (core->res->hfi_version != HFI_VERSION_1XX)
+		return 0;
+
+	inst = kzalloc(sizeof(*inst), GFP_KERNEL);
+	if (!inst)
+		return -ENOMEM;
+
+	mutex_init(&inst->lock);
+	inst->core = core;
+	inst->session_type = type;
+	if (type == VIDC_SESSION_TYPE_DEC)
+		codecs = core->dec_codecs;
+	else
+		codecs = core->enc_codecs;
+
+	ret = hfi_session_create(inst, &dummy_ops);
+	if (ret)
+		goto err;
+
+	for (i = 0; i < MAX_CODEC_NUM; i++) {
+		codec = (1 << i) & codecs;
+		if (!codec)
+			continue;
+
+		ret = hfi_session_init(inst, to_v4l2_codec_type(codec));
+		if (ret)
+			goto done;
+
+		ret = hfi_session_deinit(inst);
+		if (ret)
+			goto done;
+	}
+
+done:
+	hfi_session_destroy(inst);
+err:
+	mutex_destroy(&inst->lock);
+	kfree(inst);
+
+	return ret;
+}
+
 static int venus_probe(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
@@ -219,6 +296,14 @@ static int venus_probe(struct platform_device *pdev)
 	if (ret)
 		goto err_venus_shutdown;
 
+	ret = venus_enumerate_codecs(core, VIDC_SESSION_TYPE_DEC);
+	if (ret)
+		goto err_venus_shutdown;
+
+	ret = venus_enumerate_codecs(core, VIDC_SESSION_TYPE_ENC);
+	if (ret)
+		goto err_venus_shutdown;
+
 	ret = v4l2_device_register(dev, &core->v4l2_dev);
 	if (ret)
 		goto err_core_deinit;
diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index b5b9a84e9155..fe2d2b9e8af8 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -57,6 +57,29 @@ struct venus_format {
 	u32 type;
 };
 
+#define MAX_PLANES		4
+#define MAX_FMT_ENTRIES		32
+#define MAX_CAP_ENTRIES		32
+#define MAX_CODEC_NUM		32
+
+struct raw_formats {
+	u32 buftype;
+	u32 fmt;
+};
+
+struct venus_caps {
+	u32 codec;
+	u32 domain;
+	bool cap_bufs_mode_dynamic;
+	unsigned int num_caps;
+	struct hfi_capability caps[MAX_CAP_ENTRIES];
+	unsigned int num_pl;
+	struct hfi_profile_level pl[HFI_MAX_PROFILE_COUNT];
+	unsigned int num_fmts;
+	struct raw_formats fmts[MAX_FMT_ENTRIES];
+	bool valid;
+};
+
 /**
  * struct venus_core - holds core parameters valid for all instances
  *
@@ -120,6 +143,8 @@ struct venus_core {
 	void *priv;
 	const struct hfi_ops *ops;
 	struct delayed_work work;
+	struct venus_caps caps[MAX_CODEC_NUM];
+	unsigned int codecs_count;
 };
 
 struct vdec_controls {
@@ -224,22 +249,8 @@ struct venus_buffer {
  * @priv:	a private for HFI operations callbacks
  * @session_type:	the type of the session (decoder or encoder)
  * @hprop:	a union used as a holder by get property
- * @cap_width:	width capability
- * @cap_height:	height capability
- * @cap_mbs_per_frame:	macroblocks per frame capability
- * @cap_mbs_per_sec:	macroblocks per second capability
- * @cap_framerate:	framerate capability
- * @cap_scale_x:		horizontal scaling capability
- * @cap_scale_y:		vertical scaling capability
- * @cap_bitrate:		bitrate capability
- * @cap_hier_p:		hier capability
- * @cap_ltr_count:	LTR count capability
- * @cap_secure_output2_threshold: secure OUTPUT2 threshold capability
  * @cap_bufs_mode_static:	buffers allocation mode capability
  * @cap_bufs_mode_dynamic:	buffers allocation mode capability
- * @pl_count:	count of supported profiles/levels
- * @pl:		supported profiles/levels
- * @bufreq:	holds buffer requirements
  */
 struct venus_inst {
 	struct list_head list;
@@ -276,6 +287,7 @@ struct venus_inst {
 	bool reconfig;
 	u32 reconfig_width;
 	u32 reconfig_height;
+	u32 hfi_codec;
 	u32 sequence_cap;
 	u32 sequence_out;
 	struct v4l2_m2m_dev *m2m_dev;
@@ -287,22 +299,8 @@ struct venus_inst {
 	const struct hfi_inst_ops *ops;
 	u32 session_type;
 	union hfi_get_property hprop;
-	struct hfi_capability cap_width;
-	struct hfi_capability cap_height;
-	struct hfi_capability cap_mbs_per_frame;
-	struct hfi_capability cap_mbs_per_sec;
-	struct hfi_capability cap_framerate;
-	struct hfi_capability cap_scale_x;
-	struct hfi_capability cap_scale_y;
-	struct hfi_capability cap_bitrate;
-	struct hfi_capability cap_hier_p;
-	struct hfi_capability cap_ltr_count;
-	struct hfi_capability cap_secure_output2_threshold;
 	bool cap_bufs_mode_static;
 	bool cap_bufs_mode_dynamic;
-	unsigned int pl_count;
-	struct hfi_profile_level pl[HFI_MAX_PROFILE_COUNT];
-	struct hfi_buffer_requirements bufreq[HFI_BUFFER_TYPE_MAX];
 };
 
 #define IS_V1(core)	((core)->res->hfi_version == HFI_VERSION_1XX)
@@ -322,4 +320,18 @@ static inline void *to_hfi_priv(struct venus_core *core)
 	return core->priv;
 }
 
+static inline struct venus_caps *
+venus_caps_by_codec(struct venus_core *core, u32 codec, u32 domain)
+{
+	unsigned int c;
+
+	for (c = 0; c < MAX_CODEC_NUM; c++) {
+		if (core->caps[c].codec == codec &&
+		    core->caps[c].domain == domain)
+			return &core->caps[c];
+	}
+
+	return NULL;
+}
+
 #endif
diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
index a570fdad0de0..94ca27b0bb99 100644
--- a/drivers/media/platform/qcom/venus/hfi.c
+++ b/drivers/media/platform/qcom/venus/hfi.c
@@ -203,13 +203,12 @@ int hfi_session_init(struct venus_inst *inst, u32 pixfmt)
 {
 	struct venus_core *core = inst->core;
 	const struct hfi_ops *ops = core->ops;
-	u32 codec;
 	int ret;
 
-	codec = to_codec_type(pixfmt);
+	inst->hfi_codec = to_codec_type(pixfmt);
 	reinit_completion(&inst->done);
 
-	ret = ops->session_init(inst, inst->session_type, codec);
+	ret = ops->session_init(inst, inst->session_type, inst->hfi_codec);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/venus/hfi_helper.h
index 1bc5aab1ce6b..64cc2bc946ee 100644
--- a/drivers/media/platform/qcom/venus/hfi_helper.h
+++ b/drivers/media/platform/qcom/venus/hfi_helper.h
@@ -858,10 +858,23 @@ struct hfi_uncompressed_format_select {
 	u32 format;
 };
 
+struct hfi_uncompressed_plane_constraints {
+	u32 stride_multiples;
+	u32 max_stride;
+	u32 min_plane_buffer_height_multiple;
+	u32 buffer_alignment;
+};
+
+struct hfi_uncompressed_plane_info {
+	u32 format;
+	u32 num_planes;
+	struct hfi_uncompressed_plane_constraints plane_format[1];
+};
+
 struct hfi_uncompressed_format_supported {
 	u32 buffer_type;
 	u32 format_entries;
-	u32 format_info[1];
+	struct hfi_uncompressed_plane_info format_info[1];
 };
 
 struct hfi_uncompressed_plane_actual {
@@ -875,19 +888,6 @@ struct hfi_uncompressed_plane_actual_info {
 	struct hfi_uncompressed_plane_actual plane_format[1];
 };
 
-struct hfi_uncompressed_plane_constraints {
-	u32 stride_multiples;
-	u32 max_stride;
-	u32 min_plane_buffer_height_multiple;
-	u32 buffer_alignment;
-};
-
-struct hfi_uncompressed_plane_info {
-	u32 format;
-	u32 num_planes;
-	struct hfi_uncompressed_plane_constraints plane_format[1];
-};
-
 struct hfi_uncompressed_plane_actual_constraints_info {
 	u32 buffer_type;
 	u32 num_planes;
diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/venus/hfi_msgs.c
index 023802e62833..8a943f53a12b 100644
--- a/drivers/media/platform/qcom/venus/hfi_msgs.c
+++ b/drivers/media/platform/qcom/venus/hfi_msgs.c
@@ -21,6 +21,7 @@
 #include "hfi.h"
 #include "hfi_helper.h"
 #include "hfi_msgs.h"
+#include "hfi_parser.h"
 
 static void event_seq_changed(struct venus_core *core, struct venus_inst *inst,
 			      struct hfi_msg_event_notify_pkt *pkt)
@@ -219,81 +220,30 @@ static void hfi_sys_init_done(struct venus_core *core, struct venus_inst *inst,
 			      void *packet)
 {
 	struct hfi_msg_sys_init_done_pkt *pkt = packet;
-	u32 rem_bytes, read_bytes = 0, num_properties;
-	u32 error, ptype;
-	u8 *data;
+	u32 rem_bytes, num_properties, error;
 
 	error = pkt->error_type;
 	if (error != HFI_ERR_NONE)
-		goto err_no_prop;
+		goto done;
 
 	num_properties = pkt->num_properties;
 
 	if (!num_properties) {
 		error = HFI_ERR_SYS_INVALID_PARAMETER;
-		goto err_no_prop;
+		goto done;
 	}
 
 	rem_bytes = pkt->hdr.size - sizeof(*pkt) + sizeof(u32);
-
 	if (!rem_bytes) {
 		/* missing property data */
 		error = HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
-		goto err_no_prop;
+		goto done;
 	}
 
-	data = (u8 *)&pkt->data[0];
-
-	if (core->res->hfi_version == HFI_VERSION_3XX)
-		goto err_no_prop;
-
-	while (num_properties && rem_bytes >= sizeof(u32)) {
-		ptype = *((u32 *)data);
-		data += sizeof(u32);
-
-		switch (ptype) {
-		case HFI_PROPERTY_PARAM_CODEC_SUPPORTED: {
-			struct hfi_codec_supported *prop;
-
-			prop = (struct hfi_codec_supported *)data;
-
-			if (rem_bytes < sizeof(*prop)) {
-				error = HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
-				break;
-			}
-
-			read_bytes += sizeof(*prop) + sizeof(u32);
-			core->dec_codecs = prop->dec_codecs;
-			core->enc_codecs = prop->enc_codecs;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_MAX_SESSIONS_SUPPORTED: {
-			struct hfi_max_sessions_supported *prop;
-
-			if (rem_bytes < sizeof(*prop)) {
-				error = HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
-				break;
-			}
-
-			prop = (struct hfi_max_sessions_supported *)data;
-			read_bytes += sizeof(*prop) + sizeof(u32);
-			core->max_sessions_supported = prop->max_sessions;
-			break;
-		}
-		default:
-			error = HFI_ERR_SYS_INVALID_PARAMETER;
-			break;
-		}
-
-		if (error)
-			break;
-
-		rem_bytes -= read_bytes;
-		data += read_bytes;
-		num_properties--;
-	}
+	error = hfi_parser(core, inst, num_properties, &pkt->data[0],
+			   rem_bytes);
 
-err_no_prop:
+done:
 	core->error = error;
 	complete(&core->done);
 }
@@ -371,51 +321,6 @@ static void hfi_sys_pc_prepare_done(struct venus_core *core,
 	dev_dbg(core->dev, "pc prepare done (error %x)\n", pkt->error_type);
 }
 
-static void
-hfi_copy_cap_prop(struct hfi_capability *in, struct venus_inst *inst)
-{
-	if (!in || !inst)
-		return;
-
-	switch (in->capability_type) {
-	case HFI_CAPABILITY_FRAME_WIDTH:
-		inst->cap_width = *in;
-		break;
-	case HFI_CAPABILITY_FRAME_HEIGHT:
-		inst->cap_height = *in;
-		break;
-	case HFI_CAPABILITY_MBS_PER_FRAME:
-		inst->cap_mbs_per_frame = *in;
-		break;
-	case HFI_CAPABILITY_MBS_PER_SECOND:
-		inst->cap_mbs_per_sec = *in;
-		break;
-	case HFI_CAPABILITY_FRAMERATE:
-		inst->cap_framerate = *in;
-		break;
-	case HFI_CAPABILITY_SCALE_X:
-		inst->cap_scale_x = *in;
-		break;
-	case HFI_CAPABILITY_SCALE_Y:
-		inst->cap_scale_y = *in;
-		break;
-	case HFI_CAPABILITY_BITRATE:
-		inst->cap_bitrate = *in;
-		break;
-	case HFI_CAPABILITY_HIER_P_NUM_ENH_LAYERS:
-		inst->cap_hier_p = *in;
-		break;
-	case HFI_CAPABILITY_ENC_LTR_COUNT:
-		inst->cap_ltr_count = *in;
-		break;
-	case HFI_CAPABILITY_CP_OUTPUT2_THRESH:
-		inst->cap_secure_output2_threshold = *in;
-		break;
-	default:
-		break;
-	}
-}
-
 static unsigned int
 session_get_prop_profile_level(struct hfi_msg_session_property_info_pkt *pkt,
 			       struct hfi_profile_level *profile_level)
@@ -505,238 +410,11 @@ static void hfi_session_prop_info(struct venus_core *core,
 	complete(&inst->done);
 }
 
-static u32 init_done_read_prop(struct venus_core *core, struct venus_inst *inst,
-			       struct hfi_msg_session_init_done_pkt *pkt)
-{
-	struct device *dev = core->dev;
-	u32 rem_bytes, num_props;
-	u32 ptype, next_offset = 0;
-	u32 err;
-	u8 *data;
-
-	rem_bytes = pkt->shdr.hdr.size - sizeof(*pkt) + sizeof(u32);
-	if (!rem_bytes) {
-		dev_err(dev, "%s: missing property info\n", __func__);
-		return HFI_ERR_SESSION_INSUFFICIENT_RESOURCES;
-	}
-
-	err = pkt->error_type;
-	if (err)
-		return err;
-
-	data = (u8 *)&pkt->data[0];
-	num_props = pkt->num_properties;
-
-	while (err == HFI_ERR_NONE && num_props && rem_bytes >= sizeof(u32)) {
-		ptype = *((u32 *)data);
-		next_offset = sizeof(u32);
-
-		switch (ptype) {
-		case HFI_PROPERTY_PARAM_CODEC_MASK_SUPPORTED: {
-			struct hfi_codec_mask_supported *masks =
-				(struct hfi_codec_mask_supported *)
-				(data + next_offset);
-
-			next_offset += sizeof(*masks);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_CAPABILITY_SUPPORTED: {
-			struct hfi_capabilities *caps;
-			struct hfi_capability *cap;
-			u32 num_caps;
-
-			if ((rem_bytes - next_offset) < sizeof(*cap)) {
-				err = HFI_ERR_SESSION_INVALID_PARAMETER;
-				break;
-			}
-
-			caps = (struct hfi_capabilities *)(data + next_offset);
-
-			num_caps = caps->num_capabilities;
-			cap = &caps->data[0];
-			next_offset += sizeof(u32);
-
-			while (num_caps &&
-			       (rem_bytes - next_offset) >= sizeof(u32)) {
-				hfi_copy_cap_prop(cap, inst);
-				cap++;
-				next_offset += sizeof(*cap);
-				num_caps--;
-			}
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SUPPORTED: {
-			struct hfi_uncompressed_format_supported *prop =
-				(struct hfi_uncompressed_format_supported *)
-				(data + next_offset);
-			u32 num_fmt_entries;
-			u8 *fmt;
-			struct hfi_uncompressed_plane_info *inf;
-
-			if ((rem_bytes - next_offset) < sizeof(*prop)) {
-				err = HFI_ERR_SESSION_INVALID_PARAMETER;
-				break;
-			}
-
-			num_fmt_entries = prop->format_entries;
-			next_offset = sizeof(*prop) - sizeof(u32);
-			fmt = (u8 *)&prop->format_info[0];
-
-			dev_dbg(dev, "uncomm format support num entries:%u\n",
-				num_fmt_entries);
-
-			while (num_fmt_entries) {
-				struct hfi_uncompressed_plane_constraints *cnts;
-				u32 bytes_to_skip;
-
-				inf = (struct hfi_uncompressed_plane_info *)fmt;
-
-				if ((rem_bytes - next_offset) < sizeof(*inf)) {
-					err = HFI_ERR_SESSION_INVALID_PARAMETER;
-					break;
-				}
-
-				dev_dbg(dev, "plane info: fmt:%x, planes:%x\n",
-					inf->format, inf->num_planes);
-
-				cnts = &inf->plane_format[0];
-				dev_dbg(dev, "%u %u %u %u\n",
-					cnts->stride_multiples,
-					cnts->max_stride,
-					cnts->min_plane_buffer_height_multiple,
-					cnts->buffer_alignment);
-
-				bytes_to_skip = sizeof(*inf) - sizeof(*cnts) +
-						inf->num_planes * sizeof(*cnts);
-
-				fmt += bytes_to_skip;
-				next_offset += bytes_to_skip;
-				num_fmt_entries--;
-			}
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_PROPERTIES_SUPPORTED: {
-			struct hfi_properties_supported *prop =
-				(struct hfi_properties_supported *)
-				(data + next_offset);
-
-			next_offset += sizeof(*prop) - sizeof(u32)
-					+ prop->num_properties * sizeof(u32);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_PROFILE_LEVEL_SUPPORTED: {
-			struct hfi_profile_level_supported *prop =
-				(struct hfi_profile_level_supported *)
-				(data + next_offset);
-			struct hfi_profile_level *pl;
-			unsigned int prop_count = 0;
-			unsigned int count = 0;
-			u8 *ptr;
-
-			ptr = (u8 *)&prop->profile_level[0];
-			prop_count = prop->profile_count;
-
-			if (prop_count > HFI_MAX_PROFILE_COUNT)
-				prop_count = HFI_MAX_PROFILE_COUNT;
-
-			while (prop_count) {
-				ptr++;
-				pl = (struct hfi_profile_level *)ptr;
-
-				inst->pl[count].profile = pl->profile;
-				inst->pl[count].level = pl->level;
-				prop_count--;
-				count++;
-				ptr += sizeof(*pl) / sizeof(u32);
-			}
-
-			inst->pl_count = count;
-			next_offset += sizeof(*prop) - sizeof(*pl) +
-				       prop->profile_count * sizeof(*pl);
-
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_INTERLACE_FORMAT_SUPPORTED: {
-			next_offset +=
-				sizeof(struct hfi_interlace_format_supported);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_NAL_STREAM_FORMAT_SUPPORTED: {
-			struct hfi_nal_stream_format *nal =
-				(struct hfi_nal_stream_format *)
-				(data + next_offset);
-			dev_dbg(dev, "NAL format: %x\n", nal->format);
-			next_offset += sizeof(*nal);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_NAL_STREAM_FORMAT_SELECT: {
-			next_offset += sizeof(u32);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_MAX_SEQUENCE_HEADER_SIZE: {
-			u32 *max_seq_sz = (u32 *)(data + next_offset);
-
-			dev_dbg(dev, "max seq header sz: %x\n", *max_seq_sz);
-			next_offset += sizeof(u32);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH: {
-			next_offset += sizeof(struct hfi_intra_refresh);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE_SUPPORTED: {
-			struct hfi_buffer_alloc_mode_supported *prop =
-				(struct hfi_buffer_alloc_mode_supported *)
-				(data + next_offset);
-			unsigned int i;
-
-			for (i = 0; i < prop->num_entries; i++) {
-				if (prop->buffer_type == HFI_BUFFER_OUTPUT ||
-				    prop->buffer_type == HFI_BUFFER_OUTPUT2) {
-					switch (prop->data[i]) {
-					case HFI_BUFFER_MODE_STATIC:
-						inst->cap_bufs_mode_static = true;
-						break;
-					case HFI_BUFFER_MODE_DYNAMIC:
-						inst->cap_bufs_mode_dynamic = true;
-						break;
-					default:
-						break;
-					}
-				}
-			}
-			next_offset += sizeof(*prop) -
-				sizeof(u32) + prop->num_entries * sizeof(u32);
-			num_props--;
-			break;
-		}
-		default:
-			dev_dbg(dev, "%s: default case %#x\n", __func__, ptype);
-			break;
-		}
-
-		rem_bytes -= next_offset;
-		data += next_offset;
-	}
-
-	return err;
-}
-
 static void hfi_session_init_done(struct venus_core *core,
 				  struct venus_inst *inst, void *packet)
 {
 	struct hfi_msg_session_init_done_pkt *pkt = packet;
-	unsigned int error;
+	u32 rem_bytes, error;
 
 	error = pkt->error_type;
 	if (error != HFI_ERR_NONE)
@@ -745,8 +423,14 @@ static void hfi_session_init_done(struct venus_core *core,
 	if (core->res->hfi_version != HFI_VERSION_1XX)
 		goto done;
 
-	error = init_done_read_prop(core, inst, pkt);
+	rem_bytes = pkt->shdr.hdr.size - sizeof(*pkt) + sizeof(u32);
+	if (!rem_bytes) {
+		error = HFI_ERR_SESSION_INSUFFICIENT_RESOURCES;
+		goto done;
+	}
 
+	error = hfi_parser(core, inst, pkt->num_properties, &pkt->data[0],
+			   rem_bytes);
 done:
 	inst->error = error;
 	complete(&inst->done);
diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c b/drivers/media/platform/qcom/venus/hfi_parser.c
new file mode 100644
index 000000000000..f9181d999b23
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/hfi_parser.c
@@ -0,0 +1,295 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 Linaro Ltd.
+ *
+ * Author: Stanimir Varbanov <stanimir.varbanov@linaro.org>
+ */
+#include <linux/kernel.h>
+
+#include "core.h"
+#include "hfi_helper.h"
+#include "hfi_parser.h"
+
+typedef void (*func)(struct venus_caps *cap, void *data, unsigned int size);
+
+static void init_codecs_vcaps(struct venus_core *core)
+{
+	struct venus_caps *caps = core->caps;
+	struct venus_caps *cap;
+	unsigned int i;
+
+	for (i = 0; i < 8 * sizeof(core->dec_codecs); i++) {
+		if ((1 << i) & core->dec_codecs) {
+			cap = &caps[core->codecs_count++];
+			cap->codec = (1 << i) & core->dec_codecs;
+			cap->domain = VIDC_SESSION_TYPE_DEC;
+			cap->valid = false;
+		}
+	}
+
+	for (i = 0; i < 8 * sizeof(core->enc_codecs); i++) {
+		if ((1 << i) & core->enc_codecs) {
+			cap = &caps[core->codecs_count++];
+			cap->codec = (1 << i) & core->enc_codecs;
+			cap->domain = VIDC_SESSION_TYPE_ENC;
+			cap->valid = false;
+		}
+	}
+}
+
+static void for_each_codec(struct venus_caps *caps, unsigned int caps_num,
+			   u32 codecs, u32 domain, func cb, void *data,
+			   unsigned int size)
+{
+	struct venus_caps *cap;
+	unsigned int i;
+
+	for (i = 0; i < caps_num; i++) {
+		cap = &caps[i];
+		if (cap->valid && cap->domain == domain)
+			continue;
+		if (cap->codec & codecs && cap->domain == domain)
+			cb(cap, data, size);
+	}
+}
+
+static void fill_buf_mode(struct venus_caps *cap, void *data, unsigned int num)
+{
+	u32 *type = data;
+
+	if (*type == HFI_BUFFER_MODE_DYNAMIC)
+		cap->cap_bufs_mode_dynamic = true;
+}
+
+static void parse_alloc_mode(struct venus_core *core, struct venus_inst *inst,
+			     u32 codecs, u32 domain, void *data)
+{
+	struct hfi_buffer_alloc_mode_supported *mode = data;
+	u32 num_entries = mode->num_entries;
+	u32 *type;
+
+	if (num_entries > 16)
+		return;
+
+	type = mode->data;
+
+	while (num_entries--) {
+		if (mode->buffer_type == HFI_BUFFER_OUTPUT ||
+		    mode->buffer_type == HFI_BUFFER_OUTPUT2) {
+			if (*type == HFI_BUFFER_MODE_DYNAMIC && inst)
+				inst->cap_bufs_mode_dynamic = true;
+
+			for_each_codec(core->caps, ARRAY_SIZE(core->caps),
+				       codecs, domain, fill_buf_mode, type, 1);
+		}
+
+		type++;
+	}
+}
+
+static void parse_profile_level(u32 codecs, u32 domain, void *data)
+{
+	struct hfi_profile_level_supported *pl = data;
+	struct hfi_profile_level *proflevel = pl->profile_level;
+	u32 count = pl->profile_count;
+
+	if (count > HFI_MAX_PROFILE_COUNT)
+		return;
+
+	while (count) {
+		proflevel = (void *)proflevel + sizeof(*proflevel);
+		count--;
+	}
+}
+
+static void fill_caps(struct venus_caps *cap, void *data, unsigned int num)
+{
+	struct hfi_capability *caps = data;
+	unsigned int i;
+
+	for (i = 0; i < num; i++)
+		cap->caps[cap->num_caps++] = caps[i];
+}
+
+static void parse_caps(struct venus_core *core, struct venus_inst *inst,
+		       u32 codecs, u32 domain, void *data)
+{
+	struct hfi_capabilities *caps = data;
+	struct hfi_capability *cap = caps->data;
+	u32 num_caps = caps->num_capabilities;
+	struct hfi_capability caps_arr[MAX_CAP_ENTRIES] = {};
+	unsigned int i = 0;
+
+	if (num_caps > MAX_CAP_ENTRIES)
+		return;
+
+	while (num_caps) {
+		caps_arr[i++] = *cap;
+		cap = (void *)cap + sizeof(*cap);
+		num_caps--;
+	}
+
+	for_each_codec(core->caps, ARRAY_SIZE(core->caps), codecs, domain,
+		       fill_caps, caps_arr, i);
+}
+
+static void fill_raw_fmts(struct venus_caps *cap, void *fmts,
+			  unsigned int num_fmts)
+{
+	struct raw_formats *formats = fmts;
+	unsigned int i;
+
+	for (i = 0; i < num_fmts; i++)
+		cap->fmts[cap->num_fmts++] = formats[i];
+}
+
+static void parse_raw_formats(struct venus_core *core, struct venus_inst *inst,
+			      u32 codecs, u32 domain, void *data)
+{
+	struct hfi_uncompressed_format_supported *fmt = data;
+	struct hfi_uncompressed_plane_info *pinfo = fmt->format_info;
+	struct hfi_uncompressed_plane_constraints *constr;
+	u32 entries = fmt->format_entries;
+	u32 num_planes;
+	struct raw_formats rfmts[MAX_FMT_ENTRIES] = {};
+	unsigned int i = 0;
+
+	while (entries) {
+		num_planes = pinfo->num_planes;
+
+		rfmts[i].fmt = pinfo->format;
+		rfmts[i].buftype = fmt->buffer_type;
+		i++;
+
+		if (pinfo->num_planes > MAX_PLANES)
+			break;
+
+		constr = pinfo->plane_format;
+
+		while (pinfo->num_planes) {
+			constr = (void *)constr + sizeof(*constr);
+			pinfo->num_planes--;
+		}
+
+		pinfo = (void *)pinfo + sizeof(*constr) * num_planes +
+			2 * sizeof(u32);
+		entries--;
+	}
+
+	for_each_codec(core->caps, ARRAY_SIZE(core->caps), codecs, domain,
+		       fill_raw_fmts, rfmts, i);
+}
+
+static void parse_codecs(struct venus_core *core, void *data)
+{
+	struct hfi_codec_supported *codecs = data;
+
+	core->dec_codecs = codecs->dec_codecs;
+	core->enc_codecs = codecs->enc_codecs;
+
+	if (core->res->hfi_version == HFI_VERSION_1XX) {
+		core->dec_codecs &= ~HFI_VIDEO_CODEC_HEVC;
+		core->dec_codecs &= ~HFI_VIDEO_CODEC_SPARK;
+	}
+}
+
+static void parse_max_sessions(struct venus_core *core, void *data)
+{
+	struct hfi_max_sessions_supported *sessions = data;
+
+	core->max_sessions_supported = sessions->max_sessions;
+}
+
+static void parse_codecs_mask(u32 *codecs, u32 *domain, void *data)
+{
+	struct hfi_codec_mask_supported *mask = data;
+
+	*codecs = mask->codecs;
+	*domain = mask->video_domains;
+}
+
+static void parser_init(struct venus_core *core, struct venus_inst *inst,
+			u32 *codecs, u32 *domain)
+{
+	if (core->res->hfi_version != HFI_VERSION_1XX)
+		return;
+
+	if (!inst)
+		return;
+
+	*codecs = inst->hfi_codec;
+	*domain = inst->session_type;
+}
+
+static void parser_fini(struct venus_core *core, struct venus_inst *inst,
+			u32 codecs, u32 domain)
+{
+	struct venus_caps *caps = core->caps;
+	struct venus_caps *cap;
+	u32 dom;
+	unsigned int i;
+
+	if (core->res->hfi_version != HFI_VERSION_1XX)
+		return;
+
+	if (!inst)
+		return;
+
+	dom = inst->session_type;
+
+	for (i = 0; i < MAX_CODEC_NUM; i++) {
+		cap = &caps[i];
+		if (cap->codec & codecs && cap->domain == dom)
+			cap->valid = true;
+	}
+}
+
+u32 hfi_parser(struct venus_core *core, struct venus_inst *inst,
+	       u32 num_properties, void *buf, u32 size)
+{
+	unsigned int words_count = size >> 2;
+	u32 *word = buf, *data, codecs = 0, domain = 0;
+
+	if (size % 4)
+		return HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
+
+	parser_init(core, inst, &codecs, &domain);
+
+	while (words_count) {
+		data = word + 1;
+
+		switch (*word) {
+		case HFI_PROPERTY_PARAM_CODEC_SUPPORTED:
+			parse_codecs(core, data);
+			init_codecs_vcaps(core);
+			break;
+		case HFI_PROPERTY_PARAM_MAX_SESSIONS_SUPPORTED:
+			parse_max_sessions(core, data);
+			break;
+		case HFI_PROPERTY_PARAM_CODEC_MASK_SUPPORTED:
+			parse_codecs_mask(&codecs, &domain, data);
+			break;
+		case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SUPPORTED:
+			parse_raw_formats(core, inst, codecs, domain, data);
+			break;
+		case HFI_PROPERTY_PARAM_CAPABILITY_SUPPORTED:
+			parse_caps(core, inst, codecs, domain, data);
+			break;
+		case HFI_PROPERTY_PARAM_PROFILE_LEVEL_SUPPORTED:
+			parse_profile_level(codecs, domain, data);
+			break;
+		case HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE_SUPPORTED:
+			parse_alloc_mode(core, inst, codecs, domain, data);
+			break;
+		default:
+			break;
+		}
+
+		word++;
+		words_count--;
+	}
+
+	parser_fini(core, inst, codecs, domain);
+
+	return HFI_ERR_NONE;
+}
diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h b/drivers/media/platform/qcom/venus/hfi_parser.h
new file mode 100644
index 000000000000..c484ac91a8e2
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/hfi_parser.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2018 Linaro Ltd. */
+#ifndef __VENUS_HFI_PARSER_H__
+#define __VENUS_HFI_PARSER_H__
+
+#include "core.h"
+
+u32 hfi_parser(struct venus_core *core, struct venus_inst *inst,
+	       u32 num_properties, void *buf, u32 size);
+
+static inline struct hfi_capability *get_cap(struct venus_inst *inst, u32 type)
+{
+	struct venus_core *core = inst->core;
+	struct venus_caps *caps;
+	unsigned int i;
+
+	caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
+	if (!caps)
+		return ERR_PTR(-EINVAL);
+
+	for (i = 0; i < MAX_CAP_ENTRIES; i++) {
+		if (caps->caps[i].capability_type == type)
+			return &caps->caps[i];
+	}
+
+	return ERR_PTR(-EINVAL);
+}
+
+#define CAP_MIN(inst, type)	((get_cap(inst, type))->min)
+#define CAP_MAX(inst, type)	((get_cap(inst, type))->max)
+#define CAP_STEP(inst, type)	((get_cap(inst, type))->step_size)
+
+#define FRAME_WIDTH_MIN(inst)	CAP_MIN(inst, HFI_CAPABILITY_FRAME_WIDTH)
+#define FRAME_WIDTH_MAX(inst)	CAP_MAX(inst, HFI_CAPABILITY_FRAME_WIDTH)
+#define FRAME_WIDTH_STEP(inst)	CAP_STEP(inst, HFI_CAPABILITY_FRAME_WIDTH)
+
+#define FRAME_HEIGHT_MIN(inst)	CAP_MIN(inst, HFI_CAPABILITY_FRAME_HEIGHT)
+#define FRAME_HEIGHT_MAX(inst)	CAP_MAX(inst, HFI_CAPABILITY_FRAME_HEIGHT)
+#define FRAME_HEIGHT_STEP(inst)	CAP_STEP(inst, HFI_CAPABILITY_FRAME_HEIGHT)
+
+#define FRATE_MIN(inst)		CAP_MIN(inst, HFI_CAPABILITY_FRAMERATE)
+#define FRATE_MAX(inst)		CAP_MAX(inst, HFI_CAPABILITY_FRAMERATE)
+#define FRATE_STEP(inst)	CAP_STEP(inst, HFI_CAPABILITY_FRAMERATE)
+
+#endif
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index c45452634e7e..3b38bd1241b0 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -24,6 +24,7 @@
 #include <media/videobuf2-dma-sg.h>
 
 #include "hfi_venus_io.h"
+#include "hfi_parser.h"
 #include "core.h"
 #include "helpers.h"
 #include "vdec.h"
@@ -177,10 +178,10 @@ vdec_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 		pixmp->height = 720;
 	}
 
-	pixmp->width = clamp(pixmp->width, inst->cap_width.min,
-			     inst->cap_width.max);
-	pixmp->height = clamp(pixmp->height, inst->cap_height.min,
-			      inst->cap_height.max);
+	pixmp->width = clamp(pixmp->width, FRAME_WIDTH_MIN(inst),
+			     FRAME_WIDTH_MAX(inst));
+	pixmp->height = clamp(pixmp->height, FRAME_HEIGHT_MIN(inst),
+			      FRAME_HEIGHT_MAX(inst));
 
 	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
 		pixmp->height = ALIGN(pixmp->height, 32);
@@ -442,12 +443,12 @@ static int vdec_enum_framesizes(struct file *file, void *fh,
 
 	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
 
-	fsize->stepwise.min_width = inst->cap_width.min;
-	fsize->stepwise.max_width = inst->cap_width.max;
-	fsize->stepwise.step_width = inst->cap_width.step_size;
-	fsize->stepwise.min_height = inst->cap_height.min;
-	fsize->stepwise.max_height = inst->cap_height.max;
-	fsize->stepwise.step_height = inst->cap_height.step_size;
+	fsize->stepwise.min_width = FRAME_WIDTH_MIN(inst);
+	fsize->stepwise.max_width = FRAME_WIDTH_MAX(inst);
+	fsize->stepwise.step_width = FRAME_WIDTH_STEP(inst);
+	fsize->stepwise.min_height = FRAME_HEIGHT_MIN(inst);
+	fsize->stepwise.max_height = FRAME_HEIGHT_MAX(inst);
+	fsize->stepwise.step_height = FRAME_HEIGHT_STEP(inst);
 
 	return 0;
 }
@@ -902,22 +903,7 @@ static void vdec_inst_init(struct venus_inst *inst)
 	inst->fps = 30;
 	inst->timeperframe.numerator = 1;
 	inst->timeperframe.denominator = 30;
-
-	inst->cap_width.min = 64;
-	inst->cap_width.max = 1920;
-	if (inst->core->res->hfi_version == HFI_VERSION_3XX)
-		inst->cap_width.max = 3840;
-	inst->cap_width.step_size = 1;
-	inst->cap_height.min = 64;
-	inst->cap_height.max = ALIGN(1080, 32);
-	if (inst->core->res->hfi_version == HFI_VERSION_3XX)
-		inst->cap_height.max = ALIGN(2160, 32);
-	inst->cap_height.step_size = 1;
-	inst->cap_framerate.min = 1;
-	inst->cap_framerate.max = 30;
-	inst->cap_framerate.step_size = 1;
-	inst->cap_mbs_per_frame.min = 16;
-	inst->cap_mbs_per_frame.max = 8160;
+	inst->hfi_codec = HFI_VIDEO_CODEC_H264;
 }
 
 static const struct v4l2_m2m_ops vdec_m2m_ops = {
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index bc8c2e7a8d2c..be8ea3326386 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -24,6 +24,7 @@
 #include <media/v4l2-ctrls.h>
 
 #include "hfi_venus_io.h"
+#include "hfi_parser.h"
 #include "core.h"
 #include "helpers.h"
 #include "venc.h"
@@ -301,10 +302,10 @@ venc_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 		pixmp->height = 720;
 	}
 
-	pixmp->width = clamp(pixmp->width, inst->cap_width.min,
-			     inst->cap_width.max);
-	pixmp->height = clamp(pixmp->height, inst->cap_height.min,
-			      inst->cap_height.max);
+	pixmp->width = clamp(pixmp->width, FRAME_WIDTH_MIN(inst),
+			     FRAME_WIDTH_MAX(inst));
+	pixmp->height = clamp(pixmp->height, FRAME_HEIGHT_MIN(inst),
+			      FRAME_HEIGHT_MAX(inst));
 
 	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
 		pixmp->height = ALIGN(pixmp->height, 32);
@@ -553,12 +554,12 @@ static int venc_enum_framesizes(struct file *file, void *fh,
 	if (fsize->index)
 		return -EINVAL;
 
-	fsize->stepwise.min_width = inst->cap_width.min;
-	fsize->stepwise.max_width = inst->cap_width.max;
-	fsize->stepwise.step_width = inst->cap_width.step_size;
-	fsize->stepwise.min_height = inst->cap_height.min;
-	fsize->stepwise.max_height = inst->cap_height.max;
-	fsize->stepwise.step_height = inst->cap_height.step_size;
+	fsize->stepwise.min_width = FRAME_WIDTH_MIN(inst);
+	fsize->stepwise.max_width = FRAME_WIDTH_MAX(inst);
+	fsize->stepwise.step_width = FRAME_WIDTH_STEP(inst);
+	fsize->stepwise.min_height = FRAME_HEIGHT_MIN(inst);
+	fsize->stepwise.max_height = FRAME_HEIGHT_MAX(inst);
+	fsize->stepwise.step_height = FRAME_HEIGHT_STEP(inst);
 
 	return 0;
 }
@@ -586,18 +587,18 @@ static int venc_enum_frameintervals(struct file *file, void *fh,
 	if (!fival->width || !fival->height)
 		return -EINVAL;
 
-	if (fival->width > inst->cap_width.max ||
-	    fival->width < inst->cap_width.min ||
-	    fival->height > inst->cap_height.max ||
-	    fival->height < inst->cap_height.min)
+	if (fival->width > FRAME_WIDTH_MAX(inst) ||
+	    fival->width < FRAME_WIDTH_MIN(inst) ||
+	    fival->height > FRAME_HEIGHT_MAX(inst) ||
+	    fival->height < FRAME_HEIGHT_MIN(inst))
 		return -EINVAL;
 
 	fival->stepwise.min.numerator = 1;
-	fival->stepwise.min.denominator = inst->cap_framerate.max;
+	fival->stepwise.min.denominator = FRATE_MAX(inst);
 	fival->stepwise.max.numerator = 1;
-	fival->stepwise.max.denominator = inst->cap_framerate.min;
+	fival->stepwise.max.denominator = FRATE_MIN(inst);
 	fival->stepwise.step.numerator = 1;
-	fival->stepwise.step.denominator = inst->cap_framerate.max;
+	fival->stepwise.step.denominator = FRATE_MAX(inst);
 
 	return 0;
 }
@@ -1091,22 +1092,7 @@ static void venc_inst_init(struct venus_inst *inst)
 	inst->fps = 15;
 	inst->timeperframe.numerator = 1;
 	inst->timeperframe.denominator = 15;
-
-	inst->cap_width.min = 96;
-	inst->cap_width.max = 1920;
-	if (inst->core->res->hfi_version == HFI_VERSION_3XX)
-		inst->cap_width.max = 3840;
-	inst->cap_width.step_size = 2;
-	inst->cap_height.min = 64;
-	inst->cap_height.max = ALIGN(1080, 32);
-	if (inst->core->res->hfi_version == HFI_VERSION_3XX)
-		inst->cap_height.max = ALIGN(2160, 32);
-	inst->cap_height.step_size = 2;
-	inst->cap_framerate.min = 1;
-	inst->cap_framerate.max = 30;
-	inst->cap_framerate.step_size = 1;
-	inst->cap_mbs_per_frame.min = 24;
-	inst->cap_mbs_per_frame.max = 8160;
+	inst->hfi_codec = HFI_VIDEO_CODEC_H264;
 }
 
 static int venc_open(struct file *file)
-- 
2.14.1

Re: [PATCH v2 12/29] venus: add common capability parser

[Tomasz Figa]

Hi Stanimir,

On Tue, May 15, 2018 at 5:08 PM Stanimir Varbanov <
stanimir.varbanov@linaro.org> wrote:
[snip]
> diff --git a/drivers/media/platform/qcom/venus/core.c
b/drivers/media/platform/qcom/venus/core.c
> index 41eef376eb2d..381bfdd688db 100644
> --- a/drivers/media/platform/qcom/venus/core.c
> +++ b/drivers/media/platform/qcom/venus/core.c
[snip]
> +static int venus_enumerate_codecs(struct venus_core *core, u32 type)
> +{
[snip]
> +       for (i = 0; i < MAX_CODEC_NUM; i++) {
> +               codec = (1 << i) & codecs;
> +               if (!codec)
> +                       continue;

Could be simplified to

          for_each_set_bit(i, &codecs, MAX_CODEC_NUM) {

after making codecs an unsigned long.

[snip]
> diff --git a/drivers/media/platform/qcom/venus/core.h
b/drivers/media/platform/qcom/venus/core.h
> index b5b9a84e9155..fe2d2b9e8af8 100644
> --- a/drivers/media/platform/qcom/venus/core.h
> +++ b/drivers/media/platform/qcom/venus/core.h
> @@ -57,6 +57,29 @@ struct venus_format {
>           u32 type;
>    };

> +#define MAX_PLANES             4

We have VIDEO_MAX_PLANES (== 8) already.

[snip]
> @@ -224,22 +249,8 @@ struct venus_buffer {
>     * @priv:      a private for HFI operations callbacks
>     * @session_type:      the type of the session (decoder or encoder)
>     * @hprop:     a union used as a holder by get property
> - * @cap_width: width capability
> - * @cap_height:        height capability
> - * @cap_mbs_per_frame: macroblocks per frame capability
> - * @cap_mbs_per_sec:   macroblocks per second capability
> - * @cap_framerate:     framerate capability
> - * @cap_scale_x:               horizontal scaling capability
> - * @cap_scale_y:               vertical scaling capability
> - * @cap_bitrate:               bitrate capability
> - * @cap_hier_p:                hier capability
> - * @cap_ltr_count:     LTR count capability
> - * @cap_secure_output2_threshold: secure OUTPUT2 threshold capability
>     * @cap_bufs_mode_static:      buffers allocation mode capability
>     * @cap_bufs_mode_dynamic:     buffers allocation mode capability
> - * @pl_count:  count of supported profiles/levels
> - * @pl:                supported profiles/levels
> - * @bufreq:    holds buffer requirements
>     */
>    struct venus_inst {
>           struct list_head list;
> @@ -276,6 +287,7 @@ struct venus_inst {
>           bool reconfig;
>           u32 reconfig_width;
>           u32 reconfig_height;
> +       u32 hfi_codec;

Respective line not added to the kerneldoc above.

>           u32 sequence_cap;
>           u32 sequence_out;
>           struct v4l2_m2m_dev *m2m_dev;
> @@ -287,22 +299,8 @@ struct venus_inst {
>           const struct hfi_inst_ops *ops;
>           u32 session_type;
>           union hfi_get_property hprop;
> -       struct hfi_capability cap_width;
> -       struct hfi_capability cap_height;
> -       struct hfi_capability cap_mbs_per_frame;
> -       struct hfi_capability cap_mbs_per_sec;
> -       struct hfi_capability cap_framerate;
> -       struct hfi_capability cap_scale_x;
> -       struct hfi_capability cap_scale_y;
> -       struct hfi_capability cap_bitrate;
> -       struct hfi_capability cap_hier_p;
> -       struct hfi_capability cap_ltr_count;
> -       struct hfi_capability cap_secure_output2_threshold;
>           bool cap_bufs_mode_static;
>           bool cap_bufs_mode_dynamic;
> -       unsigned int pl_count;
> -       struct hfi_profile_level pl[HFI_MAX_PROFILE_COUNT];
> -       struct hfi_buffer_requirements bufreq[HFI_BUFFER_TYPE_MAX];
>    };

>    #define IS_V1(core)    ((core)->res->hfi_version == HFI_VERSION_1XX)
> @@ -322,4 +320,18 @@ static inline void *to_hfi_priv(struct venus_core
*core)
>           return core->priv;
>    }

> +static inline struct venus_caps *

I'd leave the decision whether to inline this or not to the compiler.
(Although these days the "inline" keyword is just a hint anyway... but
still just wasted bytes in kernel's git repo.)

> +venus_caps_by_codec(struct venus_core *core, u32 codec, u32 domain)
> +{
> +       unsigned int c;
> +
> +       for (c = 0; c < MAX_CODEC_NUM; c++) {

Shouldn't we iterate up to core->codecs_count?

> +               if (core->caps[c].codec == codec &&
> +                   core->caps[c].domain == domain)
> +                       return &core->caps[c];
> +       }
> +
> +       return NULL;
> +}
> +
[snip]
> +       error = hfi_parser(core, inst, num_properties, &pkt->data[0],

nit: pkt->data?

[snip]
>   static void hfi_session_init_done(struct venus_core *core,
>                                    struct venus_inst *inst, void *packet)
>   {
>          struct hfi_msg_session_init_done_pkt *pkt = packet;
> -       unsigned int error;
> +       u32 rem_bytes, error;

>          error = pkt->error_type;
>          if (error != HFI_ERR_NONE)
> @@ -745,8 +423,14 @@ static void hfi_session_init_done(struct venus_core
*core,
>          if (core->res->hfi_version != HFI_VERSION_1XX)
>                  goto done;

> -       error = init_done_read_prop(core, inst, pkt);
> +       rem_bytes = pkt->shdr.hdr.size - sizeof(*pkt) + sizeof(u32);
> +       if (!rem_bytes) {

I’m not sure how likely it is to happen, but given that pkt->shdr.hdr.size
seems to come from hardware, perhaps we should make rem_bytes signed and
check for <= 0?

> +               error = HFI_ERR_SESSION_INSUFFICIENT_RESOURCES;
> +               goto done;
> +       }

> +       error = hfi_parser(core, inst, pkt->num_properties, &pkt->data[0],
> +                          rem_bytes);

nit: pkt->data?

>   done:
>          inst->error = error;
>          complete(&inst->done);
> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c
b/drivers/media/platform/qcom/venus/hfi_parser.c
> new file mode 100644
> index 000000000000..f9181d999b23
> --- /dev/null
> +++ b/drivers/media/platform/qcom/venus/hfi_parser.c
> @@ -0,0 +1,295 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2018 Linaro Ltd.
> + *
> + * Author: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> + */
> +#include <linux/kernel.h>
> +
> +#include "core.h"
> +#include "hfi_helper.h"
> +#include "hfi_parser.h"
> +
> +typedef void (*func)(struct venus_caps *cap, void *data, unsigned int
size);

Should we perhaps make data const? (My understanding is that it comes from
firmware packet.)

> +
> +static void init_codecs_vcaps(struct venus_core *core)
> +{
> +       struct venus_caps *caps = core->caps;
> +       struct venus_caps *cap;
> +       unsigned int i;
> +
> +       for (i = 0; i < 8 * sizeof(core->dec_codecs); i++) {
> +               if ((1 << i) & core->dec_codecs) {

for_each_set_bit()?

> +                       cap = &caps[core->codecs_count++];
> +                       cap->codec = (1 << i) & core->dec_codecs;

Any need to AND with core->dec_codecs? This code wouldn’t be executed if
the bit wasn’t set in the first place.

Also BIT(i).

> +                       cap->domain = VIDC_SESSION_TYPE_DEC;
> +                       cap->valid = false;
> +               }
> +       }
> +
> +       for (i = 0; i < 8 * sizeof(core->enc_codecs); i++) {
> +               if ((1 << i) & core->enc_codecs) {

Ditto.

> +                       cap = &caps[core->codecs_count++];
> +                       cap->codec = (1 << i) & core->enc_codecs;

Ditto.

> +                       cap->domain = VIDC_SESSION_TYPE_ENC;
> +                       cap->valid = false;
> +               }
> +       }
> +}
> +
> +static void for_each_codec(struct venus_caps *caps, unsigned int
caps_num,
> +                          u32 codecs, u32 domain, func cb, void *data,
> +                          unsigned int size)
> +{
> +       struct venus_caps *cap;
> +       unsigned int i;
> +
> +       for (i = 0; i < caps_num; i++) {
> +               cap = &caps[i];
> +               if (cap->valid && cap->domain == domain)

Is there any need to check cap->domain == domain? We could just skip if
cap->valid.

If we want to shorten the code, we could even do (cap->valid || cap->domain
!= domain) and remove domain check from the if below.

> +                       continue;
> +               if (cap->codec & codecs && cap->domain == domain)
> +                       cb(cap, data, size);
> +       }
> +}
> +
> +static void fill_buf_mode(struct venus_caps *cap, void *data, unsigned
int num)
> +{
> +       u32 *type = data;
> +
> +       if (*type == HFI_BUFFER_MODE_DYNAMIC)
> +               cap->cap_bufs_mode_dynamic = true;
> +}
> +
> +static void parse_alloc_mode(struct venus_core *core, struct venus_inst
*inst,
> +                            u32 codecs, u32 domain, void *data)
> +{
> +       struct hfi_buffer_alloc_mode_supported *mode = data;
> +       u32 num_entries = mode->num_entries;
> +       u32 *type;
> +
> +       if (num_entries > 16)

What is 16? We should have a macro for it.

> +               return;
> +
> +       type = mode->data;
> +
> +       while (num_entries--) {
> +               if (mode->buffer_type == HFI_BUFFER_OUTPUT ||
> +                   mode->buffer_type == HFI_BUFFER_OUTPUT2) {
> +                       if (*type == HFI_BUFFER_MODE_DYNAMIC && inst)
> +                               inst->cap_bufs_mode_dynamic = true;
> +
> +                       for_each_codec(core->caps, ARRAY_SIZE(core->caps),
> +                                      codecs, domain, fill_buf_mode,
type, 1);
> +               }
> +
> +               type++;
> +       }
> +}
> +
> +static void parse_profile_level(u32 codecs, u32 domain, void *data)
> +{
> +       struct hfi_profile_level_supported *pl = data;
> +       struct hfi_profile_level *proflevel = pl->profile_level;
> +       u32 count = pl->profile_count;
> +
> +       if (count > HFI_MAX_PROFILE_COUNT)
> +               return;
> +
> +       while (count) {
> +               proflevel = (void *)proflevel + sizeof(*proflevel);

Isn’t this just ++proflevel?

> +               count--;
> +       }

Am I missing something or this function doesn’t to do anything?

> +}
> +
> +static void fill_caps(struct venus_caps *cap, void *data, unsigned int
num)
> +{
> +       struct hfi_capability *caps = data;
> +       unsigned int i;
> +

Should we have some check to avoid overflowing cap->caps[]?

> +       for (i = 0; i < num; i++)
> +               cap->caps[cap->num_caps++] = caps[i];
> +}
> +
> +static void parse_caps(struct venus_core *core, struct venus_inst *inst,
> +                      u32 codecs, u32 domain, void *data)
> +{
> +       struct hfi_capabilities *caps = data;
> +       struct hfi_capability *cap = caps->data;
> +       u32 num_caps = caps->num_capabilities;
> +       struct hfi_capability caps_arr[MAX_CAP_ENTRIES] = {};
> +       unsigned int i = 0;
> +
> +       if (num_caps > MAX_CAP_ENTRIES)
> +               return;
> +
> +       while (num_caps) {
> +               caps_arr[i++] = *cap;
> +               cap = (void *)cap + sizeof(*cap);

++cap?

> +               num_caps--;
> +       }

Hmm, isn’t the whole loop just

memcpy(caps_arr, cap, num_caps * sizeof(*cap));

?

> +
> +       for_each_codec(core->caps, ARRAY_SIZE(core->caps), codecs, domain,
> +                      fill_caps, caps_arr, i);
> +}
> +
> +static void fill_raw_fmts(struct venus_caps *cap, void *fmts,
> +                         unsigned int num_fmts)
> +{
> +       struct raw_formats *formats = fmts;
> +       unsigned int i;
> +
> +       for (i = 0; i < num_fmts; i++)
> +               cap->fmts[cap->num_fmts++] = formats[i];
> +}
> +
> +static void parse_raw_formats(struct venus_core *core, struct venus_inst
*inst,
> +                             u32 codecs, u32 domain, void *data)
> +{
> +       struct hfi_uncompressed_format_supported *fmt = data;
> +       struct hfi_uncompressed_plane_info *pinfo = fmt->format_info;
> +       struct hfi_uncompressed_plane_constraints *constr;
> +       u32 entries = fmt->format_entries;
> +       u32 num_planes;
> +       struct raw_formats rfmts[MAX_FMT_ENTRIES] = {};
> +       unsigned int i = 0;
> +
> +       while (entries) {
> +               num_planes = pinfo->num_planes;
> +
> +               rfmts[i].fmt = pinfo->format;
> +               rfmts[i].buftype = fmt->buffer_type;
> +               i++;
> +
> +               if (pinfo->num_planes > MAX_PLANES)
> +                       break;
> +
> +               constr = pinfo->plane_format;
> +
> +               while (pinfo->num_planes) {
> +                       constr = (void *)constr + sizeof(*constr);

++constr?

> +                       pinfo->num_planes--;
> +               }

What is this loop supposed to do?

> +
> +               pinfo = (void *)pinfo + sizeof(*constr) * num_planes +
> +                       2 * sizeof(u32);
> +               entries--;
> +       }
> +
> +       for_each_codec(core->caps, ARRAY_SIZE(core->caps), codecs, domain,
> +                      fill_raw_fmts, rfmts, i);
> +}
[snip]
> +static void parser_fini(struct venus_core *core, struct venus_inst *inst,
> +                       u32 codecs, u32 domain)
> +{
> +       struct venus_caps *caps = core->caps;
> +       struct venus_caps *cap;
> +       u32 dom;
> +       unsigned int i;
> +
> +       if (core->res->hfi_version != HFI_VERSION_1XX)
> +               return;

Hmm, if the code below is executed only for 1XX, who will set cap->valid to
true for newer versions?

> +
> +       if (!inst)
> +               return;
> +
> +       dom = inst->session_type;
> +
> +       for (i = 0; i < MAX_CODEC_NUM; i++) {
> +               cap = &caps[i];
> +               if (cap->codec & codecs && cap->domain == dom)
> +                       cap->valid = true;
> +       }
> +}
> +
> +u32 hfi_parser(struct venus_core *core, struct venus_inst *inst,
> +              u32 num_properties, void *buf, u32 size)
> +{
> +       unsigned int words_count = size >> 2;
> +       u32 *word = buf, *data, codecs = 0, domain = 0;
> +
> +       if (size % 4)
> +               return HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
> +
> +       parser_init(core, inst, &codecs, &domain);
> +
> +       while (words_count) {
> +               data = word + 1;
> +
> +               switch (*word) {
> +               case HFI_PROPERTY_PARAM_CODEC_SUPPORTED:
> +                       parse_codecs(core, data);
> +                       init_codecs_vcaps(core);
> +                       break;
> +               case HFI_PROPERTY_PARAM_MAX_SESSIONS_SUPPORTED:
> +                       parse_max_sessions(core, data);
> +                       break;
> +               case HFI_PROPERTY_PARAM_CODEC_MASK_SUPPORTED:
> +                       parse_codecs_mask(&codecs, &domain, data);
> +                       break;
> +               case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SUPPORTED:
> +                       parse_raw_formats(core, inst, codecs, domain,
data);
> +                       break;
> +               case HFI_PROPERTY_PARAM_CAPABILITY_SUPPORTED:
> +                       parse_caps(core, inst, codecs, domain, data);
> +                       break;
> +               case HFI_PROPERTY_PARAM_PROFILE_LEVEL_SUPPORTED:
> +                       parse_profile_level(codecs, domain, data);
> +                       break;
> +               case HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE_SUPPORTED:
> +                       parse_alloc_mode(core, inst, codecs, domain,
data);
> +                       break;
> +               default:

Should we perhaps print something to let us know that something
unrecognized was reported? (Or it is expected that something unrecognized
is there?)

> +                       break;
> +               }
> +
> +               word++;
> +               words_count--;

If data is at |word + 1|, shouldn’t we increment |word| by |1 + |data
size||?

> +       }
> +
> +       parser_fini(core, inst, codecs, domain);
> +
> +       return HFI_ERR_NONE;
> +}
> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h
b/drivers/media/platform/qcom/venus/hfi_parser.h
> new file mode 100644
> index 000000000000..c484ac91a8e2
> --- /dev/null
> +++ b/drivers/media/platform/qcom/venus/hfi_parser.h
> @@ -0,0 +1,45 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/* Copyright (C) 2018 Linaro Ltd. */
> +#ifndef __VENUS_HFI_PARSER_H__
> +#define __VENUS_HFI_PARSER_H__
> +
> +#include "core.h"
> +
> +u32 hfi_parser(struct venus_core *core, struct venus_inst *inst,
> +              u32 num_properties, void *buf, u32 size);
> +
> +static inline struct hfi_capability *get_cap(struct venus_inst *inst,
u32 type)
> +{
> +       struct venus_core *core = inst->core;
> +       struct venus_caps *caps;
> +       unsigned int i;
> +
> +       caps = venus_caps_by_codec(core, inst->hfi_codec,
inst->session_type);
> +       if (!caps)
> +               return ERR_PTR(-EINVAL);
> +
> +       for (i = 0; i < MAX_CAP_ENTRIES; i++) {

Shouldn’t this iterate up to caps->num_capabilities? Also, shouldn’t
caps->caps[i]->valid be checked as well?

> +               if (caps->caps[i].capability_type == type)
> +                       return &caps->caps[i];
> +       }
> +
> +       return ERR_PTR(-EINVAL);
> +}

Best regards,
Tomasz

Re: [PATCH v2 12/29] venus: add common capability parser

[Stanimir Varbanov]

Hi Tomasz,

On 05/24/2018 05:16 PM, Tomasz Figa wrote:
> Hi Stanimir,
> 
> On Tue, May 15, 2018 at 5:08 PM Stanimir Varbanov <
> stanimir.varbanov@linaro.org> wrote:
> [snip]
>> diff --git a/drivers/media/platform/qcom/venus/core.c
> b/drivers/media/platform/qcom/venus/core.c
>> index 41eef376eb2d..381bfdd688db 100644
>> --- a/drivers/media/platform/qcom/venus/core.c
>> +++ b/drivers/media/platform/qcom/venus/core.c
> [snip]
>> +static int venus_enumerate_codecs(struct venus_core *core, u32 type)
>> +{
> [snip]
>> +       for (i = 0; i < MAX_CODEC_NUM; i++) {
>> +               codec = (1 << i) & codecs;
>> +               if (!codec)
>> +                       continue;
> 
> Could be simplified to
> 
>           for_each_set_bit(i, &codecs, MAX_CODEC_NUM) {
> 
> after making codecs an unsigned long.

OK, will rework that part.

> 
> [snip]
>> diff --git a/drivers/media/platform/qcom/venus/core.h
> b/drivers/media/platform/qcom/venus/core.h
>> index b5b9a84e9155..fe2d2b9e8af8 100644
>> --- a/drivers/media/platform/qcom/venus/core.h
>> +++ b/drivers/media/platform/qcom/venus/core.h
>> @@ -57,6 +57,29 @@ struct venus_format {
>>           u32 type;
>>    };
> 
>> +#define MAX_PLANES             4
> 
> We have VIDEO_MAX_PLANES (== 8) already.

yes, but venus has maximum of 4

> 
> [snip]
>> @@ -224,22 +249,8 @@ struct venus_buffer {
>>     * @priv:      a private for HFI operations callbacks
>>     * @session_type:      the type of the session (decoder or encoder)
>>     * @hprop:     a union used as a holder by get property
>> - * @cap_width: width capability
>> - * @cap_height:        height capability
>> - * @cap_mbs_per_frame: macroblocks per frame capability
>> - * @cap_mbs_per_sec:   macroblocks per second capability
>> - * @cap_framerate:     framerate capability
>> - * @cap_scale_x:               horizontal scaling capability
>> - * @cap_scale_y:               vertical scaling capability
>> - * @cap_bitrate:               bitrate capability
>> - * @cap_hier_p:                hier capability
>> - * @cap_ltr_count:     LTR count capability
>> - * @cap_secure_output2_threshold: secure OUTPUT2 threshold capability
>>     * @cap_bufs_mode_static:      buffers allocation mode capability
>>     * @cap_bufs_mode_dynamic:     buffers allocation mode capability
>> - * @pl_count:  count of supported profiles/levels
>> - * @pl:                supported profiles/levels
>> - * @bufreq:    holds buffer requirements
>>     */
>>    struct venus_inst {
>>           struct list_head list;
>> @@ -276,6 +287,7 @@ struct venus_inst {
>>           bool reconfig;
>>           u32 reconfig_width;
>>           u32 reconfig_height;
>> +       u32 hfi_codec;
> 
> Respective line not added to the kerneldoc above.

will add a description.

> 
>>           u32 sequence_cap;
>>           u32 sequence_out;
>>           struct v4l2_m2m_dev *m2m_dev;
>> @@ -287,22 +299,8 @@ struct venus_inst {
>>           const struct hfi_inst_ops *ops;
>>           u32 session_type;
>>           union hfi_get_property hprop;
>> -       struct hfi_capability cap_width;
>> -       struct hfi_capability cap_height;
>> -       struct hfi_capability cap_mbs_per_frame;
>> -       struct hfi_capability cap_mbs_per_sec;
>> -       struct hfi_capability cap_framerate;
>> -       struct hfi_capability cap_scale_x;
>> -       struct hfi_capability cap_scale_y;
>> -       struct hfi_capability cap_bitrate;
>> -       struct hfi_capability cap_hier_p;
>> -       struct hfi_capability cap_ltr_count;
>> -       struct hfi_capability cap_secure_output2_threshold;
>>           bool cap_bufs_mode_static;
>>           bool cap_bufs_mode_dynamic;
>> -       unsigned int pl_count;
>> -       struct hfi_profile_level pl[HFI_MAX_PROFILE_COUNT];
>> -       struct hfi_buffer_requirements bufreq[HFI_BUFFER_TYPE_MAX];
>>    };
> 
>>    #define IS_V1(core)    ((core)->res->hfi_version == HFI_VERSION_1XX)
>> @@ -322,4 +320,18 @@ static inline void *to_hfi_priv(struct venus_core
> *core)
>>           return core->priv;
>>    }
> 
>> +static inline struct venus_caps *
> 
> I'd leave the decision whether to inline this or not to the compiler.
> (Although these days the "inline" keyword is just a hint anyway... but
> still just wasted bytes in kernel's git repo.)

I just followed the other code examples in the kernel and in venus. If
you insist I can drop 'inline'.

> 
>> +venus_caps_by_codec(struct venus_core *core, u32 codec, u32 domain)
>> +{
>> +       unsigned int c;
>> +
>> +       for (c = 0; c < MAX_CODEC_NUM; c++) {
> 
> Shouldn't we iterate up to core->codecs_count?

yes, will correct.

> 
>> +               if (core->caps[c].codec == codec &&
>> +                   core->caps[c].domain == domain)
>> +                       return &core->caps[c];
>> +       }
>> +
>> +       return NULL;
>> +}
>> +

> [snip]
>> +       error = hfi_parser(core, inst, num_properties, &pkt->data[0],
> 
> nit: pkt->data?

OK. And also will drop num_properties because it is not used by
hfi_parser().

> 
> [snip]
>>   static void hfi_session_init_done(struct venus_core *core,
>>                                    struct venus_inst *inst, void *packet)
>>   {
>>          struct hfi_msg_session_init_done_pkt *pkt = packet;
>> -       unsigned int error;
>> +       u32 rem_bytes, error;
> 
>>          error = pkt->error_type;
>>          if (error != HFI_ERR_NONE)
>> @@ -745,8 +423,14 @@ static void hfi_session_init_done(struct venus_core
> *core,
>>          if (core->res->hfi_version != HFI_VERSION_1XX)
>>                  goto done;
> 
>> -       error = init_done_read_prop(core, inst, pkt);
>> +       rem_bytes = pkt->shdr.hdr.size - sizeof(*pkt) + sizeof(u32);
>> +       if (!rem_bytes) {
> 
> I’m not sure how likely it is to happen, but given that pkt->shdr.hdr.size
> seems to come from hardware, perhaps we should make rem_bytes signed and
> check for <= 0?

It comes from firmware through HFI interface. And yes we can check for
such anomalies.

> 
>> +               error = HFI_ERR_SESSION_INSUFFICIENT_RESOURCES;
>> +               goto done;
>> +       }
> 
>> +       error = hfi_parser(core, inst, pkt->num_properties, &pkt->data[0],
>> +                          rem_bytes);
> 
> nit: pkt->data?

OK.

> 
>>   done:
>>          inst->error = error;
>>          complete(&inst->done);
>> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c
> b/drivers/media/platform/qcom/venus/hfi_parser.c
>> new file mode 100644
>> index 000000000000..f9181d999b23
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/venus/hfi_parser.c
>> @@ -0,0 +1,295 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Copyright (C) 2018 Linaro Ltd.
>> + *
>> + * Author: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> + */
>> +#include <linux/kernel.h>
>> +
>> +#include "core.h"
>> +#include "hfi_helper.h"
>> +#include "hfi_parser.h"
>> +
>> +typedef void (*func)(struct venus_caps *cap, void *data, unsigned int
> size);
> 
> Should we perhaps make data const? (My understanding is that it comes from
> firmware packet.)

yes, it comes from firmware message packet.

> 
>> +
>> +static void init_codecs_vcaps(struct venus_core *core)
>> +{
>> +       struct venus_caps *caps = core->caps;
>> +       struct venus_caps *cap;
>> +       unsigned int i;
>> +
>> +       for (i = 0; i < 8 * sizeof(core->dec_codecs); i++) {
>> +               if ((1 << i) & core->dec_codecs) {
> 
> for_each_set_bit()?

OK, I'll give it a try.

> 
>> +                       cap = &caps[core->codecs_count++];
>> +                       cap->codec = (1 << i) & core->dec_codecs;
> 
> Any need to AND with core->dec_codecs? This code wouldn’t be executed if
> the bit wasn’t set in the first place.

Correct. Will fix it.

> 
> Also BIT(i).

Sure will use BIT().

> 
>> +                       cap->domain = VIDC_SESSION_TYPE_DEC;
>> +                       cap->valid = false;
>> +               }
>> +       }
>> +
>> +       for (i = 0; i < 8 * sizeof(core->enc_codecs); i++) {
>> +               if ((1 << i) & core->enc_codecs) {
> 
> Ditto.
> 
>> +                       cap = &caps[core->codecs_count++];
>> +                       cap->codec = (1 << i) & core->enc_codecs;
> 
> Ditto.
> 
>> +                       cap->domain = VIDC_SESSION_TYPE_ENC;
>> +                       cap->valid = false;
>> +               }
>> +       }
>> +}
>> +
>> +static void for_each_codec(struct venus_caps *caps, unsigned int
> caps_num,
>> +                          u32 codecs, u32 domain, func cb, void *data,
>> +                          unsigned int size)
>> +{
>> +       struct venus_caps *cap;
>> +       unsigned int i;
>> +
>> +       for (i = 0; i < caps_num; i++) {
>> +               cap = &caps[i];
>> +               if (cap->valid && cap->domain == domain)
> 
> Is there any need to check cap->domain == domain? We could just skip if
> cap->valid.

yes, we need to check the domain because we can have the same codec for
both domains decoder and encoder but with different capabilities.

> 
> If we want to shorten the code, we could even do (cap->valid || cap->domain
> != domain) and remove domain check from the if below.
> 
>> +                       continue;
>> +               if (cap->codec & codecs && cap->domain == domain)
>> +                       cb(cap, data, size);
>> +       }
>> +}
>> +
>> +static void fill_buf_mode(struct venus_caps *cap, void *data, unsigned
> int num)
>> +{
>> +       u32 *type = data;
>> +
>> +       if (*type == HFI_BUFFER_MODE_DYNAMIC)
>> +               cap->cap_bufs_mode_dynamic = true;
>> +}
>> +
>> +static void parse_alloc_mode(struct venus_core *core, struct venus_inst
> *inst,
>> +                            u32 codecs, u32 domain, void *data)
>> +{
>> +       struct hfi_buffer_alloc_mode_supported *mode = data;
>> +       u32 num_entries = mode->num_entries;
>> +       u32 *type;
>> +
>> +       if (num_entries > 16)
> 
> What is 16? We should have a macro for it.

sure, I forgot to add define for that.

> 
>> +               return;
>> +
>> +       type = mode->data;
>> +
>> +       while (num_entries--) {
>> +               if (mode->buffer_type == HFI_BUFFER_OUTPUT ||
>> +                   mode->buffer_type == HFI_BUFFER_OUTPUT2) {
>> +                       if (*type == HFI_BUFFER_MODE_DYNAMIC && inst)
>> +                               inst->cap_bufs_mode_dynamic = true;
>> +
>> +                       for_each_codec(core->caps, ARRAY_SIZE(core->caps),
>> +                                      codecs, domain, fill_buf_mode,
> type, 1);
>> +               }
>> +
>> +               type++;
>> +       }
>> +}
>> +
>> +static void parse_profile_level(u32 codecs, u32 domain, void *data)
>> +{
>> +       struct hfi_profile_level_supported *pl = data;
>> +       struct hfi_profile_level *proflevel = pl->profile_level;
>> +       u32 count = pl->profile_count;
>> +
>> +       if (count > HFI_MAX_PROFILE_COUNT)
>> +               return;
>> +
>> +       while (count) {
>> +               proflevel = (void *)proflevel + sizeof(*proflevel);
> 
> Isn’t this just ++proflevel?

yes

> 
>> +               count--;
>> +       }
> 
> Am I missing something or this function doesn’t to do anything?

yes, currently it is not used. I'll update it.

> 
>> +}
>> +
>> +static void fill_caps(struct venus_caps *cap, void *data, unsigned int
> num)
>> +{
>> +       struct hfi_capability *caps = data;
>> +       unsigned int i;
>> +
> 
> Should we have some check to avoid overflowing cap->caps[]?

No, we checked that below 'num_caps > MAX_CAP_ENTRIES'

> 
>> +       for (i = 0; i < num; i++)
>> +               cap->caps[cap->num_caps++] = caps[i];
>> +}
>> +
>> +static void parse_caps(struct venus_core *core, struct venus_inst *inst,
>> +                      u32 codecs, u32 domain, void *data)
>> +{
>> +       struct hfi_capabilities *caps = data;
>> +       struct hfi_capability *cap = caps->data;
>> +       u32 num_caps = caps->num_capabilities;
>> +       struct hfi_capability caps_arr[MAX_CAP_ENTRIES] = {};
>> +       unsigned int i = 0;
>> +
>> +       if (num_caps > MAX_CAP_ENTRIES)
>> +               return;
>> +
>> +       while (num_caps) {
>> +               caps_arr[i++] = *cap;
>> +               cap = (void *)cap + sizeof(*cap);
> 
> ++cap?
> 
>> +               num_caps--;
>> +       }
> 
> Hmm, isn’t the whole loop just
> 
> memcpy(caps_arr, cap, num_caps * sizeof(*cap));
> 
> ?

yes it is.

> 
>> +
>> +       for_each_codec(core->caps, ARRAY_SIZE(core->caps), codecs, domain,
>> +                      fill_caps, caps_arr, i);
>> +}
>> +
>> +static void fill_raw_fmts(struct venus_caps *cap, void *fmts,
>> +                         unsigned int num_fmts)
>> +{
>> +       struct raw_formats *formats = fmts;
>> +       unsigned int i;
>> +
>> +       for (i = 0; i < num_fmts; i++)
>> +               cap->fmts[cap->num_fmts++] = formats[i];
>> +}
>> +
>> +static void parse_raw_formats(struct venus_core *core, struct venus_inst
> *inst,
>> +                             u32 codecs, u32 domain, void *data)
>> +{
>> +       struct hfi_uncompressed_format_supported *fmt = data;
>> +       struct hfi_uncompressed_plane_info *pinfo = fmt->format_info;
>> +       struct hfi_uncompressed_plane_constraints *constr;
>> +       u32 entries = fmt->format_entries;
>> +       u32 num_planes;
>> +       struct raw_formats rfmts[MAX_FMT_ENTRIES] = {};
>> +       unsigned int i = 0;
>> +
>> +       while (entries) {
>> +               num_planes = pinfo->num_planes;
>> +
>> +               rfmts[i].fmt = pinfo->format;
>> +               rfmts[i].buftype = fmt->buffer_type;
>> +               i++;
>> +
>> +               if (pinfo->num_planes > MAX_PLANES)
>> +                       break;
>> +
>> +               constr = pinfo->plane_format;
>> +
>> +               while (pinfo->num_planes) {
>> +                       constr = (void *)constr + sizeof(*constr);
> 
> ++constr?
> 
>> +                       pinfo->num_planes--;
>> +               }
> 
> What is this loop supposed to do?

It is a leftover for constraints per format and plane. Currently we
don't use them, or at least the values returned by the firmware.

> 
>> +
>> +               pinfo = (void *)pinfo + sizeof(*constr) * num_planes +
>> +                       2 * sizeof(u32);
>> +               entries--;
>> +       }
>> +
>> +       for_each_codec(core->caps, ARRAY_SIZE(core->caps), codecs, domain,
>> +                      fill_raw_fmts, rfmts, i);
>> +}
> [snip]
>> +static void parser_fini(struct venus_core *core, struct venus_inst *inst,
>> +                       u32 codecs, u32 domain)
>> +{
>> +       struct venus_caps *caps = core->caps;
>> +       struct venus_caps *cap;
>> +       u32 dom;
>> +       unsigned int i;
>> +
>> +       if (core->res->hfi_version != HFI_VERSION_1XX)
>> +               return;
> 
> Hmm, if the code below is executed only for 1XX, who will set cap->valid to
> true for newer versions?

cap::valid is used only for v1xx. Will add a comment in the structure.

> 
>> +
>> +       if (!inst)
>> +               return;
>> +
>> +       dom = inst->session_type;
>> +
>> +       for (i = 0; i < MAX_CODEC_NUM; i++) {
>> +               cap = &caps[i];
>> +               if (cap->codec & codecs && cap->domain == dom)
>> +                       cap->valid = true;
>> +       }
>> +}
>> +
>> +u32 hfi_parser(struct venus_core *core, struct venus_inst *inst,
>> +              u32 num_properties, void *buf, u32 size)
>> +{
>> +       unsigned int words_count = size >> 2;
>> +       u32 *word = buf, *data, codecs = 0, domain = 0;
>> +
>> +       if (size % 4)
>> +               return HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
>> +
>> +       parser_init(core, inst, &codecs, &domain);
>> +
>> +       while (words_count) {
>> +               data = word + 1;
>> +
>> +               switch (*word) {
>> +               case HFI_PROPERTY_PARAM_CODEC_SUPPORTED:
>> +                       parse_codecs(core, data);
>> +                       init_codecs_vcaps(core);
>> +                       break;
>> +               case HFI_PROPERTY_PARAM_MAX_SESSIONS_SUPPORTED:
>> +                       parse_max_sessions(core, data);
>> +                       break;
>> +               case HFI_PROPERTY_PARAM_CODEC_MASK_SUPPORTED:
>> +                       parse_codecs_mask(&codecs, &domain, data);
>> +                       break;
>> +               case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SUPPORTED:
>> +                       parse_raw_formats(core, inst, codecs, domain,
> data);
>> +                       break;
>> +               case HFI_PROPERTY_PARAM_CAPABILITY_SUPPORTED:
>> +                       parse_caps(core, inst, codecs, domain, data);
>> +                       break;
>> +               case HFI_PROPERTY_PARAM_PROFILE_LEVEL_SUPPORTED:
>> +                       parse_profile_level(codecs, domain, data);
>> +                       break;
>> +               case HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE_SUPPORTED:
>> +                       parse_alloc_mode(core, inst, codecs, domain,
> data);
>> +                       break;
>> +               default:
> 
> Should we perhaps print something to let us know that something
> unrecognized was reported? (Or it is expected that something unrecognized
> is there?)

The default case will be very loaded with the data of the structures, so
I don't think a print is a good idea.

> 
>> +                       break;
>> +               }
>> +
>> +               word++;
>> +               words_count--;
> 
> If data is at |word + 1|, shouldn’t we increment |word| by |1 + |data
> size||?

yes, that could be possible but the firmware packets are with variable
data length and don't want to make the code so complex.

The idea is to search for HFI_PROPERTY_PARAM* key numbers. Yes it is not
optimal but this enumeration is happen only once during driver probe.

> 
>> +       }
>> +
>> +       parser_fini(core, inst, codecs, domain);
>> +
>> +       return HFI_ERR_NONE;
>> +}
>> diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h
> b/drivers/media/platform/qcom/venus/hfi_parser.h
>> new file mode 100644
>> index 000000000000..c484ac91a8e2
>> --- /dev/null
>> +++ b/drivers/media/platform/qcom/venus/hfi_parser.h
>> @@ -0,0 +1,45 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/* Copyright (C) 2018 Linaro Ltd. */
>> +#ifndef __VENUS_HFI_PARSER_H__
>> +#define __VENUS_HFI_PARSER_H__
>> +
>> +#include "core.h"
>> +
>> +u32 hfi_parser(struct venus_core *core, struct venus_inst *inst,
>> +              u32 num_properties, void *buf, u32 size);
>> +
>> +static inline struct hfi_capability *get_cap(struct venus_inst *inst,
> u32 type)
>> +{
>> +       struct venus_core *core = inst->core;
>> +       struct venus_caps *caps;
>> +       unsigned int i;
>> +
>> +       caps = venus_caps_by_codec(core, inst->hfi_codec,
> inst->session_type);
>> +       if (!caps)
>> +               return ERR_PTR(-EINVAL);
>> +
>> +       for (i = 0; i < MAX_CAP_ENTRIES; i++) {
> 
> Shouldn’t this iterate up to caps->num_capabilities? Also, shouldn’t
> caps->caps[i]->valid be checked as well?

most probably, will fix it.

> 
>> +               if (caps->caps[i].capability_type == type)
>> +                       return &caps->caps[i];
>> +       }
>> +
>> +       return ERR_PTR(-EINVAL);
>> +}


-- 
regards,
Stan

Re: [PATCH v2 12/29] venus: add common capability parser

[Tomasz Figa]

On Thu, May 31, 2018 at 1:21 AM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> Hi Tomasz,
>
> On 05/24/2018 05:16 PM, Tomasz Figa wrote:
> > Hi Stanimir,
> >
> > On Tue, May 15, 2018 at 5:08 PM Stanimir Varbanov <
> > [snip]
> >> diff --git a/drivers/media/platform/qcom/venus/core.h
> > b/drivers/media/platform/qcom/venus/core.h
> >> index b5b9a84e9155..fe2d2b9e8af8 100644
> >> --- a/drivers/media/platform/qcom/venus/core.h
> >> +++ b/drivers/media/platform/qcom/venus/core.h
> >> @@ -57,6 +57,29 @@ struct venus_format {
> >>           u32 type;
> >>    };
> >
> >> +#define MAX_PLANES             4
> >
> > We have VIDEO_MAX_PLANES (== 8) already.
>
> yes, but venus has maximum of 4
>

Generally we tend to avoid inventing new constants and so you can see
that many drivers just use VIDEO_MAX_PLANES. I can also see drivers
that don't, so I guess we can keep it as is.

> >>    #define IS_V1(core)    ((core)->res->hfi_version == HFI_VERSION_1XX)
> >> @@ -322,4 +320,18 @@ static inline void *to_hfi_priv(struct venus_core
> > *core)
> >>           return core->priv;
> >>    }
> >
> >> +static inline struct venus_caps *
> >
> > I'd leave the decision whether to inline this or not to the compiler.
> > (Although these days the "inline" keyword is just a hint anyway... but
> > still just wasted bytes in kernel's git repo.)
>
> I just followed the other code examples in the kernel and in venus. If
> you insist I can drop 'inline'.

https://www.kernel.org/doc/html/latest/process/coding-style.html#the-inline-disease

> >> +static void for_each_codec(struct venus_caps *caps, unsigned int
> > caps_num,
> >> +                          u32 codecs, u32 domain, func cb, void *data,
> >> +                          unsigned int size)
> >> +{
> >> +       struct venus_caps *cap;
> >> +       unsigned int i;
> >> +
> >> +       for (i = 0; i < caps_num; i++) {
> >> +               cap = &caps[i];
> >> +               if (cap->valid && cap->domain == domain)
> >
> > Is there any need to check cap->domain == domain? We could just skip if
> > cap->valid.
>
> yes, we need to check the domain because we can have the same codec for
> both domains decoder and encoder but with different capabilities.
>

Sorry, I guess my comment wasn't clear. The second if below was
already checking the domain.

> >
> > If we want to shorten the code, we could even do (cap->valid || cap->domain
> > != domain) and remove domain check from the if below.
> >
> >> +                       continue;
> >> +               if (cap->codec & codecs && cap->domain == domain)

Here ^

But generally, if we consider second part of my comment, the problem
would disappear.

> >> +                       cb(cap, data, size);
> >> +       }
> >> +}
[snip]
> >> +static void parse_profile_level(u32 codecs, u32 domain, void *data)
> >> +{
> >> +       struct hfi_profile_level_supported *pl = data;
> >> +       struct hfi_profile_level *proflevel = pl->profile_level;
> >> +       u32 count = pl->profile_count;
> >> +
> >> +       if (count > HFI_MAX_PROFILE_COUNT)
> >> +               return;
> >> +
> >> +       while (count) {
> >> +               proflevel = (void *)proflevel + sizeof(*proflevel);
> >
> > Isn’t this just ++proflevel?
>
> yes
>
> >
> >> +               count--;
> >> +       }
> >
> > Am I missing something or this function doesn’t to do anything?
>
> yes, currently it is not used. I'll update it.
>

I'd say we should just remove it for now and add it only when it is
actually needed for something.

> >
> >> +}
> >> +
> >> +static void fill_caps(struct venus_caps *cap, void *data, unsigned int
> > num)
> >> +{
> >> +       struct hfi_capability *caps = data;
> >> +       unsigned int i;
> >> +
> >
> > Should we have some check to avoid overflowing cap->caps[]?
>
> No, we checked that below 'num_caps > MAX_CAP_ENTRIES'
>

Ack.

> >> +static void parse_raw_formats(struct venus_core *core, struct venus_inst
> > *inst,
> >> +                             u32 codecs, u32 domain, void *data)
> >> +{
> >> +       struct hfi_uncompressed_format_supported *fmt = data;
> >> +       struct hfi_uncompressed_plane_info *pinfo = fmt->format_info;
> >> +       struct hfi_uncompressed_plane_constraints *constr;
> >> +       u32 entries = fmt->format_entries;
> >> +       u32 num_planes;
> >> +       struct raw_formats rfmts[MAX_FMT_ENTRIES] = {};
> >> +       unsigned int i = 0;
> >> +
> >> +       while (entries) {
> >> +               num_planes = pinfo->num_planes;
> >> +
> >> +               rfmts[i].fmt = pinfo->format;
> >> +               rfmts[i].buftype = fmt->buffer_type;
> >> +               i++;
> >> +
> >> +               if (pinfo->num_planes > MAX_PLANES)
> >> +                       break;
> >> +
> >> +               constr = pinfo->plane_format;
> >> +
> >> +               while (pinfo->num_planes) {
> >> +                       constr = (void *)constr + sizeof(*constr);
> >
> > ++constr?
> >
> >> +                       pinfo->num_planes--;
> >> +               }
> >
> > What is this loop supposed to do?
>
> It is a leftover for constraints per format and plane. Currently we
> don't use them, or at least the values returned by the firmware.
>

I guess it means we can remove it. :)

> >
> >> +
> >> +               pinfo = (void *)pinfo + sizeof(*constr) * num_planes +
> >> +                       2 * sizeof(u32);
> >> +               entries--;
> >> +       }
> >> +
> >> +       for_each_codec(core->caps, ARRAY_SIZE(core->caps), codecs, domain,
> >> +                      fill_raw_fmts, rfmts, i);
> >> +}
> > [snip]
> >> +static void parser_fini(struct venus_core *core, struct venus_inst *inst,
> >> +                       u32 codecs, u32 domain)
> >> +{
> >> +       struct venus_caps *caps = core->caps;
> >> +       struct venus_caps *cap;
> >> +       u32 dom;
> >> +       unsigned int i;
> >> +
> >> +       if (core->res->hfi_version != HFI_VERSION_1XX)
> >> +               return;
> >
> > Hmm, if the code below is executed only for 1XX, who will set cap->valid to
> > true for newer versions?
>
> cap::valid is used only for v1xx. Will add a comment in the structure.
>

Yes, please.

> >
> >> +
> >> +       if (!inst)
> >> +               return;
> >> +
> >> +       dom = inst->session_type;
> >> +
> >> +       for (i = 0; i < MAX_CODEC_NUM; i++) {
> >> +               cap = &caps[i];
> >> +               if (cap->codec & codecs && cap->domain == dom)
> >> +                       cap->valid = true;
> >> +       }
> >> +}
> >> +
> >> +u32 hfi_parser(struct venus_core *core, struct venus_inst *inst,
> >> +              u32 num_properties, void *buf, u32 size)
> >> +{
> >> +       unsigned int words_count = size >> 2;
> >> +       u32 *word = buf, *data, codecs = 0, domain = 0;
> >> +
> >> +       if (size % 4)
> >> +               return HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
> >> +
> >> +       parser_init(core, inst, &codecs, &domain);
> >> +
> >> +       while (words_count) {
> >> +               data = word + 1;
> >> +
> >> +               switch (*word) {
> >> +               case HFI_PROPERTY_PARAM_CODEC_SUPPORTED:
> >> +                       parse_codecs(core, data);
> >> +                       init_codecs_vcaps(core);
> >> +                       break;
> >> +               case HFI_PROPERTY_PARAM_MAX_SESSIONS_SUPPORTED:
> >> +                       parse_max_sessions(core, data);
> >> +                       break;
> >> +               case HFI_PROPERTY_PARAM_CODEC_MASK_SUPPORTED:
> >> +                       parse_codecs_mask(&codecs, &domain, data);
> >> +                       break;
> >> +               case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SUPPORTED:
> >> +                       parse_raw_formats(core, inst, codecs, domain,
> > data);
> >> +                       break;
> >> +               case HFI_PROPERTY_PARAM_CAPABILITY_SUPPORTED:
> >> +                       parse_caps(core, inst, codecs, domain, data);
> >> +                       break;
> >> +               case HFI_PROPERTY_PARAM_PROFILE_LEVEL_SUPPORTED:
> >> +                       parse_profile_level(codecs, domain, data);
> >> +                       break;
> >> +               case HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE_SUPPORTED:
> >> +                       parse_alloc_mode(core, inst, codecs, domain,
> > data);
> >> +                       break;
> >> +               default:
> >
> > Should we perhaps print something to let us know that something
> > unrecognized was reported? (Or it is expected that something unrecognized
> > is there?)
>
> The default case will be very loaded with the data of the structures, so
> I don't think a print is a good idea.
>

Ack.

> >
> >> +                       break;
> >> +               }
> >> +
> >> +               word++;
> >> +               words_count--;
> >
> > If data is at |word + 1|, shouldn’t we increment |word| by |1 + |data
> > size||?
>
> yes, that could be possible but the firmware packets are with variable
> data length and don't want to make the code so complex.
>
> The idea is to search for HFI_PROPERTY_PARAM* key numbers. Yes it is not
> optimal but this enumeration is happen only once during driver probe.
>

Hmm, do we have a guarantee that we will never find a value that
matches HFI_PROPERTY_PARAM*, but would be actually just some data
inside the payload?

Best regards,
Tomasz

Re: [PATCH v2 12/29] venus: add common capability parser

[Tomasz Figa]

On Thu, May 31, 2018 at 4:06 PM Tomasz Figa <tfiga@chromium.org> wrote:
>
> On Thu, May 31, 2018 at 1:21 AM Stanimir Varbanov
> <stanimir.varbanov@linaro.org> wrote:
> >
> > Hi Tomasz,
> >
> > On 05/24/2018 05:16 PM, Tomasz Figa wrote:
> > > Hi Stanimir,
> > >
> > > On Tue, May 15, 2018 at 5:08 PM Stanimir Varbanov <
[snip]
> > >
> > >> +                       break;
> > >> +               }
> > >> +
> > >> +               word++;
> > >> +               words_count--;
> > >
> > > If data is at |word + 1|, shouldn’t we increment |word| by |1 + |data
> > > size||?
> >
> > yes, that could be possible but the firmware packets are with variable
> > data length and don't want to make the code so complex.
> >
> > The idea is to search for HFI_PROPERTY_PARAM* key numbers. Yes it is not
> > optimal but this enumeration is happen only once during driver probe.
> >
>
> Hmm, do we have a guarantee that we will never find a value that
> matches HFI_PROPERTY_PARAM*, but would be actually just some data
> inside the payload?

Ping?

Best regards,
Tomasz

Re: [PATCH v2 12/29] venus: add common capability parser

[Stanimir Varbanov]

Hi Tomasz,

On 07/02/2018 12:23 PM, Tomasz Figa wrote:
> On Thu, May 31, 2018 at 4:06 PM Tomasz Figa <tfiga@chromium.org> wrote:
>>
>> On Thu, May 31, 2018 at 1:21 AM Stanimir Varbanov
>> <stanimir.varbanov@linaro.org> wrote:
>>>
>>> Hi Tomasz,
>>>
>>> On 05/24/2018 05:16 PM, Tomasz Figa wrote:
>>>> Hi Stanimir,
>>>>
>>>> On Tue, May 15, 2018 at 5:08 PM Stanimir Varbanov <
> [snip]
>>>>
>>>>> +                       break;
>>>>> +               }
>>>>> +
>>>>> +               word++;
>>>>> +               words_count--;
>>>>
>>>> If data is at |word + 1|, shouldn’t we increment |word| by |1 + |data
>>>> size||?
>>>
>>> yes, that could be possible but the firmware packets are with variable
>>> data length and don't want to make the code so complex.
>>>
>>> The idea is to search for HFI_PROPERTY_PARAM* key numbers. Yes it is not
>>> optimal but this enumeration is happen only once during driver probe.
>>>
>>
>> Hmm, do we have a guarantee that we will never find a value that
>> matches HFI_PROPERTY_PARAM*, but would be actually just some data
>> inside the payload?
> 
> Ping?

OK, you are right there is guarantee that we not mixing keywords and
data. I can make parse_* functions to return how words they consumed and
increment 'word' pointer with consumed words.

-- 
regards,
Stan

Re: [PATCH v2 12/29] venus: add common capability parser

[Tomasz Figa]

On Mon, Jul 2, 2018 at 6:59 PM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> Hi Tomasz,
>
> On 07/02/2018 12:23 PM, Tomasz Figa wrote:
> > On Thu, May 31, 2018 at 4:06 PM Tomasz Figa <tfiga@chromium.org> wrote:
> >>
> >> On Thu, May 31, 2018 at 1:21 AM Stanimir Varbanov
> >> <stanimir.varbanov@linaro.org> wrote:
> >>>
> >>> Hi Tomasz,
> >>>
> >>> On 05/24/2018 05:16 PM, Tomasz Figa wrote:
> >>>> Hi Stanimir,
> >>>>
> >>>> On Tue, May 15, 2018 at 5:08 PM Stanimir Varbanov <
> > [snip]
> >>>>
> >>>>> +                       break;
> >>>>> +               }
> >>>>> +
> >>>>> +               word++;
> >>>>> +               words_count--;
> >>>>
> >>>> If data is at |word + 1|, shouldn’t we increment |word| by |1 + |data
> >>>> size||?
> >>>
> >>> yes, that could be possible but the firmware packets are with variable
> >>> data length and don't want to make the code so complex.
> >>>
> >>> The idea is to search for HFI_PROPERTY_PARAM* key numbers. Yes it is not
> >>> optimal but this enumeration is happen only once during driver probe.
> >>>
> >>
> >> Hmm, do we have a guarantee that we will never find a value that
> >> matches HFI_PROPERTY_PARAM*, but would be actually just some data
> >> inside the payload?
> >
> > Ping?
>
> OK, you are right there is guarantee that we not mixing keywords and

Did the auto-correction engine in my head got this correctly as "no
guarantee"? :)

> data. I can make parse_* functions to return how words they consumed and
> increment 'word' pointer with consumed words.

Yes, that or maybe just returning the pointer to the first word after
consumed data. Most of the looping functions already seem to have this
value, so it would have to be just returned. (vs having to subtract
from the start pointer)

Best regards,
Tomasz

Re: [PATCH v2 12/29] venus: add common capability parser

[Stanimir Varbanov]

Hi Tomasz,

On 07/02/2018 01:05 PM, Tomasz Figa wrote:
> On Mon, Jul 2, 2018 at 6:59 PM Stanimir Varbanov
> <stanimir.varbanov@linaro.org> wrote:
>>
>> Hi Tomasz,
>>
>> On 07/02/2018 12:23 PM, Tomasz Figa wrote:
>>> On Thu, May 31, 2018 at 4:06 PM Tomasz Figa <tfiga@chromium.org> wrote:
>>>>
>>>> On Thu, May 31, 2018 at 1:21 AM Stanimir Varbanov
>>>> <stanimir.varbanov@linaro.org> wrote:
>>>>>
>>>>> Hi Tomasz,
>>>>>
>>>>> On 05/24/2018 05:16 PM, Tomasz Figa wrote:
>>>>>> Hi Stanimir,
>>>>>>
>>>>>> On Tue, May 15, 2018 at 5:08 PM Stanimir Varbanov <
>>> [snip]
>>>>>>
>>>>>>> +                       break;
>>>>>>> +               }
>>>>>>> +
>>>>>>> +               word++;
>>>>>>> +               words_count--;
>>>>>>
>>>>>> If data is at |word + 1|, shouldn’t we increment |word| by |1 + |data
>>>>>> size||?
>>>>>
>>>>> yes, that could be possible but the firmware packets are with variable
>>>>> data length and don't want to make the code so complex.
>>>>>
>>>>> The idea is to search for HFI_PROPERTY_PARAM* key numbers. Yes it is not
>>>>> optimal but this enumeration is happen only once during driver probe.
>>>>>
>>>>
>>>> Hmm, do we have a guarantee that we will never find a value that
>>>> matches HFI_PROPERTY_PARAM*, but would be actually just some data
>>>> inside the payload?
>>>
>>> Ping?
>>
>> OK, you are right there is guarantee that we not mixing keywords and
> 
> Did the auto-correction engine in my head got this correctly as "no
> guarantee"? :)

yes, your engine works better than my :)

> 
>> data. I can make parse_* functions to return how words they consumed and
>> increment 'word' pointer with consumed words.
> 
> Yes, that or maybe just returning the pointer to the first word after
> consumed data. Most of the looping functions already seem to have this
> value, so it would have to be just returned. (vs having to subtract
> from the start pointer)

One possible issue could be with not parsed params, there I have to
increment with one because the read data size is unknown.

-- 
regards,
Stan

Re: [PATCH v2 12/29] venus: add common capability parser

[Stanimir Varbanov]

Hi Tomasz,

On 07/02/2018 01:05 PM, Tomasz Figa wrote:
> On Mon, Jul 2, 2018 at 6:59 PM Stanimir Varbanov
> <stanimir.varbanov@linaro.org> wrote:
>>
>> Hi Tomasz,
>>
>> On 07/02/2018 12:23 PM, Tomasz Figa wrote:
>>> On Thu, May 31, 2018 at 4:06 PM Tomasz Figa <tfiga@chromium.org> wrote:
>>>>
>>>> On Thu, May 31, 2018 at 1:21 AM Stanimir Varbanov
>>>> <stanimir.varbanov@linaro.org> wrote:
>>>>>
>>>>> Hi Tomasz,
>>>>>
>>>>> On 05/24/2018 05:16 PM, Tomasz Figa wrote:
>>>>>> Hi Stanimir,
>>>>>>
>>>>>> On Tue, May 15, 2018 at 5:08 PM Stanimir Varbanov <
>>> [snip]
>>>>>>
>>>>>>> +                       break;
>>>>>>> +               }
>>>>>>> +
>>>>>>> +               word++;
>>>>>>> +               words_count--;
>>>>>>
>>>>>> If data is at |word + 1|, shouldn’t we increment |word| by |1 + |data
>>>>>> size||?
>>>>>
>>>>> yes, that could be possible but the firmware packets are with variable
>>>>> data length and don't want to make the code so complex.
>>>>>
>>>>> The idea is to search for HFI_PROPERTY_PARAM* key numbers. Yes it is not
>>>>> optimal but this enumeration is happen only once during driver probe.
>>>>>
>>>>
>>>> Hmm, do we have a guarantee that we will never find a value that
>>>> matches HFI_PROPERTY_PARAM*, but would be actually just some data
>>>> inside the payload?
>>>
>>> Ping?
>>
>> OK, you are right there is guarantee that we not mixing keywords and
> 
> Did the auto-correction engine in my head got this correctly as "no
> guarantee"? :)
> 
>> data. I can make parse_* functions to return how words they consumed and
>> increment 'word' pointer with consumed words.
> 
> Yes, that or maybe just returning the pointer to the first word after
> consumed data. Most of the looping functions already seem to have this
> value, so it would have to be just returned. (vs having to subtract
> from the start pointer)

I made the relevant changes to satisfy you request but the results were
fine for Venus v3 and wrong on v1. So I'd propose to postpone this
change and fix it with follow up patches because I don't want miss the
next merge window. So far the supported venus firmware versions are fine
with the current parser implementation.

What you think?

-- 
regards,
Stan

Re: [PATCH v2 12/29] venus: add common capability parser

[Tomasz Figa]

On Thu, Jul 5, 2018 at 6:45 PM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> Hi Tomasz,
>
> On 07/02/2018 01:05 PM, Tomasz Figa wrote:
> > On Mon, Jul 2, 2018 at 6:59 PM Stanimir Varbanov
> > <stanimir.varbanov@linaro.org> wrote:
> >>
> >> Hi Tomasz,
> >>
> >> On 07/02/2018 12:23 PM, Tomasz Figa wrote:
> >>> On Thu, May 31, 2018 at 4:06 PM Tomasz Figa <tfiga@chromium.org> wrote:
> >>>>
> >>>> On Thu, May 31, 2018 at 1:21 AM Stanimir Varbanov
> >>>> <stanimir.varbanov@linaro.org> wrote:
> >>>>>
> >>>>> Hi Tomasz,
> >>>>>
> >>>>> On 05/24/2018 05:16 PM, Tomasz Figa wrote:
> >>>>>> Hi Stanimir,
> >>>>>>
> >>>>>> On Tue, May 15, 2018 at 5:08 PM Stanimir Varbanov <
> >>> [snip]
> >>>>>>
> >>>>>>> +                       break;
> >>>>>>> +               }
> >>>>>>> +
> >>>>>>> +               word++;
> >>>>>>> +               words_count--;
> >>>>>>
> >>>>>> If data is at |word + 1|, shouldn’t we increment |word| by |1 + |data
> >>>>>> size||?
> >>>>>
> >>>>> yes, that could be possible but the firmware packets are with variable
> >>>>> data length and don't want to make the code so complex.
> >>>>>
> >>>>> The idea is to search for HFI_PROPERTY_PARAM* key numbers. Yes it is not
> >>>>> optimal but this enumeration is happen only once during driver probe.
> >>>>>
> >>>>
> >>>> Hmm, do we have a guarantee that we will never find a value that
> >>>> matches HFI_PROPERTY_PARAM*, but would be actually just some data
> >>>> inside the payload?
> >>>
> >>> Ping?
> >>
> >> OK, you are right there is guarantee that we not mixing keywords and
> >
> > Did the auto-correction engine in my head got this correctly as "no
> > guarantee"? :)
> >
> >> data. I can make parse_* functions to return how words they consumed and
> >> increment 'word' pointer with consumed words.
> >
> > Yes, that or maybe just returning the pointer to the first word after
> > consumed data. Most of the looping functions already seem to have this
> > value, so it would have to be just returned. (vs having to subtract
> > from the start pointer)
>
> I made the relevant changes to satisfy you request but the results were
> fine for Venus v3 and wrong on v1. So I'd propose to postpone this
> change and fix it with follow up patches because I don't want miss the
> next merge window. So far the supported venus firmware versions are fine
> with the current parser implementation.
>
> What you think?

Fair enough. Generally with the design of those metadata, fixing this
problem seems to be quite non-trivial. Let's keep it as is for now.

Best regards,
Tomasz

[PATCH v2 13/29] venus: helpers: make a commmon function for power_enable

[Stanimir Varbanov]

Make common function which will enable power when enabling/disabling
clocks and also covers Venus 3xx/4xx versions.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 51 +++++++++++++++++++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  2 ++
 drivers/media/platform/qcom/venus/vdec.c    | 25 ++++----------
 drivers/media/platform/qcom/venus/venc.c    | 25 ++++----------
 4 files changed, 67 insertions(+), 36 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index d9065cc8a7d3..2b21f6ed7502 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -13,6 +13,7 @@
  *
  */
 #include <linux/clk.h>
+#include <linux/iopoll.h>
 #include <linux/list.h>
 #include <linux/mutex.h>
 #include <linux/pm_runtime.h>
@@ -24,6 +25,7 @@
 #include "core.h"
 #include "helpers.h"
 #include "hfi_helper.h"
+#include "hfi_venus_io.h"
 
 struct intbuf {
 	struct list_head list;
@@ -781,3 +783,52 @@ void venus_helper_init_instance(struct venus_inst *inst)
 	}
 }
 EXPORT_SYMBOL_GPL(venus_helper_init_instance);
+
+int venus_helper_power_enable(struct venus_core *core, u32 session_type,
+			      bool enable)
+{
+	void __iomem *ctrl, *stat;
+	u32 val;
+	int ret;
+
+	if (!IS_V3(core) && !IS_V4(core))
+		return -EINVAL;
+
+	if (IS_V3(core)) {
+		if (session_type == VIDC_SESSION_TYPE_DEC)
+			ctrl = core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL;
+		else
+			ctrl = core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL;
+		if (enable)
+			writel(0, ctrl);
+		else
+			writel(1, ctrl);
+
+		return 0;
+	}
+
+	if (session_type == VIDC_SESSION_TYPE_DEC) {
+		ctrl = core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL;
+		stat = core->base + WRAPPER_VCODEC0_MMCC_POWER_STATUS;
+	} else {
+		ctrl = core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL;
+		stat = core->base + WRAPPER_VCODEC1_MMCC_POWER_STATUS;
+	}
+
+	if (enable) {
+		writel(0, ctrl);
+
+		ret = readl_poll_timeout(stat, val, val & BIT(1), 1, 100);
+		if (ret)
+			return ret;
+	} else {
+		writel(1, ctrl);
+
+		ret = readl_poll_timeout(stat, val, !(val & BIT(1)), 1, 100);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(venus_helper_power_enable);
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 971392be5df5..0e64aa95624a 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -43,4 +43,6 @@ int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
 void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
 void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
 void venus_helper_init_instance(struct venus_inst *inst);
+int venus_helper_power_enable(struct venus_core *core, u32 session_type,
+			      bool enable);
 #endif
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 3b38bd1241b0..2bd81de6328a 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -1123,26 +1123,21 @@ static int vdec_remove(struct platform_device *pdev)
 static __maybe_unused int vdec_runtime_suspend(struct device *dev)
 {
 	struct venus_core *core = dev_get_drvdata(dev);
+	int ret;
 
 	if (IS_V1(core))
 		return 0;
 
-	if (IS_V3(core))
-		writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(0, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+	ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, true);
 
 	if (IS_V4(core))
 		clk_disable_unprepare(core->core0_bus_clk);
 
 	clk_disable_unprepare(core->core0_clk);
 
-	if (IS_V3(core))
-		writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(1, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+	ret |= venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, false);
 
-	return 0;
+	return ret;
 }
 
 static __maybe_unused int vdec_runtime_resume(struct device *dev)
@@ -1153,20 +1148,14 @@ static __maybe_unused int vdec_runtime_resume(struct device *dev)
 	if (IS_V1(core))
 		return 0;
 
-	if (IS_V3(core))
-		writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(0, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+	ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, true);
 
-	ret = clk_prepare_enable(core->core0_clk);
+	ret |= clk_prepare_enable(core->core0_clk);
 
 	if (IS_V4(core))
 		ret |= clk_prepare_enable(core->core0_bus_clk);
 
-	if (IS_V3(core))
-		writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(1, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+	ret |= venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, false);
 
 	return ret;
 }
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index be8ea3326386..f87d891325ea 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -1267,26 +1267,21 @@ static int venc_remove(struct platform_device *pdev)
 static __maybe_unused int venc_runtime_suspend(struct device *dev)
 {
 	struct venus_core *core = dev_get_drvdata(dev);
+	int ret;
 
 	if (IS_V1(core))
 		return 0;
 
-	if (IS_V3(core))
-		writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(0, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+	ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, true);
 
 	if (IS_V4(core))
 		clk_disable_unprepare(core->core1_bus_clk);
 
 	clk_disable_unprepare(core->core1_clk);
 
-	if (IS_V3(core))
-		writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(1, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+	ret |= venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, false);
 
-	return 0;
+	return ret;
 }
 
 static __maybe_unused int venc_runtime_resume(struct device *dev)
@@ -1297,20 +1292,14 @@ static __maybe_unused int venc_runtime_resume(struct device *dev)
 	if (IS_V1(core))
 		return 0;
 
-	if (IS_V3(core))
-		writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(0, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+	ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, true);
 
-	ret = clk_prepare_enable(core->core1_clk);
+	ret |= clk_prepare_enable(core->core1_clk);
 
 	if (IS_V4(core))
 		ret |= clk_prepare_enable(core->core1_bus_clk);
 
-	if (IS_V3(core))
-		writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(1, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+	ret |= venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, false);
 
 	return ret;
 }
-- 
2.14.1

Re: [PATCH v2 13/29] venus: helpers: make a commmon function for power_enable

[Tomasz Figa]

Hi Stanimir,

On Tue, May 15, 2018 at 5:06 PM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> Make common function which will enable power when enabling/disabling
> clocks and also covers Venus 3xx/4xx versions.
>
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/helpers.c | 51 +++++++++++++++++++++++++++++
>  drivers/media/platform/qcom/venus/helpers.h |  2 ++
>  drivers/media/platform/qcom/venus/vdec.c    | 25 ++++----------
>  drivers/media/platform/qcom/venus/venc.c    | 25 ++++----------
>  4 files changed, 67 insertions(+), 36 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
> index d9065cc8a7d3..2b21f6ed7502 100644
> --- a/drivers/media/platform/qcom/venus/helpers.c
> +++ b/drivers/media/platform/qcom/venus/helpers.c
> @@ -13,6 +13,7 @@
>   *
>   */
>  #include <linux/clk.h>
> +#include <linux/iopoll.h>
>  #include <linux/list.h>
>  #include <linux/mutex.h>
>  #include <linux/pm_runtime.h>
> @@ -24,6 +25,7 @@
>  #include "core.h"
>  #include "helpers.h"
>  #include "hfi_helper.h"
> +#include "hfi_venus_io.h"
>
>  struct intbuf {
>         struct list_head list;
> @@ -781,3 +783,52 @@ void venus_helper_init_instance(struct venus_inst *inst)
>         }
>  }
>  EXPORT_SYMBOL_GPL(venus_helper_init_instance);
> +
> +int venus_helper_power_enable(struct venus_core *core, u32 session_type,
> +                             bool enable)
> +{
> +       void __iomem *ctrl, *stat;
> +       u32 val;
> +       int ret;
> +
> +       if (!IS_V3(core) && !IS_V4(core))
> +               return -EINVAL;
> +
> +       if (IS_V3(core)) {
> +               if (session_type == VIDC_SESSION_TYPE_DEC)
> +                       ctrl = core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL;
> +               else
> +                       ctrl = core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL;
> +               if (enable)
> +                       writel(0, ctrl);
> +               else
> +                       writel(1, ctrl);

nit: The value written is just !enable, but no strong preference.

> +
> +               return 0;
> +       }
> +
> +       if (session_type == VIDC_SESSION_TYPE_DEC) {
> +               ctrl = core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL;
> +               stat = core->base + WRAPPER_VCODEC0_MMCC_POWER_STATUS;
> +       } else {
> +               ctrl = core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL;
> +               stat = core->base + WRAPPER_VCODEC1_MMCC_POWER_STATUS;
> +       }
> +
> +       if (enable) {
> +               writel(0, ctrl);
> +
> +               ret = readl_poll_timeout(stat, val, val & BIT(1), 1, 100);
> +               if (ret)
> +                       return ret;
> +       } else {
> +               writel(1, ctrl);
> +
> +               ret = readl_poll_timeout(stat, val, !(val & BIT(1)), 1, 100);
> +               if (ret)
> +                       return ret;
> +       }

nit: The if/else could be just folded into code like below, but no
strong preference:

writel(!enable, ctrl);
ret = readl_poll_timeout(stat, val, !!(val & BIT(1)) == enable, 1, 100);
if (ret)
        return ret;

> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(venus_helper_power_enable);
> diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
> index 971392be5df5..0e64aa95624a 100644
> --- a/drivers/media/platform/qcom/venus/helpers.h
> +++ b/drivers/media/platform/qcom/venus/helpers.h
> @@ -43,4 +43,6 @@ int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
>  void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
>  void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
>  void venus_helper_init_instance(struct venus_inst *inst);
> +int venus_helper_power_enable(struct venus_core *core, u32 session_type,
> +                             bool enable);
>  #endif
> diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
> index 3b38bd1241b0..2bd81de6328a 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -1123,26 +1123,21 @@ static int vdec_remove(struct platform_device *pdev)
>  static __maybe_unused int vdec_runtime_suspend(struct device *dev)
>  {
>         struct venus_core *core = dev_get_drvdata(dev);
> +       int ret;
>
>         if (IS_V1(core))
>                 return 0;
>
> -       if (IS_V3(core))
> -               writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> -       else if (IS_V4(core))
> -               writel(0, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
> +       ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, true);
>
>         if (IS_V4(core))
>                 clk_disable_unprepare(core->core0_bus_clk);
>
>         clk_disable_unprepare(core->core0_clk);
>
> -       if (IS_V3(core))
> -               writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
> -       else if (IS_V4(core))
> -               writel(1, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
> +       ret |= venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, false);

If both calls return different error codes, the value of ret would
become garbage. (Same for rest of the patch.)

Best regards,
Tomasz

[PATCH v2 14/29] venus: core: delete not used flag for buffer mode

[Stanimir Varbanov]

Delete not used flag for capture buffer allocation mode.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index fe2d2b9e8af8..c46334454cd9 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -249,7 +249,6 @@ struct venus_buffer {
  * @priv:	a private for HFI operations callbacks
  * @session_type:	the type of the session (decoder or encoder)
  * @hprop:	a union used as a holder by get property
- * @cap_bufs_mode_static:	buffers allocation mode capability
  * @cap_bufs_mode_dynamic:	buffers allocation mode capability
  */
 struct venus_inst {
@@ -299,7 +298,6 @@ struct venus_inst {
 	const struct hfi_inst_ops *ops;
 	u32 session_type;
 	union hfi_get_property hprop;
-	bool cap_bufs_mode_static;
 	bool cap_bufs_mode_dynamic;
 };
 
-- 
2.14.1

[PATCH v2 15/29] venus: helpers: rename a helper function and use buffer mode from caps

[Stanimir Varbanov]

Rename is_reg_unreg_needed() to better name is_dynamic_bufmode() and
use buffer mode from enumerated per codec capabilities.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 2b21f6ed7502..1eda19adbf28 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -354,18 +354,19 @@ session_process_buf(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf)
 	return 0;
 }
 
-static inline int is_reg_unreg_needed(struct venus_inst *inst)
+static inline int is_dynamic_bufmode(struct venus_inst *inst)
 {
-	if (inst->session_type == VIDC_SESSION_TYPE_DEC &&
-	    inst->core->res->hfi_version == HFI_VERSION_3XX)
-		return 0;
+	struct venus_core *core = inst->core;
+	struct venus_caps *caps;
 
-	if (inst->session_type == VIDC_SESSION_TYPE_DEC &&
-	    inst->cap_bufs_mode_dynamic &&
-	    inst->core->res->hfi_version == HFI_VERSION_1XX)
+	caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
+	if (!caps)
 		return 0;
 
-	return 1;
+	if (caps->cap_bufs_mode_dynamic)
+		return 1;
+
+	return 0;
 }
 
 static int session_unregister_bufs(struct venus_inst *inst)
@@ -374,7 +375,7 @@ static int session_unregister_bufs(struct venus_inst *inst)
 	struct hfi_buffer_desc bd;
 	int ret = 0;
 
-	if (!is_reg_unreg_needed(inst))
+	if (is_dynamic_bufmode(inst))
 		return 0;
 
 	list_for_each_entry_safe(buf, n, &inst->registeredbufs, reg_list) {
@@ -394,7 +395,7 @@ static int session_register_bufs(struct venus_inst *inst)
 	struct venus_buffer *buf;
 	int ret = 0;
 
-	if (!is_reg_unreg_needed(inst))
+	if (is_dynamic_bufmode(inst))
 		return 0;
 
 	list_for_each_entry(buf, &inst->registeredbufs, reg_list) {
-- 
2.14.1

Re: [PATCH v2 15/29] venus: helpers: rename a helper function and use buffer mode from caps

[Tomasz Figa]

On Tue, May 15, 2018 at 5:06 PM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> Rename is_reg_unreg_needed() to better name is_dynamic_bufmode() and
> use buffer mode from enumerated per codec capabilities.
>
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/helpers.c | 21 +++++++++++----------
>  1 file changed, 11 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
> index 2b21f6ed7502..1eda19adbf28 100644
> --- a/drivers/media/platform/qcom/venus/helpers.c
> +++ b/drivers/media/platform/qcom/venus/helpers.c
> @@ -354,18 +354,19 @@ session_process_buf(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf)
>         return 0;
>  }
>
> -static inline int is_reg_unreg_needed(struct venus_inst *inst)
> +static inline int is_dynamic_bufmode(struct venus_inst *inst)

nit: Could be made bool.

>  {
> -       if (inst->session_type == VIDC_SESSION_TYPE_DEC &&
> -           inst->core->res->hfi_version == HFI_VERSION_3XX)
> -               return 0;
> +       struct venus_core *core = inst->core;
> +       struct venus_caps *caps;
>
> -       if (inst->session_type == VIDC_SESSION_TYPE_DEC &&
> -           inst->cap_bufs_mode_dynamic &&
> -           inst->core->res->hfi_version == HFI_VERSION_1XX)
> +       caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
> +       if (!caps)
>                 return 0;
>
> -       return 1;
> +       if (caps->cap_bufs_mode_dynamic)
> +               return 1;
> +
> +       return 0;

nit: return caps->cap_bufs_mode_dynamic;

Best regards,
Tomasz

Re: [PATCH v2 15/29] venus: helpers: rename a helper function and use buffer mode from caps

[Stanimir Varbanov]

Hi Tomasz,

Thanks for the review!

On 05/31/2018 10:59 AM, Tomasz Figa wrote:
> On Tue, May 15, 2018 at 5:06 PM Stanimir Varbanov
> <stanimir.varbanov@linaro.org> wrote:
>>
>> Rename is_reg_unreg_needed() to better name is_dynamic_bufmode() and
>> use buffer mode from enumerated per codec capabilities.
>>
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>  drivers/media/platform/qcom/venus/helpers.c | 21 +++++++++++----------
>>  1 file changed, 11 insertions(+), 10 deletions(-)
>>
>> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
>> index 2b21f6ed7502..1eda19adbf28 100644
>> --- a/drivers/media/platform/qcom/venus/helpers.c
>> +++ b/drivers/media/platform/qcom/venus/helpers.c
>> @@ -354,18 +354,19 @@ session_process_buf(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf)
>>         return 0;
>>  }
>>
>> -static inline int is_reg_unreg_needed(struct venus_inst *inst)
>> +static inline int is_dynamic_bufmode(struct venus_inst *inst)
> 
> nit: Could be made bool.

And drop inline I guess? :)

-- 
regards,
Stan

[PATCH v2 16/29] venus: add a helper function to set dynamic buffer mode

[Stanimir Varbanov]

Adds a new helper function to set dynamic buffer mode if it is
supported by current HFI version.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 22 ++++++++++++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  1 +
 drivers/media/platform/qcom/venus/vdec.c    | 15 +++------------
 3 files changed, 26 insertions(+), 12 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 1eda19adbf28..824ad4d2d064 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -522,6 +522,28 @@ int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
 
+int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
+{
+	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
+	struct hfi_buffer_alloc_mode mode;
+	int ret;
+
+	if (!is_dynamic_bufmode(inst))
+		return 0;
+
+	mode.type = HFI_BUFFER_OUTPUT;
+	mode.mode = HFI_BUFFER_MODE_DYNAMIC;
+
+	ret = hfi_session_set_property(inst, ptype, &mode);
+	if (ret)
+		return ret;
+
+	mode.type = HFI_BUFFER_OUTPUT2;
+
+	return hfi_session_set_property(inst, ptype, &mode);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_dyn_bufmode);
+
 static void delayed_process_buf_func(struct work_struct *work)
 {
 	struct venus_buffer *buf, *n;
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 0e64aa95624a..52b961ed491e 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -40,6 +40,7 @@ int venus_helper_set_output_resolution(struct venus_inst *inst,
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs);
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
+int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
 void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
 void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
 void venus_helper_init_instance(struct venus_inst *inst);
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 2bd81de6328a..271192273953 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -557,18 +557,9 @@ static int vdec_set_properties(struct venus_inst *inst)
 			return ret;
 	}
 
-	if (core->res->hfi_version == HFI_VERSION_3XX ||
-	    inst->cap_bufs_mode_dynamic) {
-		struct hfi_buffer_alloc_mode mode;
-
-		ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
-		mode.type = HFI_BUFFER_OUTPUT;
-		mode.mode = HFI_BUFFER_MODE_DYNAMIC;
-
-		ret = hfi_session_set_property(inst, ptype, &mode);
-		if (ret)
-			return ret;
-	}
+	ret = venus_helper_set_dyn_bufmode(inst);
+	if (ret)
+		return ret;
 
 	if (ctr->post_loop_deb_mode) {
 		ptype = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
-- 
2.14.1

Re: [PATCH v2 16/29] venus: add a helper function to set dynamic buffer mode

[Tomasz Figa]

On Tue, May 15, 2018 at 5:05 PM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> Adds a new helper function to set dynamic buffer mode if it is
> supported by current HFI version.
>
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/helpers.c | 22 ++++++++++++++++++++++
>  drivers/media/platform/qcom/venus/helpers.h |  1 +
>  drivers/media/platform/qcom/venus/vdec.c    | 15 +++------------
>  3 files changed, 26 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
> index 1eda19adbf28..824ad4d2d064 100644
> --- a/drivers/media/platform/qcom/venus/helpers.c
> +++ b/drivers/media/platform/qcom/venus/helpers.c
> @@ -522,6 +522,28 @@ int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
>  }
>  EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
>
> +int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
> +{
> +       u32 ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
> +       struct hfi_buffer_alloc_mode mode;
> +       int ret;
> +
> +       if (!is_dynamic_bufmode(inst))
> +               return 0;
> +
> +       mode.type = HFI_BUFFER_OUTPUT;
> +       mode.mode = HFI_BUFFER_MODE_DYNAMIC;
> +
> +       ret = hfi_session_set_property(inst, ptype, &mode);
> +       if (ret)
> +               return ret;
> +
> +       mode.type = HFI_BUFFER_OUTPUT2;
> +
> +       return hfi_session_set_property(inst, ptype, &mode);

The function now sets HFI_BUFFER_OUTPUT2 in addition to
HFI_BUFFER_OUTPUT only, as set by orignal code. Is it intentional? I
guess we could have this mentioned in commit message.

Best regards,
Tomasz

[PATCH v2 17/29] venus: add helper function to set actual buffer size

[Stanimir Varbanov]

Add and use a helper function to set actual buffer size for
particular buffer type. This is also preparation to use
the second decoder output.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 12 ++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  1 +
 drivers/media/platform/qcom/venus/vdec.c    | 10 ++--------
 3 files changed, 15 insertions(+), 8 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 824ad4d2d064..94664a3ce3e2 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -544,6 +544,18 @@ int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_dyn_bufmode);
 
+int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype)
+{
+	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL;
+	struct hfi_buffer_size_actual bufsz;
+
+	bufsz.type = buftype;
+	bufsz.size = bufsize;
+
+	return hfi_session_set_property(inst, ptype, &bufsz);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_bufsize);
+
 static void delayed_process_buf_func(struct work_struct *work)
 {
 	struct venus_buffer *buf, *n;
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 52b961ed491e..cd306bd8978f 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -41,6 +41,7 @@ int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs);
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
+int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype);
 void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
 void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
 void venus_helper_init_instance(struct venus_inst *inst);
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 271192273953..e8e00d0650e9 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -710,7 +710,6 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 {
 	struct venus_inst *inst = vb2_get_drv_priv(q);
 	struct venus_core *core = inst->core;
-	u32 ptype;
 	int ret;
 
 	mutex_lock(&inst->lock);
@@ -740,13 +739,8 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 		goto deinit_sess;
 
 	if (core->res->hfi_version == HFI_VERSION_3XX) {
-		struct hfi_buffer_size_actual buf_sz;
-
-		ptype = HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL;
-		buf_sz.type = HFI_BUFFER_OUTPUT;
-		buf_sz.size = inst->output_buf_size;
-
-		ret = hfi_session_set_property(inst, ptype, &buf_sz);
+		ret = venus_helper_set_bufsize(inst, inst->output_buf_size,
+					       HFI_BUFFER_OUTPUT);
 		if (ret)
 			goto deinit_sess;
 	}
-- 
2.14.1

[PATCH v2 18/29] venus: delete no longer used bufmode flag from instance

[Stanimir Varbanov]

Delete no longer used flag cap_bufs_mode_dynamic from instance
structure.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h       | 2 --
 drivers/media/platform/qcom/venus/hfi_parser.c | 6 +-----
 2 files changed, 1 insertion(+), 7 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index c46334454cd9..255292899204 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -249,7 +249,6 @@ struct venus_buffer {
  * @priv:	a private for HFI operations callbacks
  * @session_type:	the type of the session (decoder or encoder)
  * @hprop:	a union used as a holder by get property
- * @cap_bufs_mode_dynamic:	buffers allocation mode capability
  */
 struct venus_inst {
 	struct list_head list;
@@ -298,7 +297,6 @@ struct venus_inst {
 	const struct hfi_inst_ops *ops;
 	u32 session_type;
 	union hfi_get_property hprop;
-	bool cap_bufs_mode_dynamic;
 };
 
 #define IS_V1(core)	((core)->res->hfi_version == HFI_VERSION_1XX)
diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c b/drivers/media/platform/qcom/venus/hfi_parser.c
index f9181d999b23..ad039a3444b8 100644
--- a/drivers/media/platform/qcom/venus/hfi_parser.c
+++ b/drivers/media/platform/qcom/venus/hfi_parser.c
@@ -75,13 +75,9 @@ static void parse_alloc_mode(struct venus_core *core, struct venus_inst *inst,
 
 	while (num_entries--) {
 		if (mode->buffer_type == HFI_BUFFER_OUTPUT ||
-		    mode->buffer_type == HFI_BUFFER_OUTPUT2) {
-			if (*type == HFI_BUFFER_MODE_DYNAMIC && inst)
-				inst->cap_bufs_mode_dynamic = true;
-
+		    mode->buffer_type == HFI_BUFFER_OUTPUT2)
 			for_each_codec(core->caps, ARRAY_SIZE(core->caps),
 				       codecs, domain, fill_buf_mode, type, 1);
-		}
 
 		type++;
 	}
-- 
2.14.1

[PATCH v2 19/29] venus: helpers: add buffer type argument to a helper

[Stanimir Varbanov]

This adds one more function argument to pass buffer type to
set_output_resolution() helper function. That is a preparation
to support secondary decoder output.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 5 +++--
 drivers/media/platform/qcom/venus/helpers.h | 3 ++-
 drivers/media/platform/qcom/venus/venc.c    | 3 ++-
 3 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 94664a3ce3e2..5512fbfdebb9 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -456,12 +456,13 @@ int venus_helper_set_input_resolution(struct venus_inst *inst,
 EXPORT_SYMBOL_GPL(venus_helper_set_input_resolution);
 
 int venus_helper_set_output_resolution(struct venus_inst *inst,
-				       unsigned int width, unsigned int height)
+				       unsigned int width, unsigned int height,
+				       u32 buftype)
 {
 	u32 ptype = HFI_PROPERTY_PARAM_FRAME_SIZE;
 	struct hfi_framesize fs;
 
-	fs.buffer_type = HFI_BUFFER_OUTPUT;
+	fs.buffer_type = buftype;
 	fs.width = width;
 	fs.height = height;
 
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index cd306bd8978f..0de9989adcdb 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -36,7 +36,8 @@ int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
 int venus_helper_set_input_resolution(struct venus_inst *inst,
 				      unsigned int width, unsigned int height);
 int venus_helper_set_output_resolution(struct venus_inst *inst,
-				       unsigned int width, unsigned int height);
+				       unsigned int width, unsigned int height,
+				       u32 buftype);
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs);
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index f87d891325ea..8970f14b3a82 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -795,7 +795,8 @@ static int venc_init_session(struct venus_inst *inst)
 		goto deinit;
 
 	ret = venus_helper_set_output_resolution(inst, inst->width,
-						 inst->height);
+						 inst->height,
+						 HFI_BUFFER_OUTPUT);
 	if (ret)
 		goto deinit;
 
-- 
2.14.1

[PATCH v2 20/29] venus: helpers: add a new helper to set raw format

[Stanimir Varbanov]

The new helper will has one more argument for buffer type, that
way the decoder can configure the format on it's secondary
output.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 52 ++++++++++++++++++-----------
 drivers/media/platform/qcom/venus/helpers.h |  2 ++
 2 files changed, 35 insertions(+), 19 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 5512fbfdebb9..0d55604f7484 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -410,6 +410,20 @@ static int session_register_bufs(struct venus_inst *inst)
 	return ret;
 }
 
+static u32 to_hfi_raw_fmt(u32 v4l2_fmt)
+{
+	switch (v4l2_fmt) {
+	case V4L2_PIX_FMT_NV12:
+		return HFI_COLOR_FORMAT_NV12;
+	case V4L2_PIX_FMT_NV21:
+		return HFI_COLOR_FORMAT_NV21;
+	default:
+		break;
+	}
+
+	return 0;
+}
+
 int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
 			    struct hfi_buffer_requirements *req)
 {
@@ -491,35 +505,35 @@ int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_num_bufs);
 
-int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
+int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
+				u32 buftype)
 {
-	struct hfi_uncompressed_format_select fmt;
 	u32 ptype = HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT;
-	int ret;
+	struct hfi_uncompressed_format_select fmt;
+
+	fmt.buffer_type = buftype;
+	fmt.format = hfi_format;
+
+	return hfi_session_set_property(inst, ptype, &fmt);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_raw_format);
+
+int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
+{
+	u32 hfi_format, buftype;
 
 	if (inst->session_type == VIDC_SESSION_TYPE_DEC)
-		fmt.buffer_type = HFI_BUFFER_OUTPUT;
+		buftype = HFI_BUFFER_OUTPUT;
 	else if (inst->session_type == VIDC_SESSION_TYPE_ENC)
-		fmt.buffer_type = HFI_BUFFER_INPUT;
+		buftype = HFI_BUFFER_INPUT;
 	else
 		return -EINVAL;
 
-	switch (pixfmt) {
-	case V4L2_PIX_FMT_NV12:
-		fmt.format = HFI_COLOR_FORMAT_NV12;
-		break;
-	case V4L2_PIX_FMT_NV21:
-		fmt.format = HFI_COLOR_FORMAT_NV21;
-		break;
-	default:
+	hfi_format = to_hfi_raw_fmt(pixfmt);
+	if (!hfi_format)
 		return -EINVAL;
-	}
 
-	ret = hfi_session_set_property(inst, ptype, &fmt);
-	if (ret)
-		return ret;
-
-	return 0;
+	return venus_helper_set_raw_format(inst, hfi_format, buftype);
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
 
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 0de9989adcdb..79af7845efbd 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -40,6 +40,8 @@ int venus_helper_set_output_resolution(struct venus_inst *inst,
 				       u32 buftype);
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs);
+int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
+				u32 buftype);
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
 int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype);
-- 
2.14.1

[PATCH v2 21/29] venus: helpers,vdec,venc: add helpers to set work mode and core usage

[Stanimir Varbanov]

These are new properties applicable to Venus version 4xx. Add the
helpers and call them from decoder and encoder drivers.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 28 ++++++++++++++++++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  2 ++
 drivers/media/platform/qcom/venus/vdec.c    |  8 ++++++++
 drivers/media/platform/qcom/venus/venc.c    |  8 ++++++++
 4 files changed, 46 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 0d55604f7484..adf8701a64bb 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -484,6 +484,34 @@ int venus_helper_set_output_resolution(struct venus_inst *inst,
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_output_resolution);
 
+int venus_helper_set_work_mode(struct venus_inst *inst, u32 mode)
+{
+	u32 ptype = HFI_PROPERTY_PARAM_WORK_MODE;
+	struct hfi_video_work_mode wm;
+
+	if (!IS_V4(inst->core))
+		return 0;
+
+	wm.video_work_mode = mode;
+
+	return hfi_session_set_property(inst, ptype, &wm);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_work_mode);
+
+int venus_helper_set_core_usage(struct venus_inst *inst, u32 usage)
+{
+	u32 ptype = HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE;
+	struct hfi_videocores_usage_type cu;
+
+	if (!IS_V4(inst->core))
+		return 0;
+
+	cu.video_core_enable_mask = usage;
+
+	return hfi_session_set_property(inst, ptype, &cu);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_core_usage);
+
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs)
 {
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 79af7845efbd..d5e727e1ecab 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -38,6 +38,8 @@ int venus_helper_set_input_resolution(struct venus_inst *inst,
 int venus_helper_set_output_resolution(struct venus_inst *inst,
 				       unsigned int width, unsigned int height,
 				       u32 buftype);
+int venus_helper_set_work_mode(struct venus_inst *inst, u32 mode);
+int venus_helper_set_core_usage(struct venus_inst *inst, u32 usage);
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs);
 int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index e8e00d0650e9..3c7ffebe4bad 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -550,6 +550,14 @@ static int vdec_set_properties(struct venus_inst *inst)
 	u32 ptype;
 	int ret;
 
+	ret = venus_helper_set_work_mode(inst, VIDC_WORK_MODE_2);
+	if (ret)
+		return ret;
+
+	ret = venus_helper_set_core_usage(inst, VIDC_CORE_ID_1);
+	if (ret)
+		return ret;
+
 	if (core->res->hfi_version == HFI_VERSION_1XX) {
 		ptype = HFI_PROPERTY_PARAM_VDEC_CONTINUE_DATA_TRANSFER;
 		ret = hfi_session_set_property(inst, ptype, &en);
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 8970f14b3a82..3b3299bff1cd 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -643,6 +643,14 @@ static int venc_set_properties(struct venus_inst *inst)
 	u32 ptype, rate_control, bitrate, profile = 0, level = 0;
 	int ret;
 
+	ret = venus_helper_set_work_mode(inst, VIDC_WORK_MODE_2);
+	if (ret)
+		return ret;
+
+	ret = venus_helper_set_core_usage(inst, VIDC_CORE_ID_2);
+	if (ret)
+		return ret;
+
 	ptype = HFI_PROPERTY_CONFIG_FRAME_RATE;
 	frate.buffer_type = HFI_BUFFER_OUTPUT;
 	frate.framerate = inst->fps * (1 << 16);
-- 
2.14.1

[PATCH v2 22/29] venus: helpers: extend set_num_bufs helper with one more argument

[Stanimir Varbanov]

Extend venus_helper_set_num_bufs() helper function with one more
argument to set number of output buffers for the secondary decoder
output.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 16 ++++++++++++++--
 drivers/media/platform/qcom/venus/helpers.h |  3 ++-
 drivers/media/platform/qcom/venus/vdec.c    |  2 +-
 drivers/media/platform/qcom/venus/venc.c    |  2 +-
 4 files changed, 18 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index adf8701a64bb..f04d16953b3a 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -513,7 +513,8 @@ int venus_helper_set_core_usage(struct venus_inst *inst, u32 usage)
 EXPORT_SYMBOL_GPL(venus_helper_set_core_usage);
 
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
-			      unsigned int output_bufs)
+			      unsigned int output_bufs,
+			      unsigned int output2_bufs)
 {
 	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
 	struct hfi_buffer_count_actual buf_count;
@@ -529,7 +530,18 @@ int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 	buf_count.type = HFI_BUFFER_OUTPUT;
 	buf_count.count_actual = output_bufs;
 
-	return hfi_session_set_property(inst, ptype, &buf_count);
+	ret = hfi_session_set_property(inst, ptype, &buf_count);
+	if (ret)
+		return ret;
+
+	if (output2_bufs) {
+		buf_count.type = HFI_BUFFER_OUTPUT2;
+		buf_count.count_actual = output2_bufs;
+
+		ret = hfi_session_set_property(inst, ptype, &buf_count);
+	}
+
+	return ret;
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_num_bufs);
 
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index d5e727e1ecab..8ff4bd3ef958 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -41,7 +41,8 @@ int venus_helper_set_output_resolution(struct venus_inst *inst,
 int venus_helper_set_work_mode(struct venus_inst *inst, u32 mode);
 int venus_helper_set_core_usage(struct venus_inst *inst, u32 usage);
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
-			      unsigned int output_bufs);
+			      unsigned int output_bufs,
+			      unsigned int output2_bufs);
 int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
 				u32 buftype);
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 3c7ffebe4bad..898c5edb91f5 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -758,7 +758,7 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 		goto deinit_sess;
 
 	ret = venus_helper_set_num_bufs(inst, inst->num_input_bufs,
-					VB2_MAX_FRAME);
+					VB2_MAX_FRAME, VB2_MAX_FRAME);
 	if (ret)
 		goto deinit_sess;
 
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 3b3299bff1cd..c9c40d1ce7c6 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -963,7 +963,7 @@ static int venc_start_streaming(struct vb2_queue *q, unsigned int count)
 		goto deinit_sess;
 
 	ret = venus_helper_set_num_bufs(inst, inst->num_input_bufs,
-					inst->num_output_bufs);
+					inst->num_output_bufs, 0);
 	if (ret)
 		goto deinit_sess;
 
-- 
2.14.1

[PATCH v2 23/29] venus: helpers: add a helper to return opb buffer sizes

[Stanimir Varbanov]

Add a helper function to return current output picture buffer size.
OPB sizes can vary depending on the selected decoder output(s).

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h    | 10 ++++++++++
 drivers/media/platform/qcom/venus/helpers.c | 15 +++++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  1 +
 3 files changed, 26 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 255292899204..4d6c05f156c4 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -234,6 +234,11 @@ struct venus_buffer {
  * @num_output_bufs:	holds number of output buffers
  * @input_buf_size	holds input buffer size
  * @output_buf_size:	holds output buffer size
+ * @output2_buf_size:	holds secondary decoder output buffer size
+ * @dpb_buftype:	decoded picture buffer type
+ * @dpb_fmt:		decodec picture buffre raw format
+ * @opb_buftype:	output picture buffer type
+ * @opb_fmt:		output picture buffer raw format
  * @reconfig:	a flag raised by decoder when the stream resolution changed
  * @reconfig_width:	holds the new width
  * @reconfig_height:	holds the new height
@@ -282,6 +287,11 @@ struct venus_inst {
 	unsigned int num_output_bufs;
 	unsigned int input_buf_size;
 	unsigned int output_buf_size;
+	unsigned int output2_buf_size;
+	u32 dpb_buftype;
+	u32 dpb_fmt;
+	u32 opb_buftype;
+	u32 opb_fmt;
 	bool reconfig;
 	u32 reconfig_width;
 	u32 reconfig_height;
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index f04d16953b3a..f0a0fca60c76 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -611,6 +611,21 @@ int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype)
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_bufsize);
 
+unsigned int venus_helper_get_opb_size(struct venus_inst *inst)
+{
+	/* the encoder has only one output */
+	if (inst->session_type == VIDC_SESSION_TYPE_ENC)
+		return inst->output_buf_size;
+
+	if (inst->opb_buftype == HFI_BUFFER_OUTPUT)
+		return inst->output_buf_size;
+	else if (inst->opb_buftype == HFI_BUFFER_OUTPUT2)
+		return inst->output2_buf_size;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(venus_helper_get_opb_size);
+
 static void delayed_process_buf_func(struct work_struct *work)
 {
 	struct venus_buffer *buf, *n;
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 8ff4bd3ef958..92be45894a69 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -48,6 +48,7 @@ int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
 int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype);
+unsigned int venus_helper_get_opb_size(struct venus_inst *inst);
 void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
 void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
 void venus_helper_init_instance(struct venus_inst *inst);
-- 
2.14.1

Re: [PATCH v2 23/29] venus: helpers: add a helper to return opb buffer sizes

[Tomasz Figa]

On Tue, May 15, 2018 at 5:02 PM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> Add a helper function to return current output picture buffer size.
> OPB sizes can vary depending on the selected decoder output(s).
>
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/core.h    | 10 ++++++++++
>  drivers/media/platform/qcom/venus/helpers.c | 15 +++++++++++++++
>  drivers/media/platform/qcom/venus/helpers.h |  1 +
>  3 files changed, 26 insertions(+)
>
> diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
> index 255292899204..4d6c05f156c4 100644
> --- a/drivers/media/platform/qcom/venus/core.h
> +++ b/drivers/media/platform/qcom/venus/core.h
> @@ -234,6 +234,11 @@ struct venus_buffer {
>   * @num_output_bufs:   holds number of output buffers
>   * @input_buf_size     holds input buffer size
>   * @output_buf_size:   holds output buffer size
> + * @output2_buf_size:  holds secondary decoder output buffer size
> + * @dpb_buftype:       decoded picture buffer type
> + * @dpb_fmt:           decodec picture buffre raw format

typo: s/decodec/decoded/ and s/buffre/buffer/

> + * @opb_buftype:       output picture buffer type
> + * @opb_fmt:           output picture buffer raw format
>   * @reconfig:  a flag raised by decoder when the stream resolution changed
>   * @reconfig_width:    holds the new width
>   * @reconfig_height:   holds the new height
> @@ -282,6 +287,11 @@ struct venus_inst {
>         unsigned int num_output_bufs;
>         unsigned int input_buf_size;
>         unsigned int output_buf_size;
> +       unsigned int output2_buf_size;
> +       u32 dpb_buftype;
> +       u32 dpb_fmt;

These 2 don't seem to be used.

Best regards,
Tomasz

[PATCH v2 24/29] venus: vdec: get required input buffers as well

[Stanimir Varbanov]

Rework and rename vdec_cap_num_buffers() to get the number of
input buffers too.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/vdec.c | 41 +++++++++++++++++++-------------
 1 file changed, 24 insertions(+), 17 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 898c5edb91f5..5a5e3e2fece4 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -603,19 +603,32 @@ static int vdec_init_session(struct venus_inst *inst)
 	return ret;
 }
 
-static int vdec_cap_num_buffers(struct venus_inst *inst, unsigned int *num)
+static int vdec_num_buffers(struct venus_inst *inst, unsigned int *in_num,
+			    unsigned int *out_num)
 {
+	enum hfi_version ver = inst->core->res->hfi_version;
 	struct hfi_buffer_requirements bufreq;
 	int ret;
 
+	*in_num = *out_num = 0;
+
 	ret = vdec_init_session(inst);
 	if (ret)
 		return ret;
 
+	ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);
+	if (ret)
+		goto deinit;
+
+	*in_num = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
+
 	ret = venus_helper_get_bufreq(inst, HFI_BUFFER_OUTPUT, &bufreq);
+	if (ret)
+		goto deinit;
 
-	*num = bufreq.count_actual;
+	*out_num = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
 
+deinit:
 	hfi_session_deinit(inst);
 
 	return ret;
@@ -626,7 +639,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
 			    unsigned int sizes[], struct device *alloc_devs[])
 {
 	struct venus_inst *inst = vb2_get_drv_priv(q);
-	unsigned int p, num;
+	unsigned int p, in_num, out_num;
 	int ret = 0;
 
 	if (*num_planes) {
@@ -649,35 +662,29 @@ static int vdec_queue_setup(struct vb2_queue *q,
 		return 0;
 	}
 
+	ret = vdec_num_buffers(inst, &in_num, &out_num);
+	if (ret)
+		return ret;
+
 	switch (q->type) {
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
 		*num_planes = inst->fmt_out->num_planes;
 		sizes[0] = get_framesize_compressed(inst->out_width,
 						    inst->out_height);
 		inst->input_buf_size = sizes[0];
+		*num_buffers = max(*num_buffers, in_num);
 		inst->num_input_bufs = *num_buffers;
-
-		ret = vdec_cap_num_buffers(inst, &num);
-		if (ret)
-			break;
-
-		inst->num_output_bufs = num;
+		inst->num_output_bufs = out_num;
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
 		*num_planes = inst->fmt_cap->num_planes;
 
-		ret = vdec_cap_num_buffers(inst, &num);
-		if (ret)
-			break;
-
-		*num_buffers = max(*num_buffers, num);
-
 		for (p = 0; p < *num_planes; p++)
 			sizes[p] = get_framesize_uncompressed(p, inst->width,
 							      inst->height);
-
-		inst->num_output_bufs = *num_buffers;
 		inst->output_buf_size = sizes[0];
+		*num_buffers = max(*num_buffers, out_num);
+		inst->num_output_bufs = *num_buffers;
 		break;
 	default:
 		ret = -EINVAL;
-- 
2.14.1

[PATCH v2 25/29] venus: vdec: new function for output configuration

[Stanimir Varbanov]

Make a new function vdec_output_conf() for decoder output
configuration. vdec_output_conf() will set properties via
HFI interface related to the output configuration, and
keep vdec_set_properties() which will set properties
related to decoding parameters.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/vdec.c | 35 ++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 15 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 5a5e3e2fece4..3a699af0ab58 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -545,6 +545,23 @@ static const struct v4l2_ioctl_ops vdec_ioctl_ops = {
 static int vdec_set_properties(struct venus_inst *inst)
 {
 	struct vdec_controls *ctr = &inst->controls.dec;
+	struct hfi_enable en = { .enable = 1 };
+	u32 ptype;
+	int ret;
+
+	if (ctr->post_loop_deb_mode) {
+		ptype = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
+		en.enable = 1;
+		ret = hfi_session_set_property(inst, ptype, &en);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+static int vdec_output_conf(struct venus_inst *inst)
+{
 	struct venus_core *core = inst->core;
 	struct hfi_enable en = { .enable = 1 };
 	u32 ptype;
@@ -569,14 +586,6 @@ static int vdec_set_properties(struct venus_inst *inst)
 	if (ret)
 		return ret;
 
-	if (ctr->post_loop_deb_mode) {
-		ptype = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
-		en.enable = 1;
-		ret = hfi_session_set_property(inst, ptype, &en);
-		if (ret)
-			return ret;
-	}
-
 	return 0;
 }
 
@@ -724,7 +733,6 @@ static int vdec_verify_conf(struct venus_inst *inst)
 static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 {
 	struct venus_inst *inst = vb2_get_drv_priv(q);
-	struct venus_core *core = inst->core;
 	int ret;
 
 	mutex_lock(&inst->lock);
@@ -753,12 +761,9 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 	if (ret)
 		goto deinit_sess;
 
-	if (core->res->hfi_version == HFI_VERSION_3XX) {
-		ret = venus_helper_set_bufsize(inst, inst->output_buf_size,
-					       HFI_BUFFER_OUTPUT);
-		if (ret)
-			goto deinit_sess;
-	}
+	ret = vdec_output_conf(inst);
+	if (ret)
+		goto deinit_sess;
 
 	ret = vdec_verify_conf(inst);
 	if (ret)
-- 
2.14.1

Re: [PATCH v2 25/29] venus: vdec: new function for output configuration

[Tomasz Figa]

On Tue, May 15, 2018 at 5:01 PM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> Make a new function vdec_output_conf() for decoder output
> configuration. vdec_output_conf() will set properties via
> HFI interface related to the output configuration, and
> keep vdec_set_properties() which will set properties
> related to decoding parameters.
>
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/vdec.c | 35 ++++++++++++++++++--------------
>  1 file changed, 20 insertions(+), 15 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
> index 5a5e3e2fece4..3a699af0ab58 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -545,6 +545,23 @@ static const struct v4l2_ioctl_ops vdec_ioctl_ops = {
>  static int vdec_set_properties(struct venus_inst *inst)
>  {
>         struct vdec_controls *ctr = &inst->controls.dec;
> +       struct hfi_enable en = { .enable = 1 };
> +       u32 ptype;
> +       int ret;
> +
> +       if (ctr->post_loop_deb_mode) {
> +               ptype = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
> +               en.enable = 1;

en.enable was already set to 1 in the definition.

Best regards,
Tomasz

[PATCH v2 26/29] venus: move frame size calculations in common place

[Stanimir Varbanov]

move calculations of raw and compressed in a common helper
and make it identical for encoder and decoder.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 98 +++++++++++++++++++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  2 +
 drivers/media/platform/qcom/venus/vdec.c    | 54 ++++------------
 drivers/media/platform/qcom/venus/venc.c    | 56 ++++-------------
 4 files changed, 126 insertions(+), 84 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index f0a0fca60c76..ed569705ecac 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -455,6 +455,104 @@ int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
 }
 EXPORT_SYMBOL_GPL(venus_helper_get_bufreq);
 
+static u32 get_framesize_raw_nv12(u32 width, u32 height)
+{
+	u32 y_stride, uv_stride, y_plane;
+	u32 y_sclines, uv_sclines, uv_plane;
+	u32 size;
+
+	y_stride = ALIGN(width, 128);
+	uv_stride = ALIGN(width, 128);
+	y_sclines = ALIGN(height, 32);
+	uv_sclines = ALIGN(((height + 1) >> 1), 16);
+
+	y_plane = y_stride * y_sclines;
+	uv_plane = uv_stride * uv_sclines + SZ_4K;
+	size = y_plane + uv_plane + SZ_8K;
+
+	return ALIGN(size, SZ_4K);
+}
+
+static u32 get_framesize_raw_nv12_ubwc(u32 width, u32 height)
+{
+	u32 y_meta_stride, y_meta_plane;
+	u32 y_stride, y_plane;
+	u32 uv_meta_stride, uv_meta_plane;
+	u32 uv_stride, uv_plane;
+	u32 extradata = SZ_16K;
+
+	y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64);
+	y_meta_plane = y_meta_stride * ALIGN(DIV_ROUND_UP(height, 8), 16);
+	y_meta_plane = ALIGN(y_meta_plane, SZ_4K);
+
+	y_stride = ALIGN(width, 128);
+	y_plane = ALIGN(y_stride * ALIGN(height, 32), SZ_4K);
+
+	uv_meta_stride = ALIGN(DIV_ROUND_UP(width / 2, 16), 64);
+	uv_meta_plane = uv_meta_stride * ALIGN(DIV_ROUND_UP(height / 2, 8), 16);
+	uv_meta_plane = ALIGN(uv_meta_plane, SZ_4K);
+
+	uv_stride = ALIGN(width, 128);
+	uv_plane = ALIGN(uv_stride * ALIGN(height / 2, 32), SZ_4K);
+
+	return ALIGN(y_meta_plane + y_plane + uv_meta_plane + uv_plane +
+		     max(extradata, y_stride * 48), SZ_4K);
+}
+
+u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height)
+{
+	switch (hfi_fmt) {
+	case HFI_COLOR_FORMAT_NV12:
+	case HFI_COLOR_FORMAT_NV21:
+		return get_framesize_raw_nv12(width, height);
+	case HFI_COLOR_FORMAT_NV12_UBWC:
+		return get_framesize_raw_nv12_ubwc(width, height);
+	default:
+		return 0;
+	}
+}
+EXPORT_SYMBOL_GPL(venus_helper_get_framesz_raw);
+
+u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height)
+{
+	u32 hfi_fmt, sz;
+	bool compressed;
+
+	switch (v4l2_fmt) {
+	case V4L2_PIX_FMT_MPEG:
+	case V4L2_PIX_FMT_H264:
+	case V4L2_PIX_FMT_H264_NO_SC:
+	case V4L2_PIX_FMT_H264_MVC:
+	case V4L2_PIX_FMT_H263:
+	case V4L2_PIX_FMT_MPEG1:
+	case V4L2_PIX_FMT_MPEG2:
+	case V4L2_PIX_FMT_MPEG4:
+	case V4L2_PIX_FMT_XVID:
+	case V4L2_PIX_FMT_VC1_ANNEX_G:
+	case V4L2_PIX_FMT_VC1_ANNEX_L:
+	case V4L2_PIX_FMT_VP8:
+	case V4L2_PIX_FMT_VP9:
+	case V4L2_PIX_FMT_HEVC:
+		compressed = true;
+		break;
+	default:
+		compressed = false;
+		break;
+	}
+
+	if (compressed) {
+		sz = ALIGN(height, 32) * ALIGN(width, 32) * 3 / 2 / 2;
+		return ALIGN(sz, SZ_4K);
+	}
+
+	hfi_fmt = to_hfi_raw_fmt(v4l2_fmt);
+	if (!hfi_fmt)
+		return 0;
+
+	return venus_helper_get_framesz_raw(hfi_fmt, width, height);
+}
+EXPORT_SYMBOL_GPL(venus_helper_get_framesz);
+
 int venus_helper_set_input_resolution(struct venus_inst *inst,
 				      unsigned int width, unsigned int height)
 {
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 92be45894a69..92b167a47166 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -33,6 +33,8 @@ void venus_helper_m2m_device_run(void *priv);
 void venus_helper_m2m_job_abort(void *priv);
 int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
 			    struct hfi_buffer_requirements *req);
+u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height);
+u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height);
 int venus_helper_set_input_resolution(struct venus_inst *inst,
 				      unsigned int width, unsigned int height);
 int venus_helper_set_output_resolution(struct venus_inst *inst,
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 3a699af0ab58..28db28fb5f21 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -29,29 +29,6 @@
 #include "helpers.h"
 #include "vdec.h"
 
-static u32 get_framesize_uncompressed(unsigned int plane, u32 width, u32 height)
-{
-	u32 y_stride, uv_stride, y_plane;
-	u32 y_sclines, uv_sclines, uv_plane;
-	u32 size;
-
-	y_stride = ALIGN(width, 128);
-	uv_stride = ALIGN(width, 128);
-	y_sclines = ALIGN(height, 32);
-	uv_sclines = ALIGN(((height + 1) >> 1), 16);
-
-	y_plane = y_stride * y_sclines;
-	uv_plane = uv_stride * uv_sclines + SZ_4K;
-	size = y_plane + uv_plane + SZ_8K;
-
-	return ALIGN(size, SZ_4K);
-}
-
-static u32 get_framesize_compressed(unsigned int width, unsigned int height)
-{
-	return ((width * height * 3 / 2) / 2) + 128;
-}
-
 /*
  * Three resons to keep MPLANE formats (despite that the number of planes
  * currently is one):
@@ -160,7 +137,6 @@ vdec_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
 	struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
 	const struct venus_format *fmt;
-	unsigned int p;
 
 	memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
 	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
@@ -191,18 +167,14 @@ vdec_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 	pixmp->num_planes = fmt->num_planes;
 	pixmp->flags = 0;
 
-	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
-		for (p = 0; p < pixmp->num_planes; p++) {
-			pfmt[p].sizeimage =
-				get_framesize_uncompressed(p, pixmp->width,
-							   pixmp->height);
-			pfmt[p].bytesperline = ALIGN(pixmp->width, 128);
-		}
-	} else {
-		pfmt[0].sizeimage = get_framesize_compressed(pixmp->width,
-							     pixmp->height);
+	pfmt[0].sizeimage = venus_helper_get_framesz(pixmp->pixelformat,
+						     pixmp->width,
+						     pixmp->height);
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		pfmt[0].bytesperline = ALIGN(pixmp->width, 128);
+	else
 		pfmt[0].bytesperline = 0;
-	}
 
 	return fmt;
 }
@@ -648,7 +620,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
 			    unsigned int sizes[], struct device *alloc_devs[])
 {
 	struct venus_inst *inst = vb2_get_drv_priv(q);
-	unsigned int p, in_num, out_num;
+	unsigned int in_num, out_num;
 	int ret = 0;
 
 	if (*num_planes) {
@@ -678,7 +650,8 @@ static int vdec_queue_setup(struct vb2_queue *q,
 	switch (q->type) {
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
 		*num_planes = inst->fmt_out->num_planes;
-		sizes[0] = get_framesize_compressed(inst->out_width,
+		sizes[0] = venus_helper_get_framesz(inst->fmt_out->pixfmt,
+						    inst->out_width,
 						    inst->out_height);
 		inst->input_buf_size = sizes[0];
 		*num_buffers = max(*num_buffers, in_num);
@@ -687,10 +660,9 @@ static int vdec_queue_setup(struct vb2_queue *q,
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
 		*num_planes = inst->fmt_cap->num_planes;
-
-		for (p = 0; p < *num_planes; p++)
-			sizes[p] = get_framesize_uncompressed(p, inst->width,
-							      inst->height);
+		sizes[0] = venus_helper_get_framesz(inst->fmt_cap->pixfmt,
+						    inst->width,
+						    inst->height);
 		inst->output_buf_size = sizes[0];
 		*num_buffers = max(*num_buffers, out_num);
 		inst->num_output_bufs = *num_buffers;
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index c9c40d1ce7c6..54f253b98b24 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -31,32 +31,6 @@
 
 #define NUM_B_FRAMES_MAX	4
 
-static u32 get_framesize_uncompressed(unsigned int plane, u32 width, u32 height)
-{
-	u32 y_stride, uv_stride, y_plane;
-	u32 y_sclines, uv_sclines, uv_plane;
-	u32 size;
-
-	y_stride = ALIGN(width, 128);
-	uv_stride = ALIGN(width, 128);
-	y_sclines = ALIGN(height, 32);
-	uv_sclines = ALIGN(((height + 1) >> 1), 16);
-
-	y_plane = y_stride * y_sclines;
-	uv_plane = uv_stride * uv_sclines + SZ_4K;
-	size = y_plane + uv_plane + SZ_8K;
-	size = ALIGN(size, SZ_4K);
-
-	return size;
-}
-
-static u32 get_framesize_compressed(u32 width, u32 height)
-{
-	u32 sz = ALIGN(height, 32) * ALIGN(width, 32) * 3 / 2 / 2;
-
-	return ALIGN(sz, SZ_4K);
-}
-
 /*
  * Three resons to keep MPLANE formats (despite that the number of planes
  * currently is one):
@@ -284,7 +258,6 @@ venc_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
 	struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
 	const struct venus_format *fmt;
-	unsigned int p;
 
 	memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
 	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
@@ -318,19 +291,14 @@ venc_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 	pixmp->num_planes = fmt->num_planes;
 	pixmp->flags = 0;
 
-	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
-		for (p = 0; p < pixmp->num_planes; p++) {
-			pfmt[p].sizeimage =
-				get_framesize_uncompressed(p, pixmp->width,
-							   pixmp->height);
+	pfmt[0].sizeimage = venus_helper_get_framesz(pixmp->pixelformat,
+						     pixmp->width,
+						     pixmp->height);
 
-			pfmt[p].bytesperline = ALIGN(pixmp->width, 128);
-		}
-	} else {
-		pfmt[0].sizeimage = get_framesize_compressed(pixmp->width,
-							     pixmp->height);
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		pfmt[0].bytesperline = ALIGN(pixmp->width, 128);
+	else
 		pfmt[0].bytesperline = 0;
-	}
 
 	return fmt;
 }
@@ -845,7 +813,7 @@ static int venc_queue_setup(struct vb2_queue *q,
 			    unsigned int sizes[], struct device *alloc_devs[])
 {
 	struct venus_inst *inst = vb2_get_drv_priv(q);
-	unsigned int p, num, min = 4;
+	unsigned int num, min = 4;
 	int ret = 0;
 
 	if (*num_planes) {
@@ -880,16 +848,18 @@ static int venc_queue_setup(struct vb2_queue *q,
 		*num_buffers = max(*num_buffers, num);
 		inst->num_input_bufs = *num_buffers;
 
-		for (p = 0; p < *num_planes; ++p)
-			sizes[p] = get_framesize_uncompressed(p, inst->width,
-							      inst->height);
+		sizes[0] = venus_helper_get_framesz(inst->fmt_out->pixfmt,
+						    inst->width,
+						    inst->height);
 		inst->input_buf_size = sizes[0];
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
 		*num_planes = inst->fmt_cap->num_planes;
 		*num_buffers = max(*num_buffers, min);
 		inst->num_output_bufs = *num_buffers;
-		sizes[0] = get_framesize_compressed(inst->width, inst->height);
+		sizes[0] = venus_helper_get_framesz(inst->fmt_cap->pixfmt,
+						    inst->width,
+						    inst->height);
 		inst->output_buf_size = sizes[0];
 		break;
 	default:
-- 
2.14.1

[PATCH v2 27/29] venus: implementing multi-stream support

[Stanimir Varbanov]

This is implementing a multi-stream decoder support. The multi
stream gives an option to use the secondary decoder output
with different raw format (or the same in case of crop).

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h    |   1 +
 drivers/media/platform/qcom/venus/helpers.c | 204 +++++++++++++++++++++++++++-
 drivers/media/platform/qcom/venus/helpers.h |   6 +
 drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
 drivers/media/platform/qcom/venus/venc.c    |   1 +
 5 files changed, 299 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 4d6c05f156c4..85e66e2dd672 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -259,6 +259,7 @@ struct venus_inst {
 	struct list_head list;
 	struct mutex lock;
 	struct venus_core *core;
+	struct list_head dpbbufs;
 	struct list_head internalbufs;
 	struct list_head registeredbufs;
 	struct list_head delayed_process;
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index ed569705ecac..87dcf9973e6f 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -85,6 +85,112 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
 }
 EXPORT_SYMBOL_GPL(venus_helper_check_codec);
 
+static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
+{
+	struct intbuf *buf;
+	int ret = 0;
+
+	if (list_empty(&inst->dpbbufs))
+		return 0;
+
+	list_for_each_entry(buf, &inst->dpbbufs, list) {
+		struct hfi_frame_data fdata;
+
+		memset(&fdata, 0, sizeof(fdata));
+		fdata.alloc_len = buf->size;
+		fdata.device_addr = buf->da;
+		fdata.buffer_type = buf->type;
+
+		ret = hfi_session_process_buf(inst, &fdata);
+		if (ret)
+			goto fail;
+	}
+
+fail:
+	return ret;
+}
+
+int venus_helper_free_dpb_bufs(struct venus_inst *inst)
+{
+	struct intbuf *buf, *n;
+
+	if (list_empty(&inst->dpbbufs))
+		return 0;
+
+	list_for_each_entry_safe(buf, n, &inst->dpbbufs, list) {
+		list_del_init(&buf->list);
+		dma_free_attrs(inst->core->dev, buf->size, buf->va, buf->da,
+			       buf->attrs);
+		kfree(buf);
+	}
+
+	INIT_LIST_HEAD(&inst->dpbbufs);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(venus_helper_free_dpb_bufs);
+
+int venus_helper_alloc_dpb_bufs(struct venus_inst *inst)
+{
+	struct venus_core *core = inst->core;
+	struct device *dev = core->dev;
+	enum hfi_version ver = core->res->hfi_version;
+	struct hfi_buffer_requirements bufreq;
+	u32 buftype = inst->dpb_buftype;
+	unsigned int dpb_size = 0;
+	struct intbuf *buf;
+	unsigned int i;
+	u32 count;
+	int ret;
+
+	/* no need to allocate dpb buffers */
+	if (!inst->dpb_fmt)
+		return 0;
+
+	if (inst->dpb_buftype == HFI_BUFFER_OUTPUT)
+		dpb_size = inst->output_buf_size;
+	else if (inst->dpb_buftype == HFI_BUFFER_OUTPUT2)
+		dpb_size = inst->output2_buf_size;
+
+	if (!dpb_size)
+		return 0;
+
+	ret = venus_helper_get_bufreq(inst, buftype, &bufreq);
+	if (ret)
+		return ret;
+
+	count = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
+
+	for (i = 0; i < count; i++) {
+		buf = kzalloc(sizeof(*buf), GFP_KERNEL);
+		if (!buf) {
+			ret = -ENOMEM;
+			goto fail;
+		}
+
+		buf->type = buftype;
+		buf->size = dpb_size;
+		buf->attrs = DMA_ATTR_WRITE_COMBINE |
+			     DMA_ATTR_NO_KERNEL_MAPPING;
+		buf->va = dma_alloc_attrs(dev, buf->size, &buf->da, GFP_KERNEL,
+					  buf->attrs);
+		if (!buf->va) {
+			kfree(buf);
+			ret = -ENOMEM;
+			goto fail;
+		}
+
+		list_add_tail(&buf->list, &inst->dpbbufs);
+	}
+
+	return 0;
+
+fail:
+	venus_helper_free_dpb_bufs(inst);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(venus_helper_alloc_dpb_bufs);
+
 static int intbufs_set_buffer(struct venus_inst *inst, u32 type)
 {
 	struct venus_core *core = inst->core;
@@ -342,7 +448,10 @@ session_process_buf(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf)
 		if (vbuf->flags & V4L2_BUF_FLAG_LAST || !fdata.filled_len)
 			fdata.flags |= HFI_BUFFERFLAG_EOS;
 	} else if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
-		fdata.buffer_type = HFI_BUFFER_OUTPUT;
+		if (inst->session_type == VIDC_SESSION_TYPE_ENC)
+			fdata.buffer_type = HFI_BUFFER_OUTPUT;
+		else
+			fdata.buffer_type = inst->opb_buftype;
 		fdata.filled_len = 0;
 		fdata.offset = 0;
 	}
@@ -675,6 +784,27 @@ int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
 
+int venus_helper_set_multistream(struct venus_inst *inst, bool out_en,
+				 bool out2_en)
+{
+	struct hfi_multi_stream multi = {0};
+	u32 ptype = HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM;
+	int ret;
+
+	multi.buffer_type = HFI_BUFFER_OUTPUT;
+	multi.enable = out_en;
+
+	ret = hfi_session_set_property(inst, ptype, &multi);
+	if (ret)
+		return ret;
+
+	multi.buffer_type = HFI_BUFFER_OUTPUT2;
+	multi.enable = out2_en;
+
+	return hfi_session_set_property(inst, ptype, &multi);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_multistream);
+
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
 {
 	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
@@ -822,9 +952,10 @@ EXPORT_SYMBOL_GPL(venus_helper_vb2_buf_init);
 int venus_helper_vb2_buf_prepare(struct vb2_buffer *vb)
 {
 	struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+	unsigned int out_buf_size = venus_helper_get_opb_size(inst);
 
 	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
-	    vb2_plane_size(vb, 0) < inst->output_buf_size)
+	    vb2_plane_size(vb, 0) < out_buf_size)
 		return -EINVAL;
 	if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
 	    vb2_plane_size(vb, 0) < inst->input_buf_size)
@@ -894,6 +1025,8 @@ void venus_helper_vb2_stop_streaming(struct vb2_queue *q)
 		if (ret)
 			hfi_session_abort(inst);
 
+		venus_helper_free_dpb_bufs(inst);
+
 		load_scale_clocks(core);
 		INIT_LIST_HEAD(&inst->registeredbufs);
 	}
@@ -932,8 +1065,14 @@ int venus_helper_vb2_start_streaming(struct venus_inst *inst)
 	if (ret)
 		goto err_unload_res;
 
+	ret = venus_helper_queue_dpb_bufs(inst);
+	if (ret)
+		goto err_session_stop;
+
 	return 0;
 
+err_session_stop:
+	hfi_session_stop(inst);
 err_unload_res:
 	hfi_session_unload_res(inst);
 err_unreg_bufs:
@@ -987,6 +1126,67 @@ void venus_helper_init_instance(struct venus_inst *inst)
 }
 EXPORT_SYMBOL_GPL(venus_helper_init_instance);
 
+static bool find_fmt_from_caps(struct venus_caps *caps, u32 buftype, u32 fmt)
+{
+	unsigned int i;
+
+	for (i = 0; i < caps->num_fmts; i++) {
+		if (caps->fmts[i].buftype == buftype &&
+		    caps->fmts[i].fmt == fmt)
+			return true;
+	}
+
+	return false;
+}
+
+int venus_helper_get_out_fmts(struct venus_inst *inst, u32 v4l2_fmt,
+			      u32 *out_fmt, u32 *out2_fmt, bool ubwc)
+{
+	struct venus_core *core = inst->core;
+	struct venus_caps *caps;
+	u32 ubwc_fmt, fmt = to_hfi_raw_fmt(v4l2_fmt);
+	bool found, found_ubwc;
+
+	*out_fmt = *out2_fmt = 0;
+
+	if (!fmt)
+		return -EINVAL;
+
+	caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
+	if (!caps)
+		return -EINVAL;
+
+	if (ubwc) {
+		ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE;
+		found_ubwc = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT,
+						ubwc_fmt);
+		found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
+
+		if (found_ubwc && found) {
+			*out_fmt = ubwc_fmt;
+			*out2_fmt = fmt;
+			return 0;
+		}
+	}
+
+	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT, fmt);
+	if (found) {
+		*out_fmt = fmt;
+		*out2_fmt = 0;
+		return 0;
+	}
+
+	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
+	if (found) {
+		*out_fmt = 0;
+		*out2_fmt = fmt;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+EXPORT_SYMBOL_GPL(venus_helper_get_out_fmts);
+
 int venus_helper_power_enable(struct venus_core *core, u32 session_type,
 			      bool enable)
 {
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 92b167a47166..2475f284f396 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -50,10 +50,16 @@ int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
 int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype);
+int venus_helper_set_multistream(struct venus_inst *inst, bool out_en,
+				 bool out2_en);
 unsigned int venus_helper_get_opb_size(struct venus_inst *inst);
 void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
 void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
 void venus_helper_init_instance(struct venus_inst *inst);
+int venus_helper_get_out_fmts(struct venus_inst *inst, u32 fmt, u32 *out_fmt,
+			      u32 *out2_fmt, bool ubwc);
+int venus_helper_alloc_dpb_bufs(struct venus_inst *inst);
+int venus_helper_free_dpb_bufs(struct venus_inst *inst);
 int venus_helper_power_enable(struct venus_core *core, u32 session_type,
 			      bool enable);
 #endif
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 28db28fb5f21..d2d082009686 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -532,10 +532,16 @@ static int vdec_set_properties(struct venus_inst *inst)
 	return 0;
 }
 
+#define is_ubwc_fmt(fmt) (!!((fmt) & HFI_COLOR_FORMAT_UBWC_BASE))
+
 static int vdec_output_conf(struct venus_inst *inst)
 {
 	struct venus_core *core = inst->core;
 	struct hfi_enable en = { .enable = 1 };
+	u32 width = inst->out_width;
+	u32 height = inst->out_height;
+	u32 out_fmt, out2_fmt;
+	bool ubwc = false;
 	u32 ptype;
 	int ret;
 
@@ -554,6 +560,78 @@ static int vdec_output_conf(struct venus_inst *inst)
 			return ret;
 	}
 
+	if (width > 1920 && height > ALIGN(1080, 32))
+		ubwc = true;
+
+	if (IS_V4(core))
+		ubwc = true;
+
+	ret = venus_helper_get_out_fmts(inst, inst->fmt_cap->pixfmt, &out_fmt,
+					&out2_fmt, ubwc);
+	if (ret)
+		return ret;
+
+	inst->output_buf_size =
+			venus_helper_get_framesz_raw(out_fmt, width, height);
+	inst->output2_buf_size =
+			venus_helper_get_framesz_raw(out2_fmt, width, height);
+
+	if (is_ubwc_fmt(out_fmt)) {
+		inst->opb_buftype = HFI_BUFFER_OUTPUT2;
+		inst->opb_fmt = out2_fmt;
+		inst->dpb_buftype = HFI_BUFFER_OUTPUT;
+		inst->dpb_fmt = out_fmt;
+	} else if (is_ubwc_fmt(out2_fmt)) {
+		inst->opb_buftype = HFI_BUFFER_OUTPUT;
+		inst->opb_fmt = out_fmt;
+		inst->dpb_buftype = HFI_BUFFER_OUTPUT2;
+		inst->dpb_fmt = out2_fmt;
+	} else {
+		inst->opb_buftype = HFI_BUFFER_OUTPUT;
+		inst->opb_fmt = out_fmt;
+		inst->dpb_buftype = 0;
+		inst->dpb_fmt = 0;
+	}
+
+	ret = venus_helper_set_raw_format(inst, inst->opb_fmt,
+					  inst->opb_buftype);
+	if (ret)
+		return ret;
+
+	if (inst->dpb_fmt) {
+		ret = venus_helper_set_multistream(inst, false, true);
+		if (ret)
+			return ret;
+
+		ret = venus_helper_set_raw_format(inst, inst->dpb_fmt,
+						  inst->dpb_buftype);
+		if (ret)
+			return ret;
+
+		ret = venus_helper_set_output_resolution(inst, width, height,
+							 HFI_BUFFER_OUTPUT2);
+		if (ret)
+			return ret;
+	}
+
+	if (IS_V3(core) || IS_V4(core)) {
+		if (inst->output2_buf_size) {
+			ret = venus_helper_set_bufsize(inst,
+						       inst->output2_buf_size,
+						       HFI_BUFFER_OUTPUT2);
+			if (ret)
+				return ret;
+		}
+
+		if (inst->output_buf_size) {
+			ret = venus_helper_set_bufsize(inst,
+						       inst->output_buf_size,
+						       HFI_BUFFER_OUTPUT);
+			if (ret)
+				return ret;
+		}
+	}
+
 	ret = venus_helper_set_dyn_bufmode(inst);
 	if (ret)
 		return ret;
@@ -624,6 +702,8 @@ static int vdec_queue_setup(struct vb2_queue *q,
 	int ret = 0;
 
 	if (*num_planes) {
+		unsigned int output_buf_size = venus_helper_get_opb_size(inst);
+
 		if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
 		    *num_planes != inst->fmt_out->num_planes)
 			return -EINVAL;
@@ -637,7 +717,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
 			return -EINVAL;
 
 		if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
-		    sizes[0] < inst->output_buf_size)
+		    sizes[0] < output_buf_size)
 			return -EINVAL;
 
 		return 0;
@@ -746,6 +826,10 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 	if (ret)
 		goto deinit_sess;
 
+	ret = venus_helper_alloc_dpb_bufs(inst);
+	if (ret)
+		goto deinit_sess;
+
 	ret = venus_helper_vb2_start_streaming(inst);
 	if (ret)
 		goto deinit_sess;
@@ -797,9 +881,11 @@ static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type,
 	vbuf->field = V4L2_FIELD_NONE;
 
 	if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		unsigned int opb_sz = venus_helper_get_opb_size(inst);
+
 		vb = &vbuf->vb2_buf;
 		vb->planes[0].bytesused =
-			max_t(unsigned int, inst->output_buf_size, bytesused);
+			max_t(unsigned int, opb_sz, bytesused);
 		vb->planes[0].data_offset = data_offset;
 		vb->timestamp = timestamp_us * NSEC_PER_USEC;
 		vbuf->sequence = inst->sequence_cap++;
@@ -937,6 +1023,7 @@ static int vdec_open(struct file *file)
 	if (!inst)
 		return -ENOMEM;
 
+	INIT_LIST_HEAD(&inst->dpbbufs);
 	INIT_LIST_HEAD(&inst->registeredbufs);
 	INIT_LIST_HEAD(&inst->internalbufs);
 	INIT_LIST_HEAD(&inst->list);
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 54f253b98b24..a703bce78abc 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -1084,6 +1084,7 @@ static int venc_open(struct file *file)
 	if (!inst)
 		return -ENOMEM;
 
+	INIT_LIST_HEAD(&inst->dpbbufs);
 	INIT_LIST_HEAD(&inst->registeredbufs);
 	INIT_LIST_HEAD(&inst->internalbufs);
 	INIT_LIST_HEAD(&inst->list);
-- 
2.14.1

Re: [PATCH v2 27/29] venus: implementing multi-stream support

[Hans Verkuil]

Hi Stanimir,

On 05/15/18 09:58, Stanimir Varbanov wrote:
> This is implementing a multi-stream decoder support. The multi
> stream gives an option to use the secondary decoder output
> with different raw format (or the same in case of crop).

You told me that multi-stream support is currently internal only.

It would be good if you could elaborate a bit about that in this
commit log and (I think) also add some comments in the code that
reflect this.

It's also not clear to me how and why this is used in the driver
if this is just internal. Does it enable a feature you would
otherwise not be able to use?

I have no complaints about the code, I would just like to see a
bit more background information in the source and commit log.

Regards,

	Hans

> 
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/core.h    |   1 +
>  drivers/media/platform/qcom/venus/helpers.c | 204 +++++++++++++++++++++++++++-
>  drivers/media/platform/qcom/venus/helpers.h |   6 +
>  drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
>  drivers/media/platform/qcom/venus/venc.c    |   1 +
>  5 files changed, 299 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
> index 4d6c05f156c4..85e66e2dd672 100644
> --- a/drivers/media/platform/qcom/venus/core.h
> +++ b/drivers/media/platform/qcom/venus/core.h
> @@ -259,6 +259,7 @@ struct venus_inst {
>  	struct list_head list;
>  	struct mutex lock;
>  	struct venus_core *core;
> +	struct list_head dpbbufs;
>  	struct list_head internalbufs;
>  	struct list_head registeredbufs;
>  	struct list_head delayed_process;
> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
> index ed569705ecac..87dcf9973e6f 100644
> --- a/drivers/media/platform/qcom/venus/helpers.c
> +++ b/drivers/media/platform/qcom/venus/helpers.c
> @@ -85,6 +85,112 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
>  }
>  EXPORT_SYMBOL_GPL(venus_helper_check_codec);
>  
> +static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
> +{
> +	struct intbuf *buf;
> +	int ret = 0;
> +
> +	if (list_empty(&inst->dpbbufs))
> +		return 0;
> +
> +	list_for_each_entry(buf, &inst->dpbbufs, list) {
> +		struct hfi_frame_data fdata;
> +
> +		memset(&fdata, 0, sizeof(fdata));
> +		fdata.alloc_len = buf->size;
> +		fdata.device_addr = buf->da;
> +		fdata.buffer_type = buf->type;
> +
> +		ret = hfi_session_process_buf(inst, &fdata);
> +		if (ret)
> +			goto fail;
> +	}
> +
> +fail:
> +	return ret;
> +}
> +
> +int venus_helper_free_dpb_bufs(struct venus_inst *inst)
> +{
> +	struct intbuf *buf, *n;
> +
> +	if (list_empty(&inst->dpbbufs))
> +		return 0;
> +
> +	list_for_each_entry_safe(buf, n, &inst->dpbbufs, list) {
> +		list_del_init(&buf->list);
> +		dma_free_attrs(inst->core->dev, buf->size, buf->va, buf->da,
> +			       buf->attrs);
> +		kfree(buf);
> +	}
> +
> +	INIT_LIST_HEAD(&inst->dpbbufs);
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(venus_helper_free_dpb_bufs);
> +
> +int venus_helper_alloc_dpb_bufs(struct venus_inst *inst)
> +{
> +	struct venus_core *core = inst->core;
> +	struct device *dev = core->dev;
> +	enum hfi_version ver = core->res->hfi_version;
> +	struct hfi_buffer_requirements bufreq;
> +	u32 buftype = inst->dpb_buftype;
> +	unsigned int dpb_size = 0;
> +	struct intbuf *buf;
> +	unsigned int i;
> +	u32 count;
> +	int ret;
> +
> +	/* no need to allocate dpb buffers */
> +	if (!inst->dpb_fmt)
> +		return 0;
> +
> +	if (inst->dpb_buftype == HFI_BUFFER_OUTPUT)
> +		dpb_size = inst->output_buf_size;
> +	else if (inst->dpb_buftype == HFI_BUFFER_OUTPUT2)
> +		dpb_size = inst->output2_buf_size;
> +
> +	if (!dpb_size)
> +		return 0;
> +
> +	ret = venus_helper_get_bufreq(inst, buftype, &bufreq);
> +	if (ret)
> +		return ret;
> +
> +	count = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
> +
> +	for (i = 0; i < count; i++) {
> +		buf = kzalloc(sizeof(*buf), GFP_KERNEL);
> +		if (!buf) {
> +			ret = -ENOMEM;
> +			goto fail;
> +		}
> +
> +		buf->type = buftype;
> +		buf->size = dpb_size;
> +		buf->attrs = DMA_ATTR_WRITE_COMBINE |
> +			     DMA_ATTR_NO_KERNEL_MAPPING;
> +		buf->va = dma_alloc_attrs(dev, buf->size, &buf->da, GFP_KERNEL,
> +					  buf->attrs);
> +		if (!buf->va) {
> +			kfree(buf);
> +			ret = -ENOMEM;
> +			goto fail;
> +		}
> +
> +		list_add_tail(&buf->list, &inst->dpbbufs);
> +	}
> +
> +	return 0;
> +
> +fail:
> +	venus_helper_free_dpb_bufs(inst);
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(venus_helper_alloc_dpb_bufs);
> +
>  static int intbufs_set_buffer(struct venus_inst *inst, u32 type)
>  {
>  	struct venus_core *core = inst->core;
> @@ -342,7 +448,10 @@ session_process_buf(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf)
>  		if (vbuf->flags & V4L2_BUF_FLAG_LAST || !fdata.filled_len)
>  			fdata.flags |= HFI_BUFFERFLAG_EOS;
>  	} else if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> -		fdata.buffer_type = HFI_BUFFER_OUTPUT;
> +		if (inst->session_type == VIDC_SESSION_TYPE_ENC)
> +			fdata.buffer_type = HFI_BUFFER_OUTPUT;
> +		else
> +			fdata.buffer_type = inst->opb_buftype;
>  		fdata.filled_len = 0;
>  		fdata.offset = 0;
>  	}
> @@ -675,6 +784,27 @@ int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
>  }
>  EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
>  
> +int venus_helper_set_multistream(struct venus_inst *inst, bool out_en,
> +				 bool out2_en)
> +{
> +	struct hfi_multi_stream multi = {0};
> +	u32 ptype = HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM;
> +	int ret;
> +
> +	multi.buffer_type = HFI_BUFFER_OUTPUT;
> +	multi.enable = out_en;
> +
> +	ret = hfi_session_set_property(inst, ptype, &multi);
> +	if (ret)
> +		return ret;
> +
> +	multi.buffer_type = HFI_BUFFER_OUTPUT2;
> +	multi.enable = out2_en;
> +
> +	return hfi_session_set_property(inst, ptype, &multi);
> +}
> +EXPORT_SYMBOL_GPL(venus_helper_set_multistream);
> +
>  int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
>  {
>  	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
> @@ -822,9 +952,10 @@ EXPORT_SYMBOL_GPL(venus_helper_vb2_buf_init);
>  int venus_helper_vb2_buf_prepare(struct vb2_buffer *vb)
>  {
>  	struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
> +	unsigned int out_buf_size = venus_helper_get_opb_size(inst);
>  
>  	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> -	    vb2_plane_size(vb, 0) < inst->output_buf_size)
> +	    vb2_plane_size(vb, 0) < out_buf_size)
>  		return -EINVAL;
>  	if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
>  	    vb2_plane_size(vb, 0) < inst->input_buf_size)
> @@ -894,6 +1025,8 @@ void venus_helper_vb2_stop_streaming(struct vb2_queue *q)
>  		if (ret)
>  			hfi_session_abort(inst);
>  
> +		venus_helper_free_dpb_bufs(inst);
> +
>  		load_scale_clocks(core);
>  		INIT_LIST_HEAD(&inst->registeredbufs);
>  	}
> @@ -932,8 +1065,14 @@ int venus_helper_vb2_start_streaming(struct venus_inst *inst)
>  	if (ret)
>  		goto err_unload_res;
>  
> +	ret = venus_helper_queue_dpb_bufs(inst);
> +	if (ret)
> +		goto err_session_stop;
> +
>  	return 0;
>  
> +err_session_stop:
> +	hfi_session_stop(inst);
>  err_unload_res:
>  	hfi_session_unload_res(inst);
>  err_unreg_bufs:
> @@ -987,6 +1126,67 @@ void venus_helper_init_instance(struct venus_inst *inst)
>  }
>  EXPORT_SYMBOL_GPL(venus_helper_init_instance);
>  
> +static bool find_fmt_from_caps(struct venus_caps *caps, u32 buftype, u32 fmt)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < caps->num_fmts; i++) {
> +		if (caps->fmts[i].buftype == buftype &&
> +		    caps->fmts[i].fmt == fmt)
> +			return true;
> +	}
> +
> +	return false;
> +}
> +
> +int venus_helper_get_out_fmts(struct venus_inst *inst, u32 v4l2_fmt,
> +			      u32 *out_fmt, u32 *out2_fmt, bool ubwc)
> +{
> +	struct venus_core *core = inst->core;
> +	struct venus_caps *caps;
> +	u32 ubwc_fmt, fmt = to_hfi_raw_fmt(v4l2_fmt);
> +	bool found, found_ubwc;
> +
> +	*out_fmt = *out2_fmt = 0;
> +
> +	if (!fmt)
> +		return -EINVAL;
> +
> +	caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
> +	if (!caps)
> +		return -EINVAL;
> +
> +	if (ubwc) {
> +		ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE;
> +		found_ubwc = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT,
> +						ubwc_fmt);
> +		found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
> +
> +		if (found_ubwc && found) {
> +			*out_fmt = ubwc_fmt;
> +			*out2_fmt = fmt;
> +			return 0;
> +		}
> +	}
> +
> +	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT, fmt);
> +	if (found) {
> +		*out_fmt = fmt;
> +		*out2_fmt = 0;
> +		return 0;
> +	}
> +
> +	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
> +	if (found) {
> +		*out_fmt = 0;
> +		*out2_fmt = fmt;
> +		return 0;
> +	}
> +
> +	return -EINVAL;
> +}
> +EXPORT_SYMBOL_GPL(venus_helper_get_out_fmts);
> +
>  int venus_helper_power_enable(struct venus_core *core, u32 session_type,
>  			      bool enable)
>  {
> diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
> index 92b167a47166..2475f284f396 100644
> --- a/drivers/media/platform/qcom/venus/helpers.h
> +++ b/drivers/media/platform/qcom/venus/helpers.h
> @@ -50,10 +50,16 @@ int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
>  int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
>  int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
>  int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype);
> +int venus_helper_set_multistream(struct venus_inst *inst, bool out_en,
> +				 bool out2_en);
>  unsigned int venus_helper_get_opb_size(struct venus_inst *inst);
>  void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
>  void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
>  void venus_helper_init_instance(struct venus_inst *inst);
> +int venus_helper_get_out_fmts(struct venus_inst *inst, u32 fmt, u32 *out_fmt,
> +			      u32 *out2_fmt, bool ubwc);
> +int venus_helper_alloc_dpb_bufs(struct venus_inst *inst);
> +int venus_helper_free_dpb_bufs(struct venus_inst *inst);
>  int venus_helper_power_enable(struct venus_core *core, u32 session_type,
>  			      bool enable);
>  #endif
> diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
> index 28db28fb5f21..d2d082009686 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -532,10 +532,16 @@ static int vdec_set_properties(struct venus_inst *inst)
>  	return 0;
>  }
>  
> +#define is_ubwc_fmt(fmt) (!!((fmt) & HFI_COLOR_FORMAT_UBWC_BASE))
> +
>  static int vdec_output_conf(struct venus_inst *inst)
>  {
>  	struct venus_core *core = inst->core;
>  	struct hfi_enable en = { .enable = 1 };
> +	u32 width = inst->out_width;
> +	u32 height = inst->out_height;
> +	u32 out_fmt, out2_fmt;
> +	bool ubwc = false;
>  	u32 ptype;
>  	int ret;
>  
> @@ -554,6 +560,78 @@ static int vdec_output_conf(struct venus_inst *inst)
>  			return ret;
>  	}
>  
> +	if (width > 1920 && height > ALIGN(1080, 32))
> +		ubwc = true;
> +
> +	if (IS_V4(core))
> +		ubwc = true;
> +
> +	ret = venus_helper_get_out_fmts(inst, inst->fmt_cap->pixfmt, &out_fmt,
> +					&out2_fmt, ubwc);
> +	if (ret)
> +		return ret;
> +
> +	inst->output_buf_size =
> +			venus_helper_get_framesz_raw(out_fmt, width, height);
> +	inst->output2_buf_size =
> +			venus_helper_get_framesz_raw(out2_fmt, width, height);
> +
> +	if (is_ubwc_fmt(out_fmt)) {
> +		inst->opb_buftype = HFI_BUFFER_OUTPUT2;
> +		inst->opb_fmt = out2_fmt;
> +		inst->dpb_buftype = HFI_BUFFER_OUTPUT;
> +		inst->dpb_fmt = out_fmt;
> +	} else if (is_ubwc_fmt(out2_fmt)) {
> +		inst->opb_buftype = HFI_BUFFER_OUTPUT;
> +		inst->opb_fmt = out_fmt;
> +		inst->dpb_buftype = HFI_BUFFER_OUTPUT2;
> +		inst->dpb_fmt = out2_fmt;
> +	} else {
> +		inst->opb_buftype = HFI_BUFFER_OUTPUT;
> +		inst->opb_fmt = out_fmt;
> +		inst->dpb_buftype = 0;
> +		inst->dpb_fmt = 0;
> +	}
> +
> +	ret = venus_helper_set_raw_format(inst, inst->opb_fmt,
> +					  inst->opb_buftype);
> +	if (ret)
> +		return ret;
> +
> +	if (inst->dpb_fmt) {
> +		ret = venus_helper_set_multistream(inst, false, true);
> +		if (ret)
> +			return ret;
> +
> +		ret = venus_helper_set_raw_format(inst, inst->dpb_fmt,
> +						  inst->dpb_buftype);
> +		if (ret)
> +			return ret;
> +
> +		ret = venus_helper_set_output_resolution(inst, width, height,
> +							 HFI_BUFFER_OUTPUT2);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	if (IS_V3(core) || IS_V4(core)) {
> +		if (inst->output2_buf_size) {
> +			ret = venus_helper_set_bufsize(inst,
> +						       inst->output2_buf_size,
> +						       HFI_BUFFER_OUTPUT2);
> +			if (ret)
> +				return ret;
> +		}
> +
> +		if (inst->output_buf_size) {
> +			ret = venus_helper_set_bufsize(inst,
> +						       inst->output_buf_size,
> +						       HFI_BUFFER_OUTPUT);
> +			if (ret)
> +				return ret;
> +		}
> +	}
> +
>  	ret = venus_helper_set_dyn_bufmode(inst);
>  	if (ret)
>  		return ret;
> @@ -624,6 +702,8 @@ static int vdec_queue_setup(struct vb2_queue *q,
>  	int ret = 0;
>  
>  	if (*num_planes) {
> +		unsigned int output_buf_size = venus_helper_get_opb_size(inst);
> +
>  		if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
>  		    *num_planes != inst->fmt_out->num_planes)
>  			return -EINVAL;
> @@ -637,7 +717,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
>  			return -EINVAL;
>  
>  		if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> -		    sizes[0] < inst->output_buf_size)
> +		    sizes[0] < output_buf_size)
>  			return -EINVAL;
>  
>  		return 0;
> @@ -746,6 +826,10 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
>  	if (ret)
>  		goto deinit_sess;
>  
> +	ret = venus_helper_alloc_dpb_bufs(inst);
> +	if (ret)
> +		goto deinit_sess;
> +
>  	ret = venus_helper_vb2_start_streaming(inst);
>  	if (ret)
>  		goto deinit_sess;
> @@ -797,9 +881,11 @@ static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type,
>  	vbuf->field = V4L2_FIELD_NONE;
>  
>  	if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		unsigned int opb_sz = venus_helper_get_opb_size(inst);
> +
>  		vb = &vbuf->vb2_buf;
>  		vb->planes[0].bytesused =
> -			max_t(unsigned int, inst->output_buf_size, bytesused);
> +			max_t(unsigned int, opb_sz, bytesused);
>  		vb->planes[0].data_offset = data_offset;
>  		vb->timestamp = timestamp_us * NSEC_PER_USEC;
>  		vbuf->sequence = inst->sequence_cap++;
> @@ -937,6 +1023,7 @@ static int vdec_open(struct file *file)
>  	if (!inst)
>  		return -ENOMEM;
>  
> +	INIT_LIST_HEAD(&inst->dpbbufs);
>  	INIT_LIST_HEAD(&inst->registeredbufs);
>  	INIT_LIST_HEAD(&inst->internalbufs);
>  	INIT_LIST_HEAD(&inst->list);
> diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
> index 54f253b98b24..a703bce78abc 100644
> --- a/drivers/media/platform/qcom/venus/venc.c
> +++ b/drivers/media/platform/qcom/venus/venc.c
> @@ -1084,6 +1084,7 @@ static int venc_open(struct file *file)
>  	if (!inst)
>  		return -ENOMEM;
>  
> +	INIT_LIST_HEAD(&inst->dpbbufs);
>  	INIT_LIST_HEAD(&inst->registeredbufs);
>  	INIT_LIST_HEAD(&inst->internalbufs);
>  	INIT_LIST_HEAD(&inst->list);
> 

Re: [PATCH v2 27/29] venus: implementing multi-stream support

[Stanimir Varbanov]

Hi Hans,

On 05/15/2018 11:17 AM, Hans Verkuil wrote:
> Hi Stanimir,
> 
> On 05/15/18 09:58, Stanimir Varbanov wrote:
>> This is implementing a multi-stream decoder support. The multi
>> stream gives an option to use the secondary decoder output
>> with different raw format (or the same in case of crop).
> 
> You told me that multi-stream support is currently internal only.
> 
> It would be good if you could elaborate a bit about that in this
> commit log and (I think) also add some comments in the code that
> reflect this.
> 
> It's also not clear to me how and why this is used in the driver
> if this is just internal. Does it enable a feature you would
> otherwise not be able to use?
> 
> I have no complaints about the code, I would just like to see a
> bit more background information in the source and commit log.

Thanks for the fast comments! Sure I will try to document multi-stream
support in the patch description and in the code if it makes sense.

-- 
regards,
Stan

[PATCH v3 27/29] venus: implementing multi-stream support

[Stanimir Varbanov]

This is implementing multi-stream decoder support. The multi-stream
will be used to enable/disable the primary/secondary decoder
outputs. Depending on formats on both decoder outputs we could
implement downscale, dithering and supporting UBWC (universal
bandwidth compression) formats. The UBWC compressed raw format is
used to optimize interconnect bandwidth for bigger resolutions
like 4K and hence we will get some power-saving benefits as well.

Both decoder outputs are distinguished by buffer_type field in
the HFI packets. For example HFI_BUFFER_OUTPUT is the buffer type
for primary decoder output and HFI_BUFFER_OUTPUT2 is for secondary
decoder output.

Starting from Venus 4xx the DPB buffers format must be UBWC, so
the multi-stream becomes mandatory for this Venus version. That
means that we need to allocate internally in the driver a set of
DPB buffers (with UBWC NV12 format) and give them to the firmware.
The other decoder output (we called it OPB) format will be NV12
linear format and with the same resolution (or smaller in case
the user wants to downscale).

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---

Hi Hans, I have updated patch description a bit. Is it now clearer?

If you have some particular questions I could try to answer and
extend the description.

Also, as Tomasz Figa started to review the patchset I'll postpone
patchset v3 a bit.

 drivers/media/platform/qcom/venus/core.h    |   2 +
 drivers/media/platform/qcom/venus/helpers.c | 204 +++++++++++++++++++++++++++-
 drivers/media/platform/qcom/venus/helpers.h |   6 +
 drivers/media/platform/qcom/venus/vdec.c    |  93 ++++++++++++-
 drivers/media/platform/qcom/venus/venc.c    |   1 +
 5 files changed, 302 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 1cac8dc79cd1..afa4e49e0e06 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -210,6 +210,7 @@ struct venus_buffer {
  * @list:	used for attach an instance to the core
  * @lock:	instance lock
  * @core:	a reference to the core struct
+ * @dpbbufs:	a list of decoded picture buffers
  * @internalbufs:	a list of internal bufferes
  * @registeredbufs:	a list of registered capture bufferes
  * @delayed_process	a list of delayed buffers
@@ -259,6 +260,7 @@ struct venus_inst {
 	struct list_head list;
 	struct mutex lock;
 	struct venus_core *core;
+	struct list_head dpbbufs;
 	struct list_head internalbufs;
 	struct list_head registeredbufs;
 	struct list_head delayed_process;
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index ed569705ecac..87dcf9973e6f 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -85,6 +85,112 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
 }
 EXPORT_SYMBOL_GPL(venus_helper_check_codec);
 
+static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
+{
+	struct intbuf *buf;
+	int ret = 0;
+
+	if (list_empty(&inst->dpbbufs))
+		return 0;
+
+	list_for_each_entry(buf, &inst->dpbbufs, list) {
+		struct hfi_frame_data fdata;
+
+		memset(&fdata, 0, sizeof(fdata));
+		fdata.alloc_len = buf->size;
+		fdata.device_addr = buf->da;
+		fdata.buffer_type = buf->type;
+
+		ret = hfi_session_process_buf(inst, &fdata);
+		if (ret)
+			goto fail;
+	}
+
+fail:
+	return ret;
+}
+
+int venus_helper_free_dpb_bufs(struct venus_inst *inst)
+{
+	struct intbuf *buf, *n;
+
+	if (list_empty(&inst->dpbbufs))
+		return 0;
+
+	list_for_each_entry_safe(buf, n, &inst->dpbbufs, list) {
+		list_del_init(&buf->list);
+		dma_free_attrs(inst->core->dev, buf->size, buf->va, buf->da,
+			       buf->attrs);
+		kfree(buf);
+	}
+
+	INIT_LIST_HEAD(&inst->dpbbufs);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(venus_helper_free_dpb_bufs);
+
+int venus_helper_alloc_dpb_bufs(struct venus_inst *inst)
+{
+	struct venus_core *core = inst->core;
+	struct device *dev = core->dev;
+	enum hfi_version ver = core->res->hfi_version;
+	struct hfi_buffer_requirements bufreq;
+	u32 buftype = inst->dpb_buftype;
+	unsigned int dpb_size = 0;
+	struct intbuf *buf;
+	unsigned int i;
+	u32 count;
+	int ret;
+
+	/* no need to allocate dpb buffers */
+	if (!inst->dpb_fmt)
+		return 0;
+
+	if (inst->dpb_buftype == HFI_BUFFER_OUTPUT)
+		dpb_size = inst->output_buf_size;
+	else if (inst->dpb_buftype == HFI_BUFFER_OUTPUT2)
+		dpb_size = inst->output2_buf_size;
+
+	if (!dpb_size)
+		return 0;
+
+	ret = venus_helper_get_bufreq(inst, buftype, &bufreq);
+	if (ret)
+		return ret;
+
+	count = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
+
+	for (i = 0; i < count; i++) {
+		buf = kzalloc(sizeof(*buf), GFP_KERNEL);
+		if (!buf) {
+			ret = -ENOMEM;
+			goto fail;
+		}
+
+		buf->type = buftype;
+		buf->size = dpb_size;
+		buf->attrs = DMA_ATTR_WRITE_COMBINE |
+			     DMA_ATTR_NO_KERNEL_MAPPING;
+		buf->va = dma_alloc_attrs(dev, buf->size, &buf->da, GFP_KERNEL,
+					  buf->attrs);
+		if (!buf->va) {
+			kfree(buf);
+			ret = -ENOMEM;
+			goto fail;
+		}
+
+		list_add_tail(&buf->list, &inst->dpbbufs);
+	}
+
+	return 0;
+
+fail:
+	venus_helper_free_dpb_bufs(inst);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(venus_helper_alloc_dpb_bufs);
+
 static int intbufs_set_buffer(struct venus_inst *inst, u32 type)
 {
 	struct venus_core *core = inst->core;
@@ -342,7 +448,10 @@ session_process_buf(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf)
 		if (vbuf->flags & V4L2_BUF_FLAG_LAST || !fdata.filled_len)
 			fdata.flags |= HFI_BUFFERFLAG_EOS;
 	} else if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
-		fdata.buffer_type = HFI_BUFFER_OUTPUT;
+		if (inst->session_type == VIDC_SESSION_TYPE_ENC)
+			fdata.buffer_type = HFI_BUFFER_OUTPUT;
+		else
+			fdata.buffer_type = inst->opb_buftype;
 		fdata.filled_len = 0;
 		fdata.offset = 0;
 	}
@@ -675,6 +784,27 @@ int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
 
+int venus_helper_set_multistream(struct venus_inst *inst, bool out_en,
+				 bool out2_en)
+{
+	struct hfi_multi_stream multi = {0};
+	u32 ptype = HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM;
+	int ret;
+
+	multi.buffer_type = HFI_BUFFER_OUTPUT;
+	multi.enable = out_en;
+
+	ret = hfi_session_set_property(inst, ptype, &multi);
+	if (ret)
+		return ret;
+
+	multi.buffer_type = HFI_BUFFER_OUTPUT2;
+	multi.enable = out2_en;
+
+	return hfi_session_set_property(inst, ptype, &multi);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_multistream);
+
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
 {
 	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
@@ -822,9 +952,10 @@ EXPORT_SYMBOL_GPL(venus_helper_vb2_buf_init);
 int venus_helper_vb2_buf_prepare(struct vb2_buffer *vb)
 {
 	struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+	unsigned int out_buf_size = venus_helper_get_opb_size(inst);
 
 	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
-	    vb2_plane_size(vb, 0) < inst->output_buf_size)
+	    vb2_plane_size(vb, 0) < out_buf_size)
 		return -EINVAL;
 	if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
 	    vb2_plane_size(vb, 0) < inst->input_buf_size)
@@ -894,6 +1025,8 @@ void venus_helper_vb2_stop_streaming(struct vb2_queue *q)
 		if (ret)
 			hfi_session_abort(inst);
 
+		venus_helper_free_dpb_bufs(inst);
+
 		load_scale_clocks(core);
 		INIT_LIST_HEAD(&inst->registeredbufs);
 	}
@@ -932,8 +1065,14 @@ int venus_helper_vb2_start_streaming(struct venus_inst *inst)
 	if (ret)
 		goto err_unload_res;
 
+	ret = venus_helper_queue_dpb_bufs(inst);
+	if (ret)
+		goto err_session_stop;
+
 	return 0;
 
+err_session_stop:
+	hfi_session_stop(inst);
 err_unload_res:
 	hfi_session_unload_res(inst);
 err_unreg_bufs:
@@ -987,6 +1126,67 @@ void venus_helper_init_instance(struct venus_inst *inst)
 }
 EXPORT_SYMBOL_GPL(venus_helper_init_instance);
 
+static bool find_fmt_from_caps(struct venus_caps *caps, u32 buftype, u32 fmt)
+{
+	unsigned int i;
+
+	for (i = 0; i < caps->num_fmts; i++) {
+		if (caps->fmts[i].buftype == buftype &&
+		    caps->fmts[i].fmt == fmt)
+			return true;
+	}
+
+	return false;
+}
+
+int venus_helper_get_out_fmts(struct venus_inst *inst, u32 v4l2_fmt,
+			      u32 *out_fmt, u32 *out2_fmt, bool ubwc)
+{
+	struct venus_core *core = inst->core;
+	struct venus_caps *caps;
+	u32 ubwc_fmt, fmt = to_hfi_raw_fmt(v4l2_fmt);
+	bool found, found_ubwc;
+
+	*out_fmt = *out2_fmt = 0;
+
+	if (!fmt)
+		return -EINVAL;
+
+	caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
+	if (!caps)
+		return -EINVAL;
+
+	if (ubwc) {
+		ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE;
+		found_ubwc = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT,
+						ubwc_fmt);
+		found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
+
+		if (found_ubwc && found) {
+			*out_fmt = ubwc_fmt;
+			*out2_fmt = fmt;
+			return 0;
+		}
+	}
+
+	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT, fmt);
+	if (found) {
+		*out_fmt = fmt;
+		*out2_fmt = 0;
+		return 0;
+	}
+
+	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
+	if (found) {
+		*out_fmt = 0;
+		*out2_fmt = fmt;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+EXPORT_SYMBOL_GPL(venus_helper_get_out_fmts);
+
 int venus_helper_power_enable(struct venus_core *core, u32 session_type,
 			      bool enable)
 {
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 92b167a47166..2475f284f396 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -50,10 +50,16 @@ int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
 int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype);
+int venus_helper_set_multistream(struct venus_inst *inst, bool out_en,
+				 bool out2_en);
 unsigned int venus_helper_get_opb_size(struct venus_inst *inst);
 void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
 void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
 void venus_helper_init_instance(struct venus_inst *inst);
+int venus_helper_get_out_fmts(struct venus_inst *inst, u32 fmt, u32 *out_fmt,
+			      u32 *out2_fmt, bool ubwc);
+int venus_helper_alloc_dpb_bufs(struct venus_inst *inst);
+int venus_helper_free_dpb_bufs(struct venus_inst *inst);
 int venus_helper_power_enable(struct venus_core *core, u32 session_type,
 			      bool enable);
 #endif
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 28db28fb5f21..bab2b80d573a 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -532,10 +532,16 @@ static int vdec_set_properties(struct venus_inst *inst)
 	return 0;
 }
 
+#define is_ubwc_fmt(fmt) (!!((fmt) & HFI_COLOR_FORMAT_UBWC_BASE))
+
 static int vdec_output_conf(struct venus_inst *inst)
 {
 	struct venus_core *core = inst->core;
 	struct hfi_enable en = { .enable = 1 };
+	u32 width = inst->out_width;
+	u32 height = inst->out_height;
+	u32 out_fmt, out2_fmt;
+	bool ubwc = false;
 	u32 ptype;
 	int ret;
 
@@ -554,6 +560,80 @@ static int vdec_output_conf(struct venus_inst *inst)
 			return ret;
 	}
 
+	/* Force searching UBWC formats for bigger then HD resolutions */
+	if (width > 1920 && height > ALIGN(1080, 32))
+		ubwc = true;
+
+	/* For Venus v4 UBWC format is mandatory */
+	if (IS_V4(core))
+		ubwc = true;
+
+	ret = venus_helper_get_out_fmts(inst, inst->fmt_cap->pixfmt, &out_fmt,
+					&out2_fmt, ubwc);
+	if (ret)
+		return ret;
+
+	inst->output_buf_size =
+			venus_helper_get_framesz_raw(out_fmt, width, height);
+	inst->output2_buf_size =
+			venus_helper_get_framesz_raw(out2_fmt, width, height);
+
+	if (is_ubwc_fmt(out_fmt)) {
+		inst->opb_buftype = HFI_BUFFER_OUTPUT2;
+		inst->opb_fmt = out2_fmt;
+		inst->dpb_buftype = HFI_BUFFER_OUTPUT;
+		inst->dpb_fmt = out_fmt;
+	} else if (is_ubwc_fmt(out2_fmt)) {
+		inst->opb_buftype = HFI_BUFFER_OUTPUT;
+		inst->opb_fmt = out_fmt;
+		inst->dpb_buftype = HFI_BUFFER_OUTPUT2;
+		inst->dpb_fmt = out2_fmt;
+	} else {
+		inst->opb_buftype = HFI_BUFFER_OUTPUT;
+		inst->opb_fmt = out_fmt;
+		inst->dpb_buftype = 0;
+		inst->dpb_fmt = 0;
+	}
+
+	ret = venus_helper_set_raw_format(inst, inst->opb_fmt,
+					  inst->opb_buftype);
+	if (ret)
+		return ret;
+
+	if (inst->dpb_fmt) {
+		ret = venus_helper_set_multistream(inst, false, true);
+		if (ret)
+			return ret;
+
+		ret = venus_helper_set_raw_format(inst, inst->dpb_fmt,
+						  inst->dpb_buftype);
+		if (ret)
+			return ret;
+
+		ret = venus_helper_set_output_resolution(inst, width, height,
+							 HFI_BUFFER_OUTPUT2);
+		if (ret)
+			return ret;
+	}
+
+	if (IS_V3(core) || IS_V4(core)) {
+		if (inst->output2_buf_size) {
+			ret = venus_helper_set_bufsize(inst,
+						       inst->output2_buf_size,
+						       HFI_BUFFER_OUTPUT2);
+			if (ret)
+				return ret;
+		}
+
+		if (inst->output_buf_size) {
+			ret = venus_helper_set_bufsize(inst,
+						       inst->output_buf_size,
+						       HFI_BUFFER_OUTPUT);
+			if (ret)
+				return ret;
+		}
+	}
+
 	ret = venus_helper_set_dyn_bufmode(inst);
 	if (ret)
 		return ret;
@@ -624,6 +704,8 @@ static int vdec_queue_setup(struct vb2_queue *q,
 	int ret = 0;
 
 	if (*num_planes) {
+		unsigned int output_buf_size = venus_helper_get_opb_size(inst);
+
 		if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
 		    *num_planes != inst->fmt_out->num_planes)
 			return -EINVAL;
@@ -637,7 +719,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
 			return -EINVAL;
 
 		if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
-		    sizes[0] < inst->output_buf_size)
+		    sizes[0] < output_buf_size)
 			return -EINVAL;
 
 		return 0;
@@ -746,6 +828,10 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 	if (ret)
 		goto deinit_sess;
 
+	ret = venus_helper_alloc_dpb_bufs(inst);
+	if (ret)
+		goto deinit_sess;
+
 	ret = venus_helper_vb2_start_streaming(inst);
 	if (ret)
 		goto deinit_sess;
@@ -797,9 +883,11 @@ static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type,
 	vbuf->field = V4L2_FIELD_NONE;
 
 	if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		unsigned int opb_sz = venus_helper_get_opb_size(inst);
+
 		vb = &vbuf->vb2_buf;
 		vb->planes[0].bytesused =
-			max_t(unsigned int, inst->output_buf_size, bytesused);
+			max_t(unsigned int, opb_sz, bytesused);
 		vb->planes[0].data_offset = data_offset;
 		vb->timestamp = timestamp_us * NSEC_PER_USEC;
 		vbuf->sequence = inst->sequence_cap++;
@@ -937,6 +1025,7 @@ static int vdec_open(struct file *file)
 	if (!inst)
 		return -ENOMEM;
 
+	INIT_LIST_HEAD(&inst->dpbbufs);
 	INIT_LIST_HEAD(&inst->registeredbufs);
 	INIT_LIST_HEAD(&inst->internalbufs);
 	INIT_LIST_HEAD(&inst->list);
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 54f253b98b24..a703bce78abc 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -1084,6 +1084,7 @@ static int venc_open(struct file *file)
 	if (!inst)
 		return -ENOMEM;
 
+	INIT_LIST_HEAD(&inst->dpbbufs);
 	INIT_LIST_HEAD(&inst->registeredbufs);
 	INIT_LIST_HEAD(&inst->internalbufs);
 	INIT_LIST_HEAD(&inst->list);
-- 
2.14.1

Re: [PATCH v2 27/29] venus: implementing multi-stream support

[Tomasz Figa]

On Tue, May 15, 2018 at 5:00 PM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> This is implementing a multi-stream decoder support. The multi
> stream gives an option to use the secondary decoder output
> with different raw format (or the same in case of crop).
>
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/core.h    |   1 +
>  drivers/media/platform/qcom/venus/helpers.c | 204 +++++++++++++++++++++++++++-
>  drivers/media/platform/qcom/venus/helpers.h |   6 +
>  drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
>  drivers/media/platform/qcom/venus/venc.c    |   1 +
>  5 files changed, 299 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
> index 4d6c05f156c4..85e66e2dd672 100644
> --- a/drivers/media/platform/qcom/venus/core.h
> +++ b/drivers/media/platform/qcom/venus/core.h
> @@ -259,6 +259,7 @@ struct venus_inst {
>         struct list_head list;
>         struct mutex lock;
>         struct venus_core *core;
> +       struct list_head dpbbufs;
>         struct list_head internalbufs;
>         struct list_head registeredbufs;
>         struct list_head delayed_process;
> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
> index ed569705ecac..87dcf9973e6f 100644
> --- a/drivers/media/platform/qcom/venus/helpers.c
> +++ b/drivers/media/platform/qcom/venus/helpers.c
> @@ -85,6 +85,112 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
>  }
>  EXPORT_SYMBOL_GPL(venus_helper_check_codec);
>
> +static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
> +{
> +       struct intbuf *buf;
> +       int ret = 0;
> +
> +       if (list_empty(&inst->dpbbufs))
> +               return 0;

Does this special case give us anything other than few more source lines?

> +
> +       list_for_each_entry(buf, &inst->dpbbufs, list) {
> +               struct hfi_frame_data fdata;
> +
> +               memset(&fdata, 0, sizeof(fdata));
> +               fdata.alloc_len = buf->size;
> +               fdata.device_addr = buf->da;
> +               fdata.buffer_type = buf->type;
> +
> +               ret = hfi_session_process_buf(inst, &fdata);
> +               if (ret)
> +                       goto fail;
> +       }
> +
> +fail:
> +       return ret;
> +}
> +
> +int venus_helper_free_dpb_bufs(struct venus_inst *inst)
> +{
> +       struct intbuf *buf, *n;
> +
> +       if (list_empty(&inst->dpbbufs))
> +               return 0;

Ditto.

> +
> +       list_for_each_entry_safe(buf, n, &inst->dpbbufs, list) {
> +               list_del_init(&buf->list);
> +               dma_free_attrs(inst->core->dev, buf->size, buf->va, buf->da,
> +                              buf->attrs);
> +               kfree(buf);
> +       }
> +
> +       INIT_LIST_HEAD(&inst->dpbbufs);
> +
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(venus_helper_free_dpb_bufs);
[snip]
> +int venus_helper_get_out_fmts(struct venus_inst *inst, u32 v4l2_fmt,
> +                             u32 *out_fmt, u32 *out2_fmt, bool ubwc)
> +{
> +       struct venus_core *core = inst->core;
> +       struct venus_caps *caps;
> +       u32 ubwc_fmt, fmt = to_hfi_raw_fmt(v4l2_fmt);
> +       bool found, found_ubwc;
> +
> +       *out_fmt = *out2_fmt = 0;
> +
> +       if (!fmt)
> +               return -EINVAL;
> +
> +       caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
> +       if (!caps)
> +               return -EINVAL;
> +
> +       if (ubwc) {
> +               ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE;

Does the UBWC base format have to be the same as fmt? Looking at
HFI_COLOR_FORMAT_* macros, UBWC variants seem to exist only for few
selected raw formats, for example there is none for NV21.

Best regards,
Tomasz

Re: [PATCH v2 27/29] venus: implementing multi-stream support

[Tomasz Figa]

Hi Stanimir,

On Thu, May 31, 2018 at 6:51 PM Tomasz Figa <tfiga@chromium.org> wrote:
>
> On Tue, May 15, 2018 at 5:00 PM Stanimir Varbanov
> <stanimir.varbanov@linaro.org> wrote:
> >
> > This is implementing a multi-stream decoder support. The multi
> > stream gives an option to use the secondary decoder output
> > with different raw format (or the same in case of crop).
> >
> > Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> > ---
> >  drivers/media/platform/qcom/venus/core.h    |   1 +
> >  drivers/media/platform/qcom/venus/helpers.c | 204 +++++++++++++++++++++++++++-
> >  drivers/media/platform/qcom/venus/helpers.h |   6 +
> >  drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
> >  drivers/media/platform/qcom/venus/venc.c    |   1 +
> >  5 files changed, 299 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
> > index 4d6c05f156c4..85e66e2dd672 100644
> > --- a/drivers/media/platform/qcom/venus/core.h
> > +++ b/drivers/media/platform/qcom/venus/core.h
> > @@ -259,6 +259,7 @@ struct venus_inst {
> >         struct list_head list;
> >         struct mutex lock;
> >         struct venus_core *core;
> > +       struct list_head dpbbufs;
> >         struct list_head internalbufs;
> >         struct list_head registeredbufs;
> >         struct list_head delayed_process;
> > diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
> > index ed569705ecac..87dcf9973e6f 100644
> > --- a/drivers/media/platform/qcom/venus/helpers.c
> > +++ b/drivers/media/platform/qcom/venus/helpers.c
> > @@ -85,6 +85,112 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
> >  }
> >  EXPORT_SYMBOL_GPL(venus_helper_check_codec);
> >
> > +static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
> > +{
> > +       struct intbuf *buf;
> > +       int ret = 0;
> > +
> > +       if (list_empty(&inst->dpbbufs))
> > +               return 0;
>
> Does this special case give us anything other than few more source lines?
>
> > +
> > +       list_for_each_entry(buf, &inst->dpbbufs, list) {
> > +               struct hfi_frame_data fdata;
> > +
> > +               memset(&fdata, 0, sizeof(fdata));
> > +               fdata.alloc_len = buf->size;
> > +               fdata.device_addr = buf->da;
> > +               fdata.buffer_type = buf->type;
> > +
> > +               ret = hfi_session_process_buf(inst, &fdata);
> > +               if (ret)
> > +                       goto fail;
> > +       }
> > +
> > +fail:
> > +       return ret;
> > +}
> > +
> > +int venus_helper_free_dpb_bufs(struct venus_inst *inst)
> > +{
> > +       struct intbuf *buf, *n;
> > +
> > +       if (list_empty(&inst->dpbbufs))
> > +               return 0;
>
> Ditto.
>
> > +
> > +       list_for_each_entry_safe(buf, n, &inst->dpbbufs, list) {
> > +               list_del_init(&buf->list);
> > +               dma_free_attrs(inst->core->dev, buf->size, buf->va, buf->da,
> > +                              buf->attrs);
> > +               kfree(buf);
> > +       }
> > +
> > +       INIT_LIST_HEAD(&inst->dpbbufs);
> > +
> > +       return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(venus_helper_free_dpb_bufs);
> [snip]
> > +int venus_helper_get_out_fmts(struct venus_inst *inst, u32 v4l2_fmt,
> > +                             u32 *out_fmt, u32 *out2_fmt, bool ubwc)
> > +{
> > +       struct venus_core *core = inst->core;
> > +       struct venus_caps *caps;
> > +       u32 ubwc_fmt, fmt = to_hfi_raw_fmt(v4l2_fmt);
> > +       bool found, found_ubwc;
> > +
> > +       *out_fmt = *out2_fmt = 0;
> > +
> > +       if (!fmt)
> > +               return -EINVAL;
> > +
> > +       caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
> > +       if (!caps)
> > +               return -EINVAL;
> > +
> > +       if (ubwc) {
> > +               ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE;
>
> Does the UBWC base format have to be the same as fmt? Looking at
> HFI_COLOR_FORMAT_* macros, UBWC variants seem to exist only for few
> selected raw formats, for example there is none for NV21.

Ping.

None of the comments I posted above have been addressed in v4.

Best regards,
Tomasz

Re: [PATCH v2 27/29] venus: implementing multi-stream support

[Stanimir Varbanov]

Hi Tomasz,

On 05/31/2018 12:51 PM, Tomasz Figa wrote:
> On Tue, May 15, 2018 at 5:00 PM Stanimir Varbanov
> <stanimir.varbanov@linaro.org> wrote:
>>
>> This is implementing a multi-stream decoder support. The multi
>> stream gives an option to use the secondary decoder output
>> with different raw format (or the same in case of crop).
>>
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>  drivers/media/platform/qcom/venus/core.h    |   1 +
>>  drivers/media/platform/qcom/venus/helpers.c | 204 +++++++++++++++++++++++++++-
>>  drivers/media/platform/qcom/venus/helpers.h |   6 +
>>  drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
>>  drivers/media/platform/qcom/venus/venc.c    |   1 +
>>  5 files changed, 299 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
>> index 4d6c05f156c4..85e66e2dd672 100644
>> --- a/drivers/media/platform/qcom/venus/core.h
>> +++ b/drivers/media/platform/qcom/venus/core.h
>> @@ -259,6 +259,7 @@ struct venus_inst {
>>         struct list_head list;
>>         struct mutex lock;
>>         struct venus_core *core;
>> +       struct list_head dpbbufs;
>>         struct list_head internalbufs;
>>         struct list_head registeredbufs;
>>         struct list_head delayed_process;
>> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
>> index ed569705ecac..87dcf9973e6f 100644
>> --- a/drivers/media/platform/qcom/venus/helpers.c
>> +++ b/drivers/media/platform/qcom/venus/helpers.c
>> @@ -85,6 +85,112 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
>>  }
>>  EXPORT_SYMBOL_GPL(venus_helper_check_codec);
>>
>> +static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
>> +{
>> +       struct intbuf *buf;
>> +       int ret = 0;
>> +
>> +       if (list_empty(&inst->dpbbufs))
>> +               return 0;
> 
> Does this special case give us anything other than few more source lines?

yes, thanks for spotting, will drop above lines here and below.

> 
>> +
>> +       list_for_each_entry(buf, &inst->dpbbufs, list) {
>> +               struct hfi_frame_data fdata;
>> +
>> +               memset(&fdata, 0, sizeof(fdata));
>> +               fdata.alloc_len = buf->size;
>> +               fdata.device_addr = buf->da;
>> +               fdata.buffer_type = buf->type;
>> +
>> +               ret = hfi_session_process_buf(inst, &fdata);
>> +               if (ret)
>> +                       goto fail;
>> +       }
>> +
>> +fail:
>> +       return ret;
>> +}
>> +
>> +int venus_helper_free_dpb_bufs(struct venus_inst *inst)
>> +{
>> +       struct intbuf *buf, *n;
>> +
>> +       if (list_empty(&inst->dpbbufs))
>> +               return 0;
> 
> Ditto.
> 
>> +
>> +       list_for_each_entry_safe(buf, n, &inst->dpbbufs, list) {
>> +               list_del_init(&buf->list);
>> +               dma_free_attrs(inst->core->dev, buf->size, buf->va, buf->da,
>> +                              buf->attrs);
>> +               kfree(buf);
>> +       }
>> +
>> +       INIT_LIST_HEAD(&inst->dpbbufs);
>> +
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(venus_helper_free_dpb_bufs);
> [snip]
>> +int venus_helper_get_out_fmts(struct venus_inst *inst, u32 v4l2_fmt,
>> +                             u32 *out_fmt, u32 *out2_fmt, bool ubwc)
>> +{
>> +       struct venus_core *core = inst->core;
>> +       struct venus_caps *caps;
>> +       u32 ubwc_fmt, fmt = to_hfi_raw_fmt(v4l2_fmt);
>> +       bool found, found_ubwc;
>> +
>> +       *out_fmt = *out2_fmt = 0;
>> +
>> +       if (!fmt)
>> +               return -EINVAL;
>> +
>> +       caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
>> +       if (!caps)
>> +               return -EINVAL;
>> +
>> +       if (ubwc) {
>> +               ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE;
> 
> Does the UBWC base format have to be the same as fmt? Looking at
> HFI_COLOR_FORMAT_* macros, UBWC variants seem to exist only for few
> selected raw formats, for example there is none for NV21.

I think any raw format can have its UBWC variant. And yes we have only
one macro but we are checking against parsed capabilities from firmware
where the supported formats are stored.

-- 
regards,
Stan

Re: [PATCH v2 27/29] venus: implementing multi-stream support

[Tomasz Figa]

On Mon, Jul 2, 2018 at 9:43 PM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> Hi Tomasz,
>
> On 05/31/2018 12:51 PM, Tomasz Figa wrote:
> > On Tue, May 15, 2018 at 5:00 PM Stanimir Varbanov
> > <stanimir.varbanov@linaro.org> wrote:
> >>
> >> This is implementing a multi-stream decoder support. The multi
> >> stream gives an option to use the secondary decoder output
> >> with different raw format (or the same in case of crop).
> >>
> >> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> >> ---
> >>  drivers/media/platform/qcom/venus/core.h    |   1 +
> >>  drivers/media/platform/qcom/venus/helpers.c | 204 +++++++++++++++++++++++++++-
> >>  drivers/media/platform/qcom/venus/helpers.h |   6 +
> >>  drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
> >>  drivers/media/platform/qcom/venus/venc.c    |   1 +
> >>  5 files changed, 299 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
> >> index 4d6c05f156c4..85e66e2dd672 100644
> >> --- a/drivers/media/platform/qcom/venus/core.h
> >> +++ b/drivers/media/platform/qcom/venus/core.h
> >> @@ -259,6 +259,7 @@ struct venus_inst {
> >>         struct list_head list;
> >>         struct mutex lock;
> >>         struct venus_core *core;
> >> +       struct list_head dpbbufs;
> >>         struct list_head internalbufs;
> >>         struct list_head registeredbufs;
> >>         struct list_head delayed_process;
> >> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
> >> index ed569705ecac..87dcf9973e6f 100644
> >> --- a/drivers/media/platform/qcom/venus/helpers.c
> >> +++ b/drivers/media/platform/qcom/venus/helpers.c
> >> @@ -85,6 +85,112 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
> >>  }
> >>  EXPORT_SYMBOL_GPL(venus_helper_check_codec);
> >>
> >> +static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
> >> +{
> >> +       struct intbuf *buf;
> >> +       int ret = 0;
> >> +
> >> +       if (list_empty(&inst->dpbbufs))
> >> +               return 0;
> >
> > Does this special case give us anything other than few more source lines?
>
> yes, thanks for spotting, will drop above lines here and below.
>
> >
> >> +
> >> +       list_for_each_entry(buf, &inst->dpbbufs, list) {
> >> +               struct hfi_frame_data fdata;
> >> +
> >> +               memset(&fdata, 0, sizeof(fdata));
> >> +               fdata.alloc_len = buf->size;
> >> +               fdata.device_addr = buf->da;
> >> +               fdata.buffer_type = buf->type;
> >> +
> >> +               ret = hfi_session_process_buf(inst, &fdata);
> >> +               if (ret)
> >> +                       goto fail;
> >> +       }
> >> +
> >> +fail:
> >> +       return ret;
> >> +}
> >> +
> >> +int venus_helper_free_dpb_bufs(struct venus_inst *inst)
> >> +{
> >> +       struct intbuf *buf, *n;
> >> +
> >> +       if (list_empty(&inst->dpbbufs))
> >> +               return 0;
> >
> > Ditto.
> >
> >> +
> >> +       list_for_each_entry_safe(buf, n, &inst->dpbbufs, list) {
> >> +               list_del_init(&buf->list);
> >> +               dma_free_attrs(inst->core->dev, buf->size, buf->va, buf->da,
> >> +                              buf->attrs);
> >> +               kfree(buf);
> >> +       }
> >> +
> >> +       INIT_LIST_HEAD(&inst->dpbbufs);
> >> +
> >> +       return 0;
> >> +}
> >> +EXPORT_SYMBOL_GPL(venus_helper_free_dpb_bufs);
> > [snip]
> >> +int venus_helper_get_out_fmts(struct venus_inst *inst, u32 v4l2_fmt,
> >> +                             u32 *out_fmt, u32 *out2_fmt, bool ubwc)
> >> +{
> >> +       struct venus_core *core = inst->core;
> >> +       struct venus_caps *caps;
> >> +       u32 ubwc_fmt, fmt = to_hfi_raw_fmt(v4l2_fmt);
> >> +       bool found, found_ubwc;
> >> +
> >> +       *out_fmt = *out2_fmt = 0;
> >> +
> >> +       if (!fmt)
> >> +               return -EINVAL;
> >> +
> >> +       caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
> >> +       if (!caps)
> >> +               return -EINVAL;
> >> +
> >> +       if (ubwc) {
> >> +               ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE;
> >
> > Does the UBWC base format have to be the same as fmt? Looking at
> > HFI_COLOR_FORMAT_* macros, UBWC variants seem to exist only for few
> > selected raw formats, for example there is none for NV21.
>
> I think any raw format can have its UBWC variant. And yes we have only
> one macro but we are checking against parsed capabilities from firmware
> where the supported formats are stored.

Okay, thanks.

Best regards,
Tomasz

[PATCH v2 28/29] venus: add sdm845 compatible and resource data

[Stanimir Varbanov]

This adds sdm845 DT compatible string with it's resource
data table.

Reviewed-by: Rob Herring <robh@kernel.org>
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 .../devicetree/bindings/media/qcom,venus.txt       |  1 +
 drivers/media/platform/qcom/venus/core.c           | 22 ++++++++++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/qcom,venus.txt b/Documentation/devicetree/bindings/media/qcom,venus.txt
index 2693449daf73..00d0d1bf7647 100644
--- a/Documentation/devicetree/bindings/media/qcom,venus.txt
+++ b/Documentation/devicetree/bindings/media/qcom,venus.txt
@@ -6,6 +6,7 @@
 	Definition: Value should contain one of:
 		- "qcom,msm8916-venus"
 		- "qcom,msm8996-venus"
+		- "qcom,sdm845-venus"
 - reg:
 	Usage: required
 	Value type: <prop-encoded-array>
diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/venus/core.c
index 381bfdd688db..bb6add9d340e 100644
--- a/drivers/media/platform/qcom/venus/core.c
+++ b/drivers/media/platform/qcom/venus/core.c
@@ -450,9 +450,31 @@ static const struct venus_resources msm8996_res = {
 	.fwname = "qcom/venus-4.2/venus.mdt",
 };
 
+static const struct freq_tbl sdm845_freq_table[] = {
+	{ 1944000, 380000000 },	/* 4k UHD @ 60 */
+	{  972000, 320000000 },	/* 4k UHD @ 30 */
+	{  489600, 200000000 },	/* 1080p @ 60 */
+	{  244800, 100000000 },	/* 1080p @ 30 */
+};
+
+static const struct venus_resources sdm845_res = {
+	.freq_tbl = sdm845_freq_table,
+	.freq_tbl_size = ARRAY_SIZE(sdm845_freq_table),
+	.clks = {"core", "iface", "bus" },
+	.clks_num = 3,
+	.max_load = 2563200,
+	.hfi_version = HFI_VERSION_4XX,
+	.vmem_id = VIDC_RESOURCE_NONE,
+	.vmem_size = 0,
+	.vmem_addr = 0,
+	.dma_mask = 0xe0000000 - 1,
+	.fwname = "qcom/venus-5.2/venus.mdt",
+};
+
 static const struct of_device_id venus_dt_match[] = {
 	{ .compatible = "qcom,msm8916-venus", .data = &msm8916_res, },
 	{ .compatible = "qcom,msm8996-venus", .data = &msm8996_res, },
+	{ .compatible = "qcom,sdm845-venus", .data = &sdm845_res, },
 	{ }
 };
 MODULE_DEVICE_TABLE(of, venus_dt_match);
-- 
2.14.1

[PATCH v2 29/29] venus: add HEVC codec support

[Stanimir Varbanov]

This add HEVC codec support for venus versions 3xx and 4xx.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h    |  2 ++
 drivers/media/platform/qcom/venus/helpers.c |  3 ++
 drivers/media/platform/qcom/venus/hfi.c     |  2 ++
 drivers/media/platform/qcom/venus/vdec.c    |  4 +++
 drivers/media/platform/qcom/venus/venc.c    | 49 +++++++++++++++++++++++++++++
 5 files changed, 60 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 85e66e2dd672..2a956d1b9bd1 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -185,10 +185,12 @@ struct venc_controls {
 		u32 mpeg4;
 		u32 h264;
 		u32 vpx;
+		u32 hevc;
 	} profile;
 	struct {
 		u32 mpeg4;
 		u32 h264;
+		u32 hevc;
 	} level;
 };
 
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 87dcf9973e6f..fecadba039cf 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -71,6 +71,9 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
 	case V4L2_PIX_FMT_XVID:
 		codec = HFI_VIDEO_CODEC_DIVX;
 		break;
+	case V4L2_PIX_FMT_HEVC:
+		codec = HFI_VIDEO_CODEC_HEVC;
+		break;
 	default:
 		return false;
 	}
diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
index 94ca27b0bb99..24207829982f 100644
--- a/drivers/media/platform/qcom/venus/hfi.c
+++ b/drivers/media/platform/qcom/venus/hfi.c
@@ -49,6 +49,8 @@ static u32 to_codec_type(u32 pixfmt)
 		return HFI_VIDEO_CODEC_VP9;
 	case V4L2_PIX_FMT_XVID:
 		return HFI_VIDEO_CODEC_DIVX;
+	case V4L2_PIX_FMT_HEVC:
+		return HFI_VIDEO_CODEC_HEVC;
 	default:
 		return 0;
 	}
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index d2d082009686..d2ffd9bd44de 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -77,6 +77,10 @@ static const struct venus_format vdec_formats[] = {
 		.pixfmt = V4L2_PIX_FMT_XVID,
 		.num_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_HEVC,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
 	},
 };
 
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index a703bce78abc..50a04cb1cc22 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -59,6 +59,10 @@ static const struct venus_format venc_formats[] = {
 		.pixfmt = V4L2_PIX_FMT_VP8,
 		.num_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_HEVC,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
 	},
 };
 
@@ -220,6 +224,46 @@ static int venc_v4l2_to_hfi(int id, int value)
 		case V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_DISABLED_AT_SLICE_BOUNDARY:
 			return HFI_H264_DB_MODE_SKIP_SLICE_BOUNDARY;
 		}
+	case V4L2_CID_MPEG_VIDEO_HEVC_PROFILE:
+		switch (value) {
+		case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN:
+		default:
+			return HFI_HEVC_PROFILE_MAIN;
+		case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE:
+			return HFI_HEVC_PROFILE_MAIN_STILL_PIC;
+		case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10:
+			return HFI_HEVC_PROFILE_MAIN10;
+		}
+	case V4L2_CID_MPEG_VIDEO_HEVC_LEVEL:
+		switch (value) {
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_1:
+		default:
+			return HFI_HEVC_LEVEL_1;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_2:
+			return HFI_HEVC_LEVEL_2;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1:
+			return HFI_HEVC_LEVEL_21;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_3:
+			return HFI_HEVC_LEVEL_3;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1:
+			return HFI_HEVC_LEVEL_31;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_4:
+			return HFI_HEVC_LEVEL_4;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1:
+			return HFI_HEVC_LEVEL_41;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_5:
+			return HFI_HEVC_LEVEL_5;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1:
+			return HFI_HEVC_LEVEL_51;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2:
+			return HFI_HEVC_LEVEL_52;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_6:
+			return HFI_HEVC_LEVEL_6;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1:
+			return HFI_HEVC_LEVEL_61;
+		case V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2:
+			return HFI_HEVC_LEVEL_62;
+		}
 	}
 
 	return 0;
@@ -744,6 +788,11 @@ static int venc_set_properties(struct venus_inst *inst)
 	} else if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_H263) {
 		profile = 0;
 		level = 0;
+	} else if (inst->fmt_cap->pixfmt == V4L2_PIX_FMT_HEVC) {
+		profile = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
+					   ctr->profile.hevc);
+		level = venc_v4l2_to_hfi(V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
+					 ctr->level.hevc);
 	}
 
 	ptype = HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT;
-- 
2.14.1

Re: [PATCH v2 00/29] Venus updates

[Hans Verkuil]

Hi Stanimir,

On 05/15/18 09:58, Stanimir Varbanov wrote:
> Hello,
> 
> Here is v2 with following comments addressed:
> 
> * reworked venus suspend 3xx and reuse it for 4xx.
> * drop 10/28 patch from v1, i.e. call of session_continue when
>   buffer requirements are not sufficient.
> * fixed kbuild test robot warning in 11/28 by allocating instance
>   variable from heap.
> * spelling typo in 15/28.
> * added Reviewed-by for DT changes.
> * extended 28/28 HEVC support for encoder, now the profile and
>   level are selected properly.
> 
> Comments are welcome!

It all looks good, except for patch 27 (see my comments there).

Once there is a v3 for patch 27 and I'm OK with it, then just let me
know when it is ready in your opinion to be merged.

Regards,

	Hans

> 
> regards,
> Stan
> 
> Stanimir Varbanov (29):
>   venus: hfi_msgs: correct pointer increment
>   venus: hfi: preparation to support venus 4xx
>   venus: hfi: update sequence event to handle more properties
>   venus: hfi_cmds: add set_properties for 4xx version
>   venus: hfi: support session continue for 4xx version
>   venus: hfi: handle buffer output2 type as well
>   venus: hfi_venus: add halt AXI support for Venus 4xx
>   venus: hfi_venus: fix suspend function for venus 3xx versions
>   venus: hfi_venus: move set of default properties to core init
>   venus: hfi_venus: add suspend functionality for Venus 4xx
>   venus: venc,vdec: adds clocks needed for venus 4xx
>   venus: add common capability parser
>   venus: helpers: make a commmon function for power_enable
>   venus: core: delete not used flag for buffer mode
>   venus: helpers: rename a helper function and use buffer mode from caps
>   venus: add a helper function to set dynamic buffer mode
>   venus: add helper function to set actual buffer size
>   venus: delete no longer used bufmode flag from instance
>   venus: helpers: add buffer type argument to a helper
>   venus: helpers: add a new helper to set raw format
>   venus: helpers,vdec,venc: add helpers to set work mode and core usage
>   venus: helpers: extend set_num_bufs helper with one more argument
>   venus: helpers: add a helper to return opb buffer sizes
>   venus: vdec: get required input buffers as well
>   venus: vdec: new function for output configuration
>   venus: move frame size calculations in common place
>   venus: implementing multi-stream support
>   venus: add sdm845 compatible and resource data
>   venus: add HEVC codec support
> 
>  .../devicetree/bindings/media/qcom,venus.txt       |   1 +
>  drivers/media/platform/qcom/venus/Makefile         |   3 +-
>  drivers/media/platform/qcom/venus/core.c           | 107 ++++
>  drivers/media/platform/qcom/venus/core.h           |  93 ++--
>  drivers/media/platform/qcom/venus/helpers.c        | 558 +++++++++++++++++++--
>  drivers/media/platform/qcom/venus/helpers.h        |  23 +-
>  drivers/media/platform/qcom/venus/hfi.c            |  12 +-
>  drivers/media/platform/qcom/venus/hfi.h            |   9 +
>  drivers/media/platform/qcom/venus/hfi_cmds.c       |  64 ++-
>  drivers/media/platform/qcom/venus/hfi_helper.h     | 112 ++++-
>  drivers/media/platform/qcom/venus/hfi_msgs.c       | 401 +++------------
>  drivers/media/platform/qcom/venus/hfi_parser.c     | 291 +++++++++++
>  drivers/media/platform/qcom/venus/hfi_parser.h     |  45 ++
>  drivers/media/platform/qcom/venus/hfi_venus.c      |  95 +++-
>  drivers/media/platform/qcom/venus/hfi_venus_io.h   |  25 +
>  drivers/media/platform/qcom/venus/vdec.c           | 316 +++++++-----
>  drivers/media/platform/qcom/venus/venc.c           | 211 ++++----
>  17 files changed, 1689 insertions(+), 677 deletions(-)
>  create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.c
>  create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.h
> 

Re: [PATCH v2 00/29] Venus updates

[Tomasz Figa]

Hi Stanimir,

On Tue, May 15, 2018 at 5:14 PM Stanimir Varbanov
<stanimir.varbanov@linaro.org> wrote:
>
> Hello,
>
> Here is v2 with following comments addressed:
>
> * reworked venus suspend 3xx and reuse it for 4xx.
> * drop 10/28 patch from v1, i.e. call of session_continue when
>   buffer requirements are not sufficient.
> * fixed kbuild test robot warning in 11/28 by allocating instance
>   variable from heap.
> * spelling typo in 15/28.
> * added Reviewed-by for DT changes.
> * extended 28/28 HEVC support for encoder, now the profile and
>   level are selected properly.
>
> Comments are welcome!

Thanks a lot for the series. I finally managed to finish reviewing it.
Sorry for taking so long.

For the patches I didn't send any comments for, please feel free to
add my Reviewed-by.

Best regards,
Tomasz