[PATCH 00/28] Venus updates

[PATCH 00/28] Venus updates

[Stanimir Varbanov]

Hello,

This patch set aims to:

* add initial support for Venus version 4xx (found on sdm845).

* introduce a common capability parser to enumerate better
  supported uncompressed formats, capabilities by codec,
  supported codecs and so on.

* also contains various cleanups, readability improvements
  and fixes.

* adds HEVC codec support for the Venus versions which has
  support for it.

* add multi-stream support (secondary decoder output), which
  will give as an opportunity to use UBWC compressed formats
  to optimize internal interconnect bandwidth on higher
  resolutions.

Comments are welcome!

regards,
Stan

Stanimir Varbanov (28):
  venus: hfi_msgs: correct pointer increment
  venus: hfi: preparation to support venus 4xx
  venus: hfi: update sequence event to handle more properties
  venus: hfi_cmds: add set_properties for 4xx version
  venus: hfi: support session continue for 4xx version
  venus: hfi: handle buffer output2 type as well
  venus: hfi_venus: add halt AXI support for Venus 4xx
  venus: hfi_venus: add suspend function for 4xx version
  venus: venc,vdec: adds clocks needed for venus 4xx
  venus: vdec: call session_continue in insufficient event
  venus: add common capability parser
  venus: helpers: make a commmon function for power_enable
  venus: core: delete not used flag for buffer mode
  venus: helpers: rename a helper function and use buffer mode from caps
  venus: add a helper function to set dynamic buffer mode
  venus: add helper function to set actual buffer size
  venus: delete no longer used bufmode flag from instance
  venus: helpers: add buffer type argument to a helper
  venus: helpers: add a new helper to set raw format
  venus: helpers,vdec,venc: add helpers to set work mode and core usage
  venus: helpers: extend set_num_bufs helper with one more argument
  venus: helpers: add a helper to return opb buffer sizes
  venus: vdec: get required input buffers as well
  venus: vdec: new function for output configuration
  venus: move frame size calculations in common place
  venus: implementing multi-stream support
  venus: add sdm845 compatible and resource data
  venus: add HEVC codec support

 .../devicetree/bindings/media/qcom,venus.txt       |   1 +
 drivers/media/platform/qcom/venus/Makefile         |   3 +-
 drivers/media/platform/qcom/venus/core.c           | 102 ++++
 drivers/media/platform/qcom/venus/core.h           |  91 ++--
 drivers/media/platform/qcom/venus/helpers.c        | 558 +++++++++++++++++++--
 drivers/media/platform/qcom/venus/helpers.h        |  23 +-
 drivers/media/platform/qcom/venus/hfi.c            |  12 +-
 drivers/media/platform/qcom/venus/hfi.h            |   9 +
 drivers/media/platform/qcom/venus/hfi_cmds.c       |  64 ++-
 drivers/media/platform/qcom/venus/hfi_helper.h     | 112 ++++-
 drivers/media/platform/qcom/venus/hfi_msgs.c       | 401 +++------------
 drivers/media/platform/qcom/venus/hfi_parser.c     | 290 +++++++++++
 drivers/media/platform/qcom/venus/hfi_parser.h     |  45 ++
 drivers/media/platform/qcom/venus/hfi_venus.c      |  69 +++
 drivers/media/platform/qcom/venus/hfi_venus_io.h   |  24 +
 drivers/media/platform/qcom/venus/vdec.c           | 324 +++++++-----
 drivers/media/platform/qcom/venus/venc.c           | 166 +++---
 17 files changed, 1641 insertions(+), 653 deletions(-)
 create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.c
 create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.h

-- 
2.14.1

[PATCH 01/28] venus: hfi_msgs: correct pointer increment

[Stanimir Varbanov]

Data pointer should be incremented by size of the structure not
the size of a pointer, correct the mistake.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi_msgs.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/venus/hfi_msgs.c
index 90c93d9603dc..589e1a6b36a9 100644
--- a/drivers/media/platform/qcom/venus/hfi_msgs.c
+++ b/drivers/media/platform/qcom/venus/hfi_msgs.c
@@ -60,14 +60,14 @@ static void event_seq_changed(struct venus_core *core, struct venus_inst *inst,
 			frame_sz = (struct hfi_framesize *)data_ptr;
 			event.width = frame_sz->width;
 			event.height = frame_sz->height;
-			data_ptr += sizeof(frame_sz);
+			data_ptr += sizeof(*frame_sz);
 			break;
 		case HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT:
 			data_ptr += sizeof(u32);
 			profile_level = (struct hfi_profile_level *)data_ptr;
 			event.profile = profile_level->profile;
 			event.level = profile_level->level;
-			data_ptr += sizeof(profile_level);
+			data_ptr += sizeof(*profile_level);
 			break;
 		default:
 			break;
-- 
2.14.1

Re: [PATCH 01/28] venus: hfi_msgs: correct pointer increment

[Tomasz Figa]

Hi Stanimir,

Thanks for the series. I'll be gradually reviewing subsequent patches. Stay
tuned. :)

On Tue, Apr 24, 2018 at 9:56 PM Stanimir Varbanov <
stanimir.varbanov@linaro.org> wrote:

> Data pointer should be incremented by size of the structure not
> the size of a pointer, correct the mistake.

> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>   drivers/media/platform/qcom/venus/hfi_msgs.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)

> diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c
b/drivers/media/platform/qcom/venus/hfi_msgs.c
> index 90c93d9603dc..589e1a6b36a9 100644
> --- a/drivers/media/platform/qcom/venus/hfi_msgs.c
> +++ b/drivers/media/platform/qcom/venus/hfi_msgs.c
> @@ -60,14 +60,14 @@ static void event_seq_changed(struct venus_core
*core, struct venus_inst *inst,
>                          frame_sz = (struct hfi_framesize *)data_ptr;
>                          event.width = frame_sz->width;
>                          event.height = frame_sz->height;
> -                       data_ptr += sizeof(frame_sz);
> +                       data_ptr += sizeof(*frame_sz);
>                          break;
>                  case HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT:
>                          data_ptr += sizeof(u32);
>                          profile_level = (struct hfi_profile_level
*)data_ptr;
>                          event.profile = profile_level->profile;
>                          event.level = profile_level->level;
> -                       data_ptr += sizeof(profile_level);
> +                       data_ptr += sizeof(*profile_level);
>                          break;
>                  default:
>                          break;

Reviewed-by: Tomasz Figa <tfiga@chromium.org>

Best regards,
Tomasz

Re: [PATCH 01/28] venus: hfi_msgs: correct pointer increment

[Stanimir Varbanov]

Hi Tomasz,

Thanks for the review!

On 05/18/2018 11:33 AM, Tomasz Figa wrote:
> Hi Stanimir,
> 
> Thanks for the series. I'll be gradually reviewing subsequent patches. Stay
> tuned. :)
> 

Please consider that there is a v2 of this patchset. :)

> 
> Reviewed-by: Tomasz Figa <tfiga@chromium.org>
> 

Thanks!

-- 
regards,
Stan

Re: [PATCH 01/28] venus: hfi_msgs: correct pointer increment

[Tomasz Figa]

On Fri, May 18, 2018 at 5:52 PM Stanimir Varbanov <
stanimir.varbanov@linaro.org> wrote:

> Hi Tomasz,

> Thanks for the review!

> On 05/18/2018 11:33 AM, Tomasz Figa wrote:
> > Hi Stanimir,
> >
> > Thanks for the series. I'll be gradually reviewing subsequent patches.
Stay
> > tuned. :)
> >

> Please consider that there is a v2 of this patchset. :)

Thanks for heads up. Looks like I missed it originally. Will move to v2
with my review.

Best regards,
Tomasz

[PATCH 02/28] venus: hfi: preparation to support venus 4xx

[Stanimir Varbanov]

This covers the differences between 1xx,3xx and 4xx.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h         |  4 ++
 drivers/media/platform/qcom/venus/helpers.c      | 37 +++++++----
 drivers/media/platform/qcom/venus/hfi_helper.h   | 84 ++++++++++++++++++++++--
 drivers/media/platform/qcom/venus/hfi_venus_io.h | 24 +++++++
 drivers/media/platform/qcom/venus/vdec.c         |  5 +-
 drivers/media/platform/qcom/venus/venc.c         |  5 +-
 6 files changed, 137 insertions(+), 22 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 0360d295f4c8..8d3e150800c9 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -305,6 +305,10 @@ struct venus_inst {
 	struct hfi_buffer_requirements bufreq[HFI_BUFFER_TYPE_MAX];
 };
 
+#define IS_V1(core)	((core)->res->hfi_version == HFI_VERSION_1XX)
+#define IS_V3(core)	((core)->res->hfi_version == HFI_VERSION_3XX)
+#define IS_V4(core)	((core)->res->hfi_version == HFI_VERSION_4XX)
+
 #define ctrl_to_inst(ctrl)	\
 	container_of((ctrl)->handler, struct venus_inst, ctrl_handler)
 
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 0ce9559a2924..d9065cc8a7d3 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -166,21 +166,37 @@ static int intbufs_unset_buffers(struct venus_inst *inst)
 	return ret;
 }
 
-static const unsigned int intbuf_types[] = {
-	HFI_BUFFER_INTERNAL_SCRATCH,
-	HFI_BUFFER_INTERNAL_SCRATCH_1,
-	HFI_BUFFER_INTERNAL_SCRATCH_2,
+static const unsigned int intbuf_types_1xx[] = {
+	HFI_BUFFER_INTERNAL_SCRATCH(HFI_VERSION_1XX),
+	HFI_BUFFER_INTERNAL_SCRATCH_1(HFI_VERSION_1XX),
+	HFI_BUFFER_INTERNAL_SCRATCH_2(HFI_VERSION_1XX),
+	HFI_BUFFER_INTERNAL_PERSIST,
+	HFI_BUFFER_INTERNAL_PERSIST_1,
+};
+
+static const unsigned int intbuf_types_4xx[] = {
+	HFI_BUFFER_INTERNAL_SCRATCH(HFI_VERSION_4XX),
+	HFI_BUFFER_INTERNAL_SCRATCH_1(HFI_VERSION_4XX),
+	HFI_BUFFER_INTERNAL_SCRATCH_2(HFI_VERSION_4XX),
 	HFI_BUFFER_INTERNAL_PERSIST,
 	HFI_BUFFER_INTERNAL_PERSIST_1,
 };
 
 static int intbufs_alloc(struct venus_inst *inst)
 {
-	unsigned int i;
+	size_t arr_sz;
+	size_t i;
 	int ret;
 
-	for (i = 0; i < ARRAY_SIZE(intbuf_types); i++) {
-		ret = intbufs_set_buffer(inst, intbuf_types[i]);
+	if (IS_V4(inst->core))
+		arr_sz = ARRAY_SIZE(intbuf_types_4xx);
+	else
+		arr_sz = ARRAY_SIZE(intbuf_types_1xx);
+
+	for (i = 0; i < arr_sz; i++) {
+		ret = intbufs_set_buffer(inst,
+			    IS_V4(inst->core) ? intbuf_types_4xx[i] :
+						intbuf_types_1xx[i]);
 		if (ret)
 			goto error;
 	}
@@ -257,12 +273,11 @@ static int load_scale_clocks(struct venus_core *core)
 
 set_freq:
 
-	if (core->res->hfi_version == HFI_VERSION_3XX) {
-		ret = clk_set_rate(clk, freq);
+	ret = clk_set_rate(clk, freq);
+
+	if (IS_V3(core) || IS_V4(core)) {
 		ret |= clk_set_rate(core->core0_clk, freq);
 		ret |= clk_set_rate(core->core1_clk, freq);
-	} else {
-		ret = clk_set_rate(clk, freq);
 	}
 
 	if (ret) {
diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/venus/hfi_helper.h
index 55d8eb21403a..1bc5aab1ce6b 100644
--- a/drivers/media/platform/qcom/venus/hfi_helper.h
+++ b/drivers/media/platform/qcom/venus/hfi_helper.h
@@ -121,6 +121,7 @@
 #define HFI_EXTRADATA_METADATA_FILLER			0x7fe00002
 
 #define HFI_INDEX_EXTRADATA_INPUT_CROP			0x0700000e
+#define HFI_INDEX_EXTRADATA_OUTPUT_CROP			0x0700000f
 #define HFI_INDEX_EXTRADATA_DIGITAL_ZOOM		0x07000010
 #define HFI_INDEX_EXTRADATA_ASPECT_RATIO		0x7f100003
 
@@ -376,13 +377,18 @@
 #define HFI_BUFFER_OUTPUT2			0x3
 #define HFI_BUFFER_INTERNAL_PERSIST		0x4
 #define HFI_BUFFER_INTERNAL_PERSIST_1		0x5
-#define HFI_BUFFER_INTERNAL_SCRATCH		0x1000001
-#define HFI_BUFFER_EXTRADATA_INPUT		0x1000002
-#define HFI_BUFFER_EXTRADATA_OUTPUT		0x1000003
-#define HFI_BUFFER_EXTRADATA_OUTPUT2		0x1000004
-#define HFI_BUFFER_INTERNAL_SCRATCH_1		0x1000005
-#define HFI_BUFFER_INTERNAL_SCRATCH_2		0x1000006
-
+#define HFI_BUFFER_INTERNAL_SCRATCH(ver)	\
+	(((ver) == HFI_VERSION_4XX) ? 0x6 : 0x1000001)
+#define HFI_BUFFER_INTERNAL_SCRATCH_1(ver)	\
+	(((ver) == HFI_VERSION_4XX) ? 0x7 : 0x1000005)
+#define HFI_BUFFER_INTERNAL_SCRATCH_2(ver)	\
+	(((ver) == HFI_VERSION_4XX) ? 0x8 : 0x1000006)
+#define HFI_BUFFER_EXTRADATA_INPUT(ver)		\
+	(((ver) == HFI_VERSION_4XX) ? 0xc : 0x1000002)
+#define HFI_BUFFER_EXTRADATA_OUTPUT(ver)	\
+	(((ver) == HFI_VERSION_4XX) ? 0xa : 0x1000003)
+#define HFI_BUFFER_EXTRADATA_OUTPUT2(ver)	\
+	(((ver) == HFI_VERSION_4XX) ? 0xb : 0x1000004)
 #define HFI_BUFFER_TYPE_MAX			11
 
 #define HFI_BUFFER_MODE_STATIC			0x1000001
@@ -424,12 +430,14 @@
 #define HFI_PROPERTY_PARAM_CODEC_MASK_SUPPORTED			0x100e
 #define HFI_PROPERTY_PARAM_MVC_BUFFER_LAYOUT			0x100f
 #define HFI_PROPERTY_PARAM_MAX_SESSIONS_SUPPORTED		0x1010
+#define HFI_PROPERTY_PARAM_WORK_MODE				0x1015
 
 /*
  * HFI_PROPERTY_CONFIG_COMMON_START
  * HFI_DOMAIN_BASE_COMMON + HFI_ARCH_COMMON_OFFSET + 0x2000
  */
 #define HFI_PROPERTY_CONFIG_FRAME_RATE				0x2001
+#define HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE			0x2002
 
 /*
  * HFI_PROPERTY_PARAM_VDEC_COMMON_START
@@ -438,6 +446,9 @@
 #define HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM			0x1003001
 #define HFI_PROPERTY_PARAM_VDEC_CONCEAL_COLOR			0x1003002
 #define HFI_PROPERTY_PARAM_VDEC_NONCP_OUTPUT2			0x1003003
+#define HFI_PROPERTY_PARAM_VDEC_PIXEL_BITDEPTH			0x1003007
+#define HFI_PROPERTY_PARAM_VDEC_PIC_STRUCT			0x1003009
+#define HFI_PROPERTY_PARAM_VDEC_COLOUR_SPACE			0x100300a
 
 /*
  * HFI_PROPERTY_CONFIG_VDEC_COMMON_START
@@ -518,6 +529,7 @@
 enum hfi_version {
 	HFI_VERSION_1XX,
 	HFI_VERSION_3XX,
+	HFI_VERSION_4XX
 };
 
 struct hfi_buffer_info {
@@ -767,12 +779,56 @@ struct hfi_framesize {
 	u32 height;
 };
 
+#define VIDC_CORE_ID_DEFAULT	0
+#define VIDC_CORE_ID_1		1
+#define VIDC_CORE_ID_2		2
+#define VIDC_CORE_ID_3		3
+
+struct hfi_videocores_usage_type {
+	u32 video_core_enable_mask;
+};
+
+#define VIDC_WORK_MODE_1	1
+#define VIDC_WORK_MODE_2	2
+
+struct hfi_video_work_mode {
+	u32 video_work_mode;
+};
+
 struct hfi_h264_vui_timing_info {
 	u32 enable;
 	u32 fixed_framerate;
 	u32 time_scale;
 };
 
+struct hfi_bit_depth {
+	u32 buffer_type;
+	u32 bit_depth;
+};
+
+struct hfi_picture_type {
+	u32 is_sync_frame;
+	u32 picture_type;
+};
+
+struct hfi_pic_struct {
+	u32 progressive_only;
+};
+
+struct hfi_colour_space {
+	u32 colour_space;
+};
+
+struct hfi_extradata_input_crop {
+	u32 size;
+	u32 version;
+	u32 port_index;
+	u32 left;
+	u32 top;
+	u32 width;
+	u32 height;
+};
+
 #define HFI_COLOR_FORMAT_MONOCHROME		0x01
 #define HFI_COLOR_FORMAT_NV12			0x02
 #define HFI_COLOR_FORMAT_NV21			0x03
@@ -961,6 +1017,12 @@ struct hfi_buffer_count_actual {
 	u32 count_actual;
 };
 
+struct hfi_buffer_count_actual_4xx {
+	u32 type;
+	u32 count_actual;
+	u32 count_min_host;
+};
+
 struct hfi_buffer_size_actual {
 	u32 type;
 	u32 size;
@@ -971,6 +1033,14 @@ struct hfi_buffer_display_hold_count_actual {
 	u32 hold_count;
 };
 
+/* HFI 4XX reorder the fields, use these macros */
+#define HFI_BUFREQ_HOLD_COUNT(bufreq, ver)	\
+	((ver) == HFI_VERSION_4XX ? 0 : (bufreq)->hold_count)
+#define HFI_BUFREQ_COUNT_MIN(bufreq, ver)	\
+	((ver) == HFI_VERSION_4XX ? (bufreq)->hold_count : (bufreq)->count_min)
+#define HFI_BUFREQ_COUNT_MIN_HOST(bufreq, ver)	\
+	((ver) == HFI_VERSION_4XX ? (bufreq)->count_min : 0)
+
 struct hfi_buffer_requirements {
 	u32 type;
 	u32 size;
diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h b/drivers/media/platform/qcom/venus/hfi_venus_io.h
index 98cc350113ab..76f47936d0fa 100644
--- a/drivers/media/platform/qcom/venus/hfi_venus_io.h
+++ b/drivers/media/platform/qcom/venus/hfi_venus_io.h
@@ -110,4 +110,28 @@
 #define WRAPPER_CPU_STATUS			(WRAPPER_BASE + 0x2014)
 #define WRAPPER_SW_RESET			(WRAPPER_BASE + 0x3000)
 
+/* Venus 4xx */
+#define WRAPPER_VCODEC0_MMCC_POWER_STATUS	(WRAPPER_BASE + 0x90)
+#define WRAPPER_VCODEC0_MMCC_POWER_CONTROL	(WRAPPER_BASE + 0x94)
+
+#define WRAPPER_VCODEC1_MMCC_POWER_STATUS	(WRAPPER_BASE + 0x110)
+#define WRAPPER_VCODEC1_MMCC_POWER_CONTROL	(WRAPPER_BASE + 0x114)
+
+/* vcodec noc error log registers */
+#define VCODEC_CORE0_VIDEO_NOC_BASE_OFFS		0x4000
+#define VCODEC_CORE1_VIDEO_NOC_BASE_OFFS		0xc000
+#define VCODEC_COREX_VIDEO_NOC_ERR_SWID_LOW_OFFS	0x500
+#define VCODEC_COREX_VIDEO_NOC_ERR_SWID_HIGH_OFFS	0x504
+#define VCODEC_COREX_VIDEO_NOC_ERR_MAINCTL_LOW_OFFS	0x508
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRVLD_LOW_OFFS	0x510
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRCLR_LOW_OFFS	0x518
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG0_LOW_OFFS	0x520
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG0_HIGH_OFFS	0x524
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG1_LOW_OFFS	0x528
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG1_HIGH_OFFS	0x52c
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG2_LOW_OFFS	0x530
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG2_HIGH_OFFS	0x534
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG3_LOW_OFFS	0x538
+#define VCODEC_COREX_VIDEO_NOC_ERR_ERRLOG3_HIGH_OFFS	0x53c
+
 #endif
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 49bbd1861d3a..261a51adeef2 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -689,6 +689,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
 
 static int vdec_verify_conf(struct venus_inst *inst)
 {
+	enum hfi_version ver = inst->core->res->hfi_version;
 	struct hfi_buffer_requirements bufreq;
 	int ret;
 
@@ -700,14 +701,14 @@ static int vdec_verify_conf(struct venus_inst *inst)
 		return ret;
 
 	if (inst->num_output_bufs < bufreq.count_actual ||
-	    inst->num_output_bufs < bufreq.count_min)
+	    inst->num_output_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
 		return -EINVAL;
 
 	ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);
 	if (ret)
 		return ret;
 
-	if (inst->num_input_bufs < bufreq.count_min)
+	if (inst->num_input_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
 		return -EINVAL;
 
 	return 0;
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 6b2ce479584e..947001170a77 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -892,6 +892,7 @@ static int venc_queue_setup(struct vb2_queue *q,
 
 static int venc_verify_conf(struct venus_inst *inst)
 {
+	enum hfi_version ver = inst->core->res->hfi_version;
 	struct hfi_buffer_requirements bufreq;
 	int ret;
 
@@ -903,7 +904,7 @@ static int venc_verify_conf(struct venus_inst *inst)
 		return ret;
 
 	if (inst->num_output_bufs < bufreq.count_actual ||
-	    inst->num_output_bufs < bufreq.count_min)
+	    inst->num_output_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
 		return -EINVAL;
 
 	ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);
@@ -911,7 +912,7 @@ static int venc_verify_conf(struct venus_inst *inst)
 		return ret;
 
 	if (inst->num_input_bufs < bufreq.count_actual ||
-	    inst->num_input_bufs < bufreq.count_min)
+	    inst->num_input_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver))
 		return -EINVAL;
 
 	return 0;
-- 
2.14.1

[PATCH 03/28] venus: hfi: update sequence event to handle more properties

[Stanimir Varbanov]

HFI version 4xx can pass more properties in the sequence change
event, extend the event structure with them.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi.h      |  9 ++++++
 drivers/media/platform/qcom/venus/hfi_msgs.c | 46 ++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/venus/hfi.h
index 5466b7d60dd0..21376d93170f 100644
--- a/drivers/media/platform/qcom/venus/hfi.h
+++ b/drivers/media/platform/qcom/venus/hfi.h
@@ -74,6 +74,15 @@ struct hfi_event_data {
 	u32 tag;
 	u32 profile;
 	u32 level;
+	u32 bit_depth;
+	u32 pic_struct;
+	u32 colour_space;
+	u32 entropy_mode;
+	u32 buf_count;
+	struct {
+		u32 left, top;
+		u32 width, height;
+	} input_crop;
 };
 
 /* define core states */
diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/venus/hfi_msgs.c
index 589e1a6b36a9..5970e9b1716b 100644
--- a/drivers/media/platform/qcom/venus/hfi_msgs.c
+++ b/drivers/media/platform/qcom/venus/hfi_msgs.c
@@ -25,10 +25,17 @@
 static void event_seq_changed(struct venus_core *core, struct venus_inst *inst,
 			      struct hfi_msg_event_notify_pkt *pkt)
 {
+	enum hfi_version ver = core->res->hfi_version;
 	struct hfi_event_data event = {0};
 	int num_properties_changed;
 	struct hfi_framesize *frame_sz;
 	struct hfi_profile_level *profile_level;
+	struct hfi_bit_depth *pixel_depth;
+	struct hfi_pic_struct *pic_struct;
+	struct hfi_colour_space *colour_info;
+	struct hfi_buffer_requirements *bufreq;
+	struct hfi_extradata_input_crop *crop;
+	u32 entropy_mode = 0;
 	u8 *data_ptr;
 	u32 ptype;
 
@@ -69,6 +76,45 @@ static void event_seq_changed(struct venus_core *core, struct venus_inst *inst,
 			event.level = profile_level->level;
 			data_ptr += sizeof(*profile_level);
 			break;
+		case HFI_PROPERTY_PARAM_VDEC_PIXEL_BITDEPTH:
+			data_ptr += sizeof(u32);
+			pixel_depth = (struct hfi_bit_depth *)data_ptr;
+			event.bit_depth = pixel_depth->bit_depth;
+			data_ptr += sizeof(*pixel_depth);
+			break;
+		case HFI_PROPERTY_PARAM_VDEC_PIC_STRUCT:
+			data_ptr += sizeof(u32);
+			pic_struct = (struct hfi_pic_struct *)data_ptr;
+			event.pic_struct = pic_struct->progressive_only;
+			data_ptr += sizeof(*pic_struct);
+			break;
+		case HFI_PROPERTY_PARAM_VDEC_COLOUR_SPACE:
+			data_ptr += sizeof(u32);
+			colour_info = (struct hfi_colour_space *)data_ptr;
+			event.colour_space = colour_info->colour_space;
+			data_ptr += sizeof(*colour_info);
+			break;
+		case HFI_PROPERTY_CONFIG_VDEC_ENTROPY:
+			data_ptr += sizeof(u32);
+			entropy_mode = *(u32 *)data_ptr;
+			event.entropy_mode = entropy_mode;
+			data_ptr += sizeof(u32);
+			break;
+		case HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS:
+			data_ptr += sizeof(u32);
+			bufreq = (struct hfi_buffer_requirements *)data_ptr;
+			event.buf_count = HFI_BUFREQ_COUNT_MIN(bufreq, ver);
+			data_ptr += sizeof(*bufreq);
+			break;
+		case HFI_INDEX_EXTRADATA_INPUT_CROP:
+			data_ptr += sizeof(u32);
+			crop = (struct hfi_extradata_input_crop *)data_ptr;
+			event.input_crop.left = crop->left;
+			event.input_crop.top = crop->top;
+			event.input_crop.width = crop->width;
+			event.input_crop.height = crop->height;
+			data_ptr += sizeof(*crop);
+			break;
 		default:
 			break;
 		}
-- 
2.14.1

[PATCH 04/28] venus: hfi_cmds: add set_properties for 4xx version

[Stanimir Varbanov]

Adds set_properties method to handle newer 4xx properties and
fall-back to 3xx for the rest.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi_cmds.c | 64 +++++++++++++++++++++++++++-
 1 file changed, 63 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c b/drivers/media/platform/qcom/venus/hfi_cmds.c
index 1cfeb7743041..6bd287154796 100644
--- a/drivers/media/platform/qcom/venus/hfi_cmds.c
+++ b/drivers/media/platform/qcom/venus/hfi_cmds.c
@@ -1166,6 +1166,65 @@ pkt_session_set_property_3xx(struct hfi_session_set_property_pkt *pkt,
 	return ret;
 }
 
+static int
+pkt_session_set_property_4xx(struct hfi_session_set_property_pkt *pkt,
+			     void *cookie, u32 ptype, void *pdata)
+{
+	void *prop_data;
+	int ret = 0;
+
+	if (!pkt || !cookie || !pdata)
+		return -EINVAL;
+
+	prop_data = &pkt->data[1];
+
+	pkt->shdr.hdr.size = sizeof(*pkt);
+	pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_SET_PROPERTY;
+	pkt->shdr.session_id = hash32_ptr(cookie);
+	pkt->num_properties = 1;
+	pkt->data[0] = ptype;
+
+	/*
+	 * Any session set property which is different in 3XX packetization
+	 * should be added as a new case below. All unchanged session set
+	 * properties will be handled in the default case.
+	 */
+	switch (ptype) {
+	case HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL: {
+		struct hfi_buffer_count_actual *in = pdata;
+		struct hfi_buffer_count_actual_4xx *count = prop_data;
+
+		count->count_actual = in->count_actual;
+		count->type = in->type;
+		count->count_min_host = in->count_actual;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*count);
+		break;
+	}
+	case HFI_PROPERTY_PARAM_WORK_MODE: {
+		struct hfi_video_work_mode *in = pdata, *wm = prop_data;
+
+		wm->video_work_mode = in->video_work_mode;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*wm);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE: {
+		struct hfi_videocores_usage_type *in = pdata, *cu = prop_data;
+
+		cu->video_core_enable_mask = in->video_core_enable_mask;
+		pkt->shdr.hdr.size += sizeof(u32) + sizeof(*cu);
+		break;
+	}
+	case HFI_PROPERTY_CONFIG_VENC_MAX_BITRATE:
+		/* not implemented on Venus 4xx */
+		break;
+	default:
+		ret = pkt_session_set_property_3xx(pkt, cookie, ptype, pdata);
+		break;
+	}
+
+	return ret;
+}
+
 int pkt_session_get_property(struct hfi_session_get_property_pkt *pkt,
 			     void *cookie, u32 ptype)
 {
@@ -1181,7 +1240,10 @@ int pkt_session_set_property(struct hfi_session_set_property_pkt *pkt,
 	if (hfi_ver == HFI_VERSION_1XX)
 		return pkt_session_set_property_1x(pkt, cookie, ptype, pdata);
 
-	return pkt_session_set_property_3xx(pkt, cookie, ptype, pdata);
+	if (hfi_ver == HFI_VERSION_3XX)
+		return pkt_session_set_property_3xx(pkt, cookie, ptype, pdata);
+
+	return pkt_session_set_property_4xx(pkt, cookie, ptype, pdata);
 }
 
 void pkt_set_version(enum hfi_version version)
-- 
2.14.1

[PATCH 05/28] venus: hfi: support session continue for 4xx version

[Stanimir Varbanov]

This makes possible to handle session_continue for 4xx as well.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
index bca894a00c07..cbc6fad05e47 100644
--- a/drivers/media/platform/qcom/venus/hfi.c
+++ b/drivers/media/platform/qcom/venus/hfi.c
@@ -312,7 +312,7 @@ int hfi_session_continue(struct venus_inst *inst)
 {
 	struct venus_core *core = inst->core;
 
-	if (core->res->hfi_version != HFI_VERSION_3XX)
+	if (core->res->hfi_version == HFI_VERSION_1XX)
 		return 0;
 
 	return core->ops->session_continue(inst);
-- 
2.14.1

[PATCH 06/28] venus: hfi: handle buffer output2 type as well

[Stanimir Varbanov]

This adds handling of buffers of type OUTPUT2 which is needed to
support Venus 4xx version.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi.c      | 3 ++-
 drivers/media/platform/qcom/venus/hfi_msgs.c | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
index cbc6fad05e47..a570fdad0de0 100644
--- a/drivers/media/platform/qcom/venus/hfi.c
+++ b/drivers/media/platform/qcom/venus/hfi.c
@@ -473,7 +473,8 @@ int hfi_session_process_buf(struct venus_inst *inst, struct hfi_frame_data *fd)
 
 	if (fd->buffer_type == HFI_BUFFER_INPUT)
 		return ops->session_etb(inst, fd);
-	else if (fd->buffer_type == HFI_BUFFER_OUTPUT)
+	else if (fd->buffer_type == HFI_BUFFER_OUTPUT ||
+		 fd->buffer_type == HFI_BUFFER_OUTPUT2)
 		return ops->session_ftb(inst, fd);
 
 	return -EINVAL;
diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/venus/hfi_msgs.c
index 5970e9b1716b..023802e62833 100644
--- a/drivers/media/platform/qcom/venus/hfi_msgs.c
+++ b/drivers/media/platform/qcom/venus/hfi_msgs.c
@@ -825,7 +825,8 @@ static void hfi_session_ftb_done(struct venus_core *core,
 		error = HFI_ERR_SESSION_INVALID_PARAMETER;
 	}
 
-	if (buffer_type != HFI_BUFFER_OUTPUT)
+	if (buffer_type != HFI_BUFFER_OUTPUT &&
+	    buffer_type != HFI_BUFFER_OUTPUT2)
 		goto done;
 
 	if (hfi_flags & HFI_BUFFERFLAG_EOS)
-- 
2.14.1

[PATCH 07/28] venus: hfi_venus: add halt AXI support for Venus 4xx

[Stanimir Varbanov]

Add AXI halt support for version 4xx by using venus wrapper
registers.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi_venus.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/venus/hfi_venus.c
index 734ce11b0ed0..53546174aab8 100644
--- a/drivers/media/platform/qcom/venus/hfi_venus.c
+++ b/drivers/media/platform/qcom/venus/hfi_venus.c
@@ -532,6 +532,23 @@ static int venus_halt_axi(struct venus_hfi_device *hdev)
 	u32 val;
 	int ret;
 
+	if (hdev->core->res->hfi_version == HFI_VERSION_4XX) {
+		val = venus_readl(hdev, WRAPPER_CPU_AXI_HALT);
+		val |= BIT(16);
+		venus_writel(hdev, WRAPPER_CPU_AXI_HALT, val);
+
+		ret = readl_poll_timeout(base + WRAPPER_CPU_AXI_HALT_STATUS,
+					 val, val & BIT(24),
+					 POLL_INTERVAL_US,
+					 VBIF_AXI_HALT_ACK_TIMEOUT_US);
+		if (ret) {
+			dev_err(dev, "AXI bus port halt timeout\n");
+			return ret;
+		}
+
+		return 0;
+	}
+
 	/* Halt AXI and AXI IMEM VBIF Access */
 	val = venus_readl(hdev, VBIF_AXI_HALT_CTRL0);
 	val |= VBIF_AXI_HALT_CTRL0_HALT_REQ;
-- 
2.14.1

[PATCH 08/28] venus: hfi_venus: add suspend function for 4xx version

[Stanimir Varbanov]

This adds suspend (power collapse) function with slightly
different order of calls comparing with Venus 3xx.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/hfi_venus.c | 52 +++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/venus/hfi_venus.c
index 53546174aab8..f61d34bf61b4 100644
--- a/drivers/media/platform/qcom/venus/hfi_venus.c
+++ b/drivers/media/platform/qcom/venus/hfi_venus.c
@@ -1443,6 +1443,55 @@ static int venus_suspend_1xx(struct venus_core *core)
 	return 0;
 }
 
+static int venus_suspend_4xx(struct venus_core *core)
+{
+	struct venus_hfi_device *hdev = to_hfi_priv(core);
+	struct device *dev = core->dev;
+	u32 val;
+	int ret;
+
+	if (!hdev->power_enabled || hdev->suspended)
+		return 0;
+
+	mutex_lock(&hdev->lock);
+	ret = venus_is_valid_state(hdev);
+	mutex_unlock(&hdev->lock);
+
+	if (!ret) {
+		dev_err(dev, "bad state, cannot suspend\n");
+		return -EINVAL;
+	}
+
+	ret = venus_prepare_power_collapse(hdev, false);
+	if (ret) {
+		dev_err(dev, "prepare for power collapse fail (%d)\n", ret);
+		return ret;
+	}
+
+	ret = readl_poll_timeout(core->base + CPU_CS_SCIACMDARG0, val,
+				 val & CPU_CS_SCIACMDARG0_PC_READY,
+				 POLL_INTERVAL_US, 100000);
+	if (ret) {
+		dev_err(dev, "Polling power collapse ready timed out\n");
+		return ret;
+	}
+
+	mutex_lock(&hdev->lock);
+
+	ret = venus_power_off(hdev);
+	if (ret) {
+		dev_err(dev, "venus_power_off (%d)\n", ret);
+		mutex_unlock(&hdev->lock);
+		return ret;
+	}
+
+	hdev->suspended = true;
+
+	mutex_unlock(&hdev->lock);
+
+	return 0;
+}
+
 static int venus_suspend_3xx(struct venus_core *core)
 {
 	struct venus_hfi_device *hdev = to_hfi_priv(core);
@@ -1507,6 +1556,9 @@ static int venus_suspend(struct venus_core *core)
 	if (core->res->hfi_version == HFI_VERSION_3XX)
 		return venus_suspend_3xx(core);
 
+	if (core->res->hfi_version == HFI_VERSION_4XX)
+		return venus_suspend_4xx(core);
+
 	return venus_suspend_1xx(core);
 }
 
-- 
2.14.1

Re: [PATCH 08/28] venus: hfi_venus: add suspend function for 4xx version

[vgarodia]

Hello Stanimir,

On 2018-04-24 18:14, Stanimir Varbanov wrote:
> This adds suspend (power collapse) function with slightly
> different order of calls comparing with Venus 3xx.
> 
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/hfi_venus.c | 52 
> +++++++++++++++++++++++++++
>  1 file changed, 52 insertions(+)
> 
> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c
> b/drivers/media/platform/qcom/venus/hfi_venus.c
> index 53546174aab8..f61d34bf61b4 100644
> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
> +++ b/drivers/media/platform/qcom/venus/hfi_venus.c
> @@ -1443,6 +1443,55 @@ static int venus_suspend_1xx(struct venus_core 
> *core)
>  	return 0;
>  }
> 
> +static int venus_suspend_4xx(struct venus_core *core)
> +{
> +	struct venus_hfi_device *hdev = to_hfi_priv(core);
> +	struct device *dev = core->dev;
> +	u32 val;
> +	int ret;
> +
> +	if (!hdev->power_enabled || hdev->suspended)
> +		return 0;
> +
> +	mutex_lock(&hdev->lock);
> +	ret = venus_is_valid_state(hdev);
> +	mutex_unlock(&hdev->lock);
> +
> +	if (!ret) {
> +		dev_err(dev, "bad state, cannot suspend\n");
> +		return -EINVAL;
> +	}
> +
> +	ret = venus_prepare_power_collapse(hdev, false);
> +	if (ret) {
> +		dev_err(dev, "prepare for power collapse fail (%d)\n", ret);
> +		return ret;
> +	}
> +
> +	ret = readl_poll_timeout(core->base + CPU_CS_SCIACMDARG0, val,
> +				 val & CPU_CS_SCIACMDARG0_PC_READY,
> +				 POLL_INTERVAL_US, 100000);
> +	if (ret) {
> +		dev_err(dev, "Polling power collapse ready timed out\n");
> +		return ret;
> +	}
> +
> +	mutex_lock(&hdev->lock);
> +
> +	ret = venus_power_off(hdev);
> +	if (ret) {
> +		dev_err(dev, "venus_power_off (%d)\n", ret);
> +		mutex_unlock(&hdev->lock);
> +		return ret;
> +	}
> +
> +	hdev->suspended = true;
> +
> +	mutex_unlock(&hdev->lock);
> +
> +	return 0;
> +}
> +
>  static int venus_suspend_3xx(struct venus_core *core)
>  {
>  	struct venus_hfi_device *hdev = to_hfi_priv(core);
> @@ -1507,6 +1556,9 @@ static int venus_suspend(struct venus_core *core)
>  	if (core->res->hfi_version == HFI_VERSION_3XX)
>  		return venus_suspend_3xx(core);
> 
> +	if (core->res->hfi_version == HFI_VERSION_4XX)
> +		return venus_suspend_4xx(core);
> +
>  	return venus_suspend_1xx(core);
>  }

Let me brief on the power collapse sequence for Venus_4xx
1. Host checks for ARM9 and Video core to be idle.
    This can be done by checking for WFI bit (bit 0) in CPU status 
register for ARM9 and by checking bit 30 in Control status reg for video 
core/s.
2. Host then sends command to ARM9 to prepare for power collapse.
3. Host then checks for WFI bit and PC_READY bit before withdrawing 
going for power off.

As per this patch, host is preparing for power collapse without checking 
for #1.
Update the code to handle #3.

Thanks
Vikash

Re: [PATCH 08/28] venus: hfi_venus: add suspend function for 4xx version

[Stanimir Varbanov]

Hi Vikash,

On 05/02/2018 09:07 AM, vgarodia@codeaurora.org wrote:
> Hello Stanimir,
> 
> On 2018-04-24 18:14, Stanimir Varbanov wrote:
>> This adds suspend (power collapse) function with slightly
>> different order of calls comparing with Venus 3xx.
>>
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>  drivers/media/platform/qcom/venus/hfi_venus.c | 52
>> +++++++++++++++++++++++++++
>>  1 file changed, 52 insertions(+)
>>
>> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c
>> b/drivers/media/platform/qcom/venus/hfi_venus.c
>> index 53546174aab8..f61d34bf61b4 100644
>> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
>> +++ b/drivers/media/platform/qcom/venus/hfi_venus.c
>> @@ -1443,6 +1443,55 @@ static int venus_suspend_1xx(struct venus_core
>> *core)
>>      return 0;
>>  }
>>
>> +static int venus_suspend_4xx(struct venus_core *core)
>> +{
>> +    struct venus_hfi_device *hdev = to_hfi_priv(core);
>> +    struct device *dev = core->dev;
>> +    u32 val;
>> +    int ret;
>> +
>> +    if (!hdev->power_enabled || hdev->suspended)
>> +        return 0;
>> +
>> +    mutex_lock(&hdev->lock);
>> +    ret = venus_is_valid_state(hdev);
>> +    mutex_unlock(&hdev->lock);
>> +
>> +    if (!ret) {
>> +        dev_err(dev, "bad state, cannot suspend\n");
>> +        return -EINVAL;
>> +    }
>> +
>> +    ret = venus_prepare_power_collapse(hdev, false);
>> +    if (ret) {
>> +        dev_err(dev, "prepare for power collapse fail (%d)\n", ret);
>> +        return ret;
>> +    }
>> +
>> +    ret = readl_poll_timeout(core->base + CPU_CS_SCIACMDARG0, val,
>> +                 val & CPU_CS_SCIACMDARG0_PC_READY,
>> +                 POLL_INTERVAL_US, 100000);
>> +    if (ret) {
>> +        dev_err(dev, "Polling power collapse ready timed out\n");
>> +        return ret;
>> +    }
>> +
>> +    mutex_lock(&hdev->lock);
>> +
>> +    ret = venus_power_off(hdev);
>> +    if (ret) {
>> +        dev_err(dev, "venus_power_off (%d)\n", ret);
>> +        mutex_unlock(&hdev->lock);
>> +        return ret;
>> +    }
>> +
>> +    hdev->suspended = true;
>> +
>> +    mutex_unlock(&hdev->lock);
>> +
>> +    return 0;
>> +}
>> +
>>  static int venus_suspend_3xx(struct venus_core *core)
>>  {
>>      struct venus_hfi_device *hdev = to_hfi_priv(core);
>> @@ -1507,6 +1556,9 @@ static int venus_suspend(struct venus_core *core)
>>      if (core->res->hfi_version == HFI_VERSION_3XX)
>>          return venus_suspend_3xx(core);
>>
>> +    if (core->res->hfi_version == HFI_VERSION_4XX)
>> +        return venus_suspend_4xx(core);
>> +
>>      return venus_suspend_1xx(core);
>>  }
> 
> Let me brief on the power collapse sequence for Venus_4xx
> 1. Host checks for ARM9 and Video core to be idle.
>    This can be done by checking for WFI bit (bit 0) in CPU status
> register for ARM9 and by checking bit 30 in Control status reg for video
> core/s.
> 2. Host then sends command to ARM9 to prepare for power collapse.
> 3. Host then checks for WFI bit and PC_READY bit before withdrawing
> going for power off.
> 
> As per this patch, host is preparing for power collapse without checking
> for #1.
> Update the code to handle #3.

This looks similar to suspend for Venus 3xx. Can you confirm that the
sequence of checks for 4xx is the same as 3xx?

-- 
regards,
Stan

Re: [PATCH 08/28] venus: hfi_venus: add suspend function for 4xx version

[Vikash Garodia]

Hi Stanimir,

On 2018-05-09 16:45, Stanimir Varbanov wrote:
> Hi Vikash,
> 
> On 05/02/2018 09:07 AM, vgarodia@codeaurora.org wrote:
>> Hello Stanimir,
>> 
>> On 2018-04-24 18:14, Stanimir Varbanov wrote:
>>> This adds suspend (power collapse) function with slightly
>>> different order of calls comparing with Venus 3xx.
>>> 
>>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>>> ---
>>>  drivers/media/platform/qcom/venus/hfi_venus.c | 52
>>> +++++++++++++++++++++++++++
>>>  1 file changed, 52 insertions(+)
>>> 
>>> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c
>>> b/drivers/media/platform/qcom/venus/hfi_venus.c
>>> index 53546174aab8..f61d34bf61b4 100644
>>> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
>>> +++ b/drivers/media/platform/qcom/venus/hfi_venus.c
>>> @@ -1443,6 +1443,55 @@ static int venus_suspend_1xx(struct venus_core
>>> *core)
>>>      return 0;
>>>  }
>>> 
>>> +static int venus_suspend_4xx(struct venus_core *core)
>>> +{
>>> +    struct venus_hfi_device *hdev = to_hfi_priv(core);
>>> +    struct device *dev = core->dev;
>>> +    u32 val;
>>> +    int ret;
>>> +
>>> +    if (!hdev->power_enabled || hdev->suspended)
>>> +        return 0;
>>> +
>>> +    mutex_lock(&hdev->lock);
>>> +    ret = venus_is_valid_state(hdev);
>>> +    mutex_unlock(&hdev->lock);
>>> +
>>> +    if (!ret) {
>>> +        dev_err(dev, "bad state, cannot suspend\n");
>>> +        return -EINVAL;
>>> +    }
>>> +
>>> +    ret = venus_prepare_power_collapse(hdev, false);
>>> +    if (ret) {
>>> +        dev_err(dev, "prepare for power collapse fail (%d)\n", ret);
>>> +        return ret;
>>> +    }
>>> +
>>> +    ret = readl_poll_timeout(core->base + CPU_CS_SCIACMDARG0, val,
>>> +                 val & CPU_CS_SCIACMDARG0_PC_READY,
>>> +                 POLL_INTERVAL_US, 100000);
>>> +    if (ret) {
>>> +        dev_err(dev, "Polling power collapse ready timed out\n");
>>> +        return ret;
>>> +    }
>>> +
>>> +    mutex_lock(&hdev->lock);
>>> +
>>> +    ret = venus_power_off(hdev);
>>> +    if (ret) {
>>> +        dev_err(dev, "venus_power_off (%d)\n", ret);
>>> +        mutex_unlock(&hdev->lock);
>>> +        return ret;
>>> +    }
>>> +
>>> +    hdev->suspended = true;
>>> +
>>> +    mutex_unlock(&hdev->lock);
>>> +
>>> +    return 0;
>>> +}
>>> +
>>>  static int venus_suspend_3xx(struct venus_core *core)
>>>  {
>>>      struct venus_hfi_device *hdev = to_hfi_priv(core);
>>> @@ -1507,6 +1556,9 @@ static int venus_suspend(struct venus_core 
>>> *core)
>>>      if (core->res->hfi_version == HFI_VERSION_3XX)
>>>          return venus_suspend_3xx(core);
>>> 
>>> +    if (core->res->hfi_version == HFI_VERSION_4XX)
>>> +        return venus_suspend_4xx(core);
>>> +
>>>      return venus_suspend_1xx(core);
>>>  }
>> 
>> Let me brief on the power collapse sequence for Venus_4xx
>> 1. Host checks for ARM9 and Video core to be idle.
>>    This can be done by checking for WFI bit (bit 0) in CPU status
>> register for ARM9 and by checking bit 30 in Control status reg for 
>> video
>> core/s.
>> 2. Host then sends command to ARM9 to prepare for power collapse.
>> 3. Host then checks for WFI bit and PC_READY bit before withdrawing
>> going for power off.
>> 
>> As per this patch, host is preparing for power collapse without 
>> checking
>> for #1.
>> Update the code to handle #3.
> 
> This looks similar to suspend for Venus 3xx. Can you confirm that the
> sequence of checks for 4xx is the same as 3xx?

Do you mean the driver implementation for Suspend Venus 3xx or the 
hardware
expectation for Venus 3xx ? If hardware expectation wise, the sequence 
is
same for 3xx and 4xx.
In the suspend implementation for 3xx, i see that the host just reads 
the WFI
and idle status bits, but does not validate those bit value before 
preparing
Venus for power collapse. Sequence #2 and #3 is followed for Venus 3xx
implementation.

Re: [PATCH 08/28] venus: hfi_venus: add suspend function for 4xx version

[Stanimir Varbanov]

Hi,

On 05/09/2018 05:14 PM, Vikash Garodia wrote:
> Hi Stanimir,
> 
> On 2018-05-09 16:45, Stanimir Varbanov wrote:
>> Hi Vikash,
>>
>> On 05/02/2018 09:07 AM, vgarodia@codeaurora.org wrote:
>>> Hello Stanimir,
>>>
>>> On 2018-04-24 18:14, Stanimir Varbanov wrote:
>>>> This adds suspend (power collapse) function with slightly
>>>> different order of calls comparing with Venus 3xx.
>>>>
>>>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>>>> ---
>>>>  drivers/media/platform/qcom/venus/hfi_venus.c | 52
>>>> +++++++++++++++++++++++++++
>>>>  1 file changed, 52 insertions(+)
>>>>
>>>> diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c
>>>> b/drivers/media/platform/qcom/venus/hfi_venus.c
>>>> index 53546174aab8..f61d34bf61b4 100644
>>>> --- a/drivers/media/platform/qcom/venus/hfi_venus.c
>>>> +++ b/drivers/media/platform/qcom/venus/hfi_venus.c
>>>> @@ -1443,6 +1443,55 @@ static int venus_suspend_1xx(struct venus_core
>>>> *core)
>>>>      return 0;
>>>>  }
>>>>
>>>> +static int venus_suspend_4xx(struct venus_core *core)
>>>> +{
>>>> +    struct venus_hfi_device *hdev = to_hfi_priv(core);
>>>> +    struct device *dev = core->dev;
>>>> +    u32 val;
>>>> +    int ret;
>>>> +
>>>> +    if (!hdev->power_enabled || hdev->suspended)
>>>> +        return 0;
>>>> +
>>>> +    mutex_lock(&hdev->lock);
>>>> +    ret = venus_is_valid_state(hdev);
>>>> +    mutex_unlock(&hdev->lock);
>>>> +
>>>> +    if (!ret) {
>>>> +        dev_err(dev, "bad state, cannot suspend\n");
>>>> +        return -EINVAL;
>>>> +    }
>>>> +
>>>> +    ret = venus_prepare_power_collapse(hdev, false);
>>>> +    if (ret) {
>>>> +        dev_err(dev, "prepare for power collapse fail (%d)\n", ret);
>>>> +        return ret;
>>>> +    }
>>>> +
>>>> +    ret = readl_poll_timeout(core->base + CPU_CS_SCIACMDARG0, val,
>>>> +                 val & CPU_CS_SCIACMDARG0_PC_READY,
>>>> +                 POLL_INTERVAL_US, 100000);
>>>> +    if (ret) {
>>>> +        dev_err(dev, "Polling power collapse ready timed out\n");
>>>> +        return ret;
>>>> +    }
>>>> +
>>>> +    mutex_lock(&hdev->lock);
>>>> +
>>>> +    ret = venus_power_off(hdev);
>>>> +    if (ret) {
>>>> +        dev_err(dev, "venus_power_off (%d)\n", ret);
>>>> +        mutex_unlock(&hdev->lock);
>>>> +        return ret;
>>>> +    }
>>>> +
>>>> +    hdev->suspended = true;
>>>> +
>>>> +    mutex_unlock(&hdev->lock);
>>>> +
>>>> +    return 0;
>>>> +}
>>>> +
>>>>  static int venus_suspend_3xx(struct venus_core *core)
>>>>  {
>>>>      struct venus_hfi_device *hdev = to_hfi_priv(core);
>>>> @@ -1507,6 +1556,9 @@ static int venus_suspend(struct venus_core *core)
>>>>      if (core->res->hfi_version == HFI_VERSION_3XX)
>>>>          return venus_suspend_3xx(core);
>>>>
>>>> +    if (core->res->hfi_version == HFI_VERSION_4XX)
>>>> +        return venus_suspend_4xx(core);
>>>> +
>>>>      return venus_suspend_1xx(core);
>>>>  }
>>>
>>> Let me brief on the power collapse sequence for Venus_4xx
>>> 1. Host checks for ARM9 and Video core to be idle.
>>>    This can be done by checking for WFI bit (bit 0) in CPU status
>>> register for ARM9 and by checking bit 30 in Control status reg for video
>>> core/s.
>>> 2. Host then sends command to ARM9 to prepare for power collapse.
>>> 3. Host then checks for WFI bit and PC_READY bit before withdrawing
>>> going for power off.
>>>
>>> As per this patch, host is preparing for power collapse without checking
>>> for #1.
>>> Update the code to handle #3.
>>
>> This looks similar to suspend for Venus 3xx. Can you confirm that the
>> sequence of checks for 4xx is the same as 3xx?
> 
> Do you mean the driver implementation for Suspend Venus 3xx or the hardware
> expectation for Venus 3xx ? If hardware expectation wise, the sequence is
> same for 3xx and 4xx.
> In the suspend implementation for 3xx, i see that the host just reads
> the WFI
> and idle status bits, but does not validate those bit value before
> preparing
> Venus for power collapse. Sequence #2 and #3 is followed for Venus 3xx
> implementation.

OK, than we can reuse venus_suspend_3xx function for 4xx, just need to
fix WFI and idle bit for #1.

-- 
regards,
Stan

[PATCH 09/28] venus: venc,vdec: adds clocks needed for venus 4xx

[Stanimir Varbanov]

This extends the clocks number to support suspend and resume
on Venus version 4xx.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h |  4 +--
 drivers/media/platform/qcom/venus/vdec.c | 42 ++++++++++++++++++++++++++------
 drivers/media/platform/qcom/venus/venc.c | 42 ++++++++++++++++++++++++++------
 3 files changed, 72 insertions(+), 16 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 8d3e150800c9..b5b9a84e9155 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -92,8 +92,8 @@ struct venus_core {
 	void __iomem *base;
 	int irq;
 	struct clk *clks[VIDC_CLKS_NUM_MAX];
-	struct clk *core0_clk;
-	struct clk *core1_clk;
+	struct clk *core0_clk, *core0_bus_clk;
+	struct clk *core1_clk, *core1_bus_clk;
 	struct video_device *vdev_dec;
 	struct video_device *vdev_enc;
 	struct v4l2_device v4l2_dev;
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 261a51adeef2..c45452634e7e 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -1081,12 +1081,18 @@ static int vdec_probe(struct platform_device *pdev)
 	if (!core)
 		return -EPROBE_DEFER;
 
-	if (core->res->hfi_version == HFI_VERSION_3XX) {
+	if (IS_V3(core) || IS_V4(core)) {
 		core->core0_clk = devm_clk_get(dev, "core");
 		if (IS_ERR(core->core0_clk))
 			return PTR_ERR(core->core0_clk);
 	}
 
+	if (IS_V4(core)) {
+		core->core0_bus_clk = devm_clk_get(dev, "bus");
+		if (IS_ERR(core->core0_bus_clk))
+			return PTR_ERR(core->core0_bus_clk);
+	}
+
 	platform_set_drvdata(pdev, core);
 
 	vdev = video_device_alloc();
@@ -1132,12 +1138,23 @@ static __maybe_unused int vdec_runtime_suspend(struct device *dev)
 {
 	struct venus_core *core = dev_get_drvdata(dev);
 
-	if (core->res->hfi_version == HFI_VERSION_1XX)
+	if (IS_V1(core))
 		return 0;
 
-	writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	if (IS_V3(core))
+		writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(0, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+
+	if (IS_V4(core))
+		clk_disable_unprepare(core->core0_bus_clk);
+
 	clk_disable_unprepare(core->core0_clk);
-	writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+
+	if (IS_V3(core))
+		writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(1, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
 
 	return 0;
 }
@@ -1147,12 +1164,23 @@ static __maybe_unused int vdec_runtime_resume(struct device *dev)
 	struct venus_core *core = dev_get_drvdata(dev);
 	int ret;
 
-	if (core->res->hfi_version == HFI_VERSION_1XX)
+	if (IS_V1(core))
 		return 0;
 
-	writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	if (IS_V3(core))
+		writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(0, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+
 	ret = clk_prepare_enable(core->core0_clk);
-	writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+
+	if (IS_V4(core))
+		ret |= clk_prepare_enable(core->core0_bus_clk);
+
+	if (IS_V3(core))
+		writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(1, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
 
 	return ret;
 }
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 947001170a77..bc8c2e7a8d2c 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -1225,12 +1225,18 @@ static int venc_probe(struct platform_device *pdev)
 	if (!core)
 		return -EPROBE_DEFER;
 
-	if (core->res->hfi_version == HFI_VERSION_3XX) {
+	if (IS_V3(core) || IS_V4(core)) {
 		core->core1_clk = devm_clk_get(dev, "core");
 		if (IS_ERR(core->core1_clk))
 			return PTR_ERR(core->core1_clk);
 	}
 
+	if (IS_V4(core)) {
+		core->core1_bus_clk = devm_clk_get(dev, "bus");
+		if (IS_ERR(core->core1_bus_clk))
+			return PTR_ERR(core->core1_bus_clk);
+	}
+
 	platform_set_drvdata(pdev, core);
 
 	vdev = video_device_alloc();
@@ -1276,12 +1282,23 @@ static __maybe_unused int venc_runtime_suspend(struct device *dev)
 {
 	struct venus_core *core = dev_get_drvdata(dev);
 
-	if (core->res->hfi_version == HFI_VERSION_1XX)
+	if (IS_V1(core))
 		return 0;
 
-	writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	if (IS_V3(core))
+		writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(0, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+
+	if (IS_V4(core))
+		clk_disable_unprepare(core->core1_bus_clk);
+
 	clk_disable_unprepare(core->core1_clk);
-	writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+
+	if (IS_V3(core))
+		writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(1, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
 
 	return 0;
 }
@@ -1291,12 +1308,23 @@ static __maybe_unused int venc_runtime_resume(struct device *dev)
 	struct venus_core *core = dev_get_drvdata(dev);
 	int ret;
 
-	if (core->res->hfi_version == HFI_VERSION_1XX)
+	if (IS_V1(core))
 		return 0;
 
-	writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	if (IS_V3(core))
+		writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(0, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+
 	ret = clk_prepare_enable(core->core1_clk);
-	writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+
+	if (IS_V4(core))
+		ret |= clk_prepare_enable(core->core1_bus_clk);
+
+	if (IS_V3(core))
+		writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
+	else if (IS_V4(core))
+		writel(1, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
 
 	return ret;
 }
-- 
2.14.1

[PATCH 10/28] venus: vdec: call session_continue in insufficient event

[Stanimir Varbanov]

Call session_continue for Venus 4xx version even when the event
says that the buffer resources are not sufficient. Leaving a
comment with more information about the workaround.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/vdec.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index c45452634e7e..91c7384ff9c8 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -873,6 +873,14 @@ static void vdec_event_notify(struct venus_inst *inst, u32 event,
 
 			dev_dbg(dev, "event not sufficient resources (%ux%u)\n",
 				data->width, data->height);
+			/*
+			 * Workaround: Even that the firmware send and event for
+			 * insufficient buffer resources it is safe to call
+			 * session_continue because actually the event says that
+			 * the number of capture buffers is lower.
+			 */
+			if (IS_V4(core))
+				hfi_session_continue(inst);
 			break;
 		case HFI_EVENT_RELEASE_BUFFER_REFERENCE:
 			venus_helper_release_buf_ref(inst, data->tag);
-- 
2.14.1

Re: [PATCH 10/28] venus: vdec: call session_continue in insufficient event

[Vikash Garodia]

Hello Stanimir,

On 2018-04-24 18:14, Stanimir Varbanov wrote:
> Call session_continue for Venus 4xx version even when the event
> says that the buffer resources are not sufficient. Leaving a
> comment with more information about the workaround.
> 
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/vdec.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/drivers/media/platform/qcom/venus/vdec.c
> b/drivers/media/platform/qcom/venus/vdec.c
> index c45452634e7e..91c7384ff9c8 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -873,6 +873,14 @@ static void vdec_event_notify(struct venus_inst
> *inst, u32 event,
> 
>  			dev_dbg(dev, "event not sufficient resources (%ux%u)\n",
>  				data->width, data->height);
> +			/*
> +			 * Workaround: Even that the firmware send and event for
> +			 * insufficient buffer resources it is safe to call
> +			 * session_continue because actually the event says that
> +			 * the number of capture buffers is lower.
> +			 */
> +			if (IS_V4(core))
> +				hfi_session_continue(inst);
>  			break;
>  		case HFI_EVENT_RELEASE_BUFFER_REFERENCE:
>  			venus_helper_release_buf_ref(inst, data->tag);

Insufficient event from video firmware could be sent either,
1. due to insufficient buffer resources
2. due to lower capture buffers

It cannot be assumed that the event received by the host is due to lower 
capture
buffers. Incase the buffer resource is insufficient, let say there is a 
bitstream
resolution switch from 720p to 1080p, capture buffers needs to be 
reallocated.

The driver should be sending the V4L2_EVENT_SOURCE_CHANGE to client 
instead of ignoring
the event from firmware.

Thanks,
Vikash

Re: [PATCH 10/28] venus: vdec: call session_continue in insufficient event

[Stanimir Varbanov]

Hi Vikash,

Thanks for the comments!

On  2.05.2018 09:26, Vikash Garodia wrote:
> Hello Stanimir,
> 
> On 2018-04-24 18:14, Stanimir Varbanov wrote:
>> Call session_continue for Venus 4xx version even when the event
>> says that the buffer resources are not sufficient. Leaving a
>> comment with more information about the workaround.
>>
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>  drivers/media/platform/qcom/venus/vdec.c | 8 ++++++++
>>  1 file changed, 8 insertions(+)
>>
>> diff --git a/drivers/media/platform/qcom/venus/vdec.c
>> b/drivers/media/platform/qcom/venus/vdec.c
>> index c45452634e7e..91c7384ff9c8 100644
>> --- a/drivers/media/platform/qcom/venus/vdec.c
>> +++ b/drivers/media/platform/qcom/venus/vdec.c
>> @@ -873,6 +873,14 @@ static void vdec_event_notify(struct venus_inst
>> *inst, u32 event,
>>
>>              dev_dbg(dev, "event not sufficient resources (%ux%u)\n",
>>                  data->width, data->height);
>> +            /*
>> +             * Workaround: Even that the firmware send and event for
>> +             * insufficient buffer resources it is safe to call
>> +             * session_continue because actually the event says that
>> +             * the number of capture buffers is lower.
>> +             */
>> +            if (IS_V4(core))
>> +                hfi_session_continue(inst);
>>              break;
>>          case HFI_EVENT_RELEASE_BUFFER_REFERENCE:
>>              venus_helper_release_buf_ref(inst, data->tag);
> 
> Insufficient event from video firmware could be sent either,
> 1. due to insufficient buffer resources
> 2. due to lower capture buffers
> 
> It cannot be assumed that the event received by the host is due to lower 
> capture
> buffers. Incase the buffer resource is insufficient, let say there is a 
> bitstream
> resolution switch from 720p to 1080p, capture buffers needs to be 
> reallocated.

I agree with you. I will rework this part and call session_continue only 
for case #2.

> 
> The driver should be sending the V4L2_EVENT_SOURCE_CHANGE to client 
> instead of ignoring
> the event from firmware.

The v4l2 event is sent always to v4l clients.

regards,
Stan

Re: [PATCH 10/28] venus: vdec: call session_continue in insufficient event

[Vikash Garodia]

Hi Stanimir,

On 2018-05-03 17:06, Stanimir Varbanov wrote:
> Hi Vikash,
> 
> Thanks for the comments!
> 
> On  2.05.2018 09:26, Vikash Garodia wrote:
>> Hello Stanimir,
>> 
>> On 2018-04-24 18:14, Stanimir Varbanov wrote:
>>> Call session_continue for Venus 4xx version even when the event
>>> says that the buffer resources are not sufficient. Leaving a
>>> comment with more information about the workaround.
>>> 
>>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>>> ---
>>>  drivers/media/platform/qcom/venus/vdec.c | 8 ++++++++
>>>  1 file changed, 8 insertions(+)
>>> 
>>> diff --git a/drivers/media/platform/qcom/venus/vdec.c
>>> b/drivers/media/platform/qcom/venus/vdec.c
>>> index c45452634e7e..91c7384ff9c8 100644
>>> --- a/drivers/media/platform/qcom/venus/vdec.c
>>> +++ b/drivers/media/platform/qcom/venus/vdec.c
>>> @@ -873,6 +873,14 @@ static void vdec_event_notify(struct venus_inst
>>> *inst, u32 event,
>>> 
>>>              dev_dbg(dev, "event not sufficient resources (%ux%u)\n",
>>>                  data->width, data->height);
>>> +            /*
>>> +             * Workaround: Even that the firmware send and event for
>>> +             * insufficient buffer resources it is safe to call
>>> +             * session_continue because actually the event says that
>>> +             * the number of capture buffers is lower.
>>> +             */
>>> +            if (IS_V4(core))
>>> +                hfi_session_continue(inst);
>>>              break;
>>>          case HFI_EVENT_RELEASE_BUFFER_REFERENCE:
>>>              venus_helper_release_buf_ref(inst, data->tag);
>> 
>> Insufficient event from video firmware could be sent either,
>> 1. due to insufficient buffer resources
>> 2. due to lower capture buffers
>> 
>> It cannot be assumed that the event received by the host is due to 
>> lower capture
>> buffers. Incase the buffer resource is insufficient, let say there is 
>> a bitstream
>> resolution switch from 720p to 1080p, capture buffers needs to be 
>> reallocated.
> 
> I agree with you. I will rework this part and call session_continue
> only for case #2.

Even if the capture buffers are lower, driver should consider 
reallocation of capture
buffers with required higher count. Without this, it may happen that for 
a given video
frame, the decoded output will not be generated. The fact that the DPB 
buffer count is
same as capture buffers, will be lower than required. Hence the frame 
which needs YUV
reference beyond the DPB count, will get stuck as it cannot be decoded 
due to unavailability
of sufficient DPB buffers.
Say for ex. 10 DPB and capture buffers are allocated. For a given 
bitstream, firmware requested
the count to be 15. Frame 1 to 10 gets decoded and stored in DPB as 
references for future frame
decoding. Now when the 11th frame is queued to firmware, it can be 
decode but cannot be stored
as reference to decode future (12th) frame. Hence 11 frame will get 
stuck and will not be given
back to host driver.

>> 
>> The driver should be sending the V4L2_EVENT_SOURCE_CHANGE to client 
>> instead of ignoring
>> the event from firmware.
> 
> The v4l2 event is sent always to v4l clients.
> 
> regards,
> Stan

Re: [PATCH 10/28] venus: vdec: call session_continue in insufficient event

[Stanimir Varbanov]

Hi Vikash,

On 05/04/2018 02:09 PM, Vikash Garodia wrote:
> Hi Stanimir,
> 
> On 2018-05-03 17:06, Stanimir Varbanov wrote:
>> Hi Vikash,
>>
>> Thanks for the comments!
>>
>> On  2.05.2018 09:26, Vikash Garodia wrote:
>>> Hello Stanimir,
>>>
>>> On 2018-04-24 18:14, Stanimir Varbanov wrote:
>>>> Call session_continue for Venus 4xx version even when the event
>>>> says that the buffer resources are not sufficient. Leaving a
>>>> comment with more information about the workaround.
>>>>
>>>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>>>> ---
>>>>  drivers/media/platform/qcom/venus/vdec.c | 8 ++++++++
>>>>  1 file changed, 8 insertions(+)
>>>>
>>>> diff --git a/drivers/media/platform/qcom/venus/vdec.c
>>>> b/drivers/media/platform/qcom/venus/vdec.c
>>>> index c45452634e7e..91c7384ff9c8 100644
>>>> --- a/drivers/media/platform/qcom/venus/vdec.c
>>>> +++ b/drivers/media/platform/qcom/venus/vdec.c
>>>> @@ -873,6 +873,14 @@ static void vdec_event_notify(struct venus_inst
>>>> *inst, u32 event,
>>>>
>>>>              dev_dbg(dev, "event not sufficient resources (%ux%u)\n",
>>>>                  data->width, data->height);
>>>> +            /*
>>>> +             * Workaround: Even that the firmware send and event for
>>>> +             * insufficient buffer resources it is safe to call
>>>> +             * session_continue because actually the event says that
>>>> +             * the number of capture buffers is lower.
>>>> +             */
>>>> +            if (IS_V4(core))
>>>> +                hfi_session_continue(inst);
>>>>              break;
>>>>          case HFI_EVENT_RELEASE_BUFFER_REFERENCE:
>>>>              venus_helper_release_buf_ref(inst, data->tag);
>>>
>>> Insufficient event from video firmware could be sent either,
>>> 1. due to insufficient buffer resources
>>> 2. due to lower capture buffers
>>>
>>> It cannot be assumed that the event received by the host is due to
>>> lower capture
>>> buffers. Incase the buffer resource is insufficient, let say there is
>>> a bitstream
>>> resolution switch from 720p to 1080p, capture buffers needs to be
>>> reallocated.
>>
>> I agree with you. I will rework this part and call session_continue
>> only for case #2.
> 
> Even if the capture buffers are lower, driver should consider
> reallocation of capture
> buffers with required higher count. Without this, it may happen that for
> a given video
> frame, the decoded output will not be generated.

Thanks for the comments, I realized that this workaround is not needed
anymore, so I will drop the patch.

-- 
regards,
Stan

[PATCH 11/28] venus: add common capability parser

[Stanimir Varbanov]

This adds common capability parser for all supported Venus
versions. Having it will help to enumerate better the supported
raw formars and codecs and also the capabilities for every
codec like max/min width/height, framerate, bitrate and so on.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/Makefile     |   3 +-
 drivers/media/platform/qcom/venus/core.c       |  80 ++++++
 drivers/media/platform/qcom/venus/core.h       |  68 +++--
 drivers/media/platform/qcom/venus/hfi.c        |   5 +-
 drivers/media/platform/qcom/venus/hfi_helper.h |  28 +-
 drivers/media/platform/qcom/venus/hfi_msgs.c   | 348 ++-----------------------
 drivers/media/platform/qcom/venus/hfi_parser.c | 294 +++++++++++++++++++++
 drivers/media/platform/qcom/venus/hfi_parser.h |  45 ++++
 drivers/media/platform/qcom/venus/vdec.c       |  38 +--
 drivers/media/platform/qcom/venus/venc.c       |  52 ++--
 10 files changed, 524 insertions(+), 437 deletions(-)
 create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.c
 create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.h

diff --git a/drivers/media/platform/qcom/venus/Makefile b/drivers/media/platform/qcom/venus/Makefile
index bfd4edf7c83f..b44b11b03e12 100644
--- a/drivers/media/platform/qcom/venus/Makefile
+++ b/drivers/media/platform/qcom/venus/Makefile
@@ -2,7 +2,8 @@
 # Makefile for Qualcomm Venus driver
 
 venus-core-objs += core.o helpers.o firmware.o \
-		   hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o
+		   hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \
+		   hfi_parser.o
 
 venus-dec-objs += vdec.o vdec_ctrls.o
 venus-enc-objs += venc.o venc_ctrls.o
diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/venus/core.c
index 41eef376eb2d..1b72bfbb6297 100644
--- a/drivers/media/platform/qcom/venus/core.c
+++ b/drivers/media/platform/qcom/venus/core.c
@@ -152,6 +152,78 @@ static void venus_clks_disable(struct venus_core *core)
 		clk_disable_unprepare(core->clks[i]);
 }
 
+static u32 to_v4l2_codec_type(u32 codec)
+{
+	switch (codec) {
+	case HFI_VIDEO_CODEC_H264:
+		return V4L2_PIX_FMT_H264;
+	case HFI_VIDEO_CODEC_H263:
+		return V4L2_PIX_FMT_H263;
+	case HFI_VIDEO_CODEC_MPEG1:
+		return V4L2_PIX_FMT_MPEG1;
+	case HFI_VIDEO_CODEC_MPEG2:
+		return V4L2_PIX_FMT_MPEG2;
+	case HFI_VIDEO_CODEC_MPEG4:
+		return V4L2_PIX_FMT_MPEG4;
+	case HFI_VIDEO_CODEC_VC1:
+		return V4L2_PIX_FMT_VC1_ANNEX_G;
+	case HFI_VIDEO_CODEC_VP8:
+		return V4L2_PIX_FMT_VP8;
+	case HFI_VIDEO_CODEC_VP9:
+		return V4L2_PIX_FMT_VP9;
+	case HFI_VIDEO_CODEC_DIVX:
+	case HFI_VIDEO_CODEC_DIVX_311:
+		return V4L2_PIX_FMT_XVID;
+	default:
+		return 0;
+	}
+}
+
+static int venus_enumerate_codecs(struct venus_core *core, u32 type)
+{
+	const struct hfi_inst_ops dummy_ops = {};
+	struct venus_inst inst;
+	unsigned int i;
+	u32 codec, codecs;
+	int ret;
+
+	if (core->res->hfi_version != HFI_VERSION_1XX)
+		return 0;
+
+	memset(&inst, 0, sizeof(inst));
+	mutex_init(&inst.lock);
+	inst.core = core;
+	inst.session_type = type;
+	if (type == VIDC_SESSION_TYPE_DEC)
+		codecs = core->dec_codecs;
+	else
+		codecs = core->enc_codecs;
+
+	ret = hfi_session_create(&inst, &dummy_ops);
+	if (ret)
+		return ret;
+
+	for (i = 0; i < MAX_CODEC_NUM; i++) {
+		codec = (1 << i) & codecs;
+		if (!codec)
+			continue;
+
+		ret = hfi_session_init(&inst, to_v4l2_codec_type(codec));
+		if (ret)
+			goto done;
+
+		ret = hfi_session_deinit(&inst);
+		if (ret)
+			goto done;
+	}
+
+done:
+	hfi_session_destroy(&inst);
+	mutex_destroy(&inst.lock);
+
+	return ret;
+}
+
 static int venus_probe(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
@@ -219,6 +291,14 @@ static int venus_probe(struct platform_device *pdev)
 	if (ret)
 		goto err_venus_shutdown;
 
+	ret = venus_enumerate_codecs(core, VIDC_SESSION_TYPE_DEC);
+	if (ret)
+		goto err_venus_shutdown;
+
+	ret = venus_enumerate_codecs(core, VIDC_SESSION_TYPE_ENC);
+	if (ret)
+		goto err_venus_shutdown;
+
 	ret = v4l2_device_register(dev, &core->v4l2_dev);
 	if (ret)
 		goto err_core_deinit;
diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index b5b9a84e9155..fe2d2b9e8af8 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -57,6 +57,29 @@ struct venus_format {
 	u32 type;
 };
 
+#define MAX_PLANES		4
+#define MAX_FMT_ENTRIES		32
+#define MAX_CAP_ENTRIES		32
+#define MAX_CODEC_NUM		32
+
+struct raw_formats {
+	u32 buftype;
+	u32 fmt;
+};
+
+struct venus_caps {
+	u32 codec;
+	u32 domain;
+	bool cap_bufs_mode_dynamic;
+	unsigned int num_caps;
+	struct hfi_capability caps[MAX_CAP_ENTRIES];
+	unsigned int num_pl;
+	struct hfi_profile_level pl[HFI_MAX_PROFILE_COUNT];
+	unsigned int num_fmts;
+	struct raw_formats fmts[MAX_FMT_ENTRIES];
+	bool valid;
+};
+
 /**
  * struct venus_core - holds core parameters valid for all instances
  *
@@ -120,6 +143,8 @@ struct venus_core {
 	void *priv;
 	const struct hfi_ops *ops;
 	struct delayed_work work;
+	struct venus_caps caps[MAX_CODEC_NUM];
+	unsigned int codecs_count;
 };
 
 struct vdec_controls {
@@ -224,22 +249,8 @@ struct venus_buffer {
  * @priv:	a private for HFI operations callbacks
  * @session_type:	the type of the session (decoder or encoder)
  * @hprop:	a union used as a holder by get property
- * @cap_width:	width capability
- * @cap_height:	height capability
- * @cap_mbs_per_frame:	macroblocks per frame capability
- * @cap_mbs_per_sec:	macroblocks per second capability
- * @cap_framerate:	framerate capability
- * @cap_scale_x:		horizontal scaling capability
- * @cap_scale_y:		vertical scaling capability
- * @cap_bitrate:		bitrate capability
- * @cap_hier_p:		hier capability
- * @cap_ltr_count:	LTR count capability
- * @cap_secure_output2_threshold: secure OUTPUT2 threshold capability
  * @cap_bufs_mode_static:	buffers allocation mode capability
  * @cap_bufs_mode_dynamic:	buffers allocation mode capability
- * @pl_count:	count of supported profiles/levels
- * @pl:		supported profiles/levels
- * @bufreq:	holds buffer requirements
  */
 struct venus_inst {
 	struct list_head list;
@@ -276,6 +287,7 @@ struct venus_inst {
 	bool reconfig;
 	u32 reconfig_width;
 	u32 reconfig_height;
+	u32 hfi_codec;
 	u32 sequence_cap;
 	u32 sequence_out;
 	struct v4l2_m2m_dev *m2m_dev;
@@ -287,22 +299,8 @@ struct venus_inst {
 	const struct hfi_inst_ops *ops;
 	u32 session_type;
 	union hfi_get_property hprop;
-	struct hfi_capability cap_width;
-	struct hfi_capability cap_height;
-	struct hfi_capability cap_mbs_per_frame;
-	struct hfi_capability cap_mbs_per_sec;
-	struct hfi_capability cap_framerate;
-	struct hfi_capability cap_scale_x;
-	struct hfi_capability cap_scale_y;
-	struct hfi_capability cap_bitrate;
-	struct hfi_capability cap_hier_p;
-	struct hfi_capability cap_ltr_count;
-	struct hfi_capability cap_secure_output2_threshold;
 	bool cap_bufs_mode_static;
 	bool cap_bufs_mode_dynamic;
-	unsigned int pl_count;
-	struct hfi_profile_level pl[HFI_MAX_PROFILE_COUNT];
-	struct hfi_buffer_requirements bufreq[HFI_BUFFER_TYPE_MAX];
 };
 
 #define IS_V1(core)	((core)->res->hfi_version == HFI_VERSION_1XX)
@@ -322,4 +320,18 @@ static inline void *to_hfi_priv(struct venus_core *core)
 	return core->priv;
 }
 
+static inline struct venus_caps *
+venus_caps_by_codec(struct venus_core *core, u32 codec, u32 domain)
+{
+	unsigned int c;
+
+	for (c = 0; c < MAX_CODEC_NUM; c++) {
+		if (core->caps[c].codec == codec &&
+		    core->caps[c].domain == domain)
+			return &core->caps[c];
+	}
+
+	return NULL;
+}
+
 #endif
diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
index a570fdad0de0..94ca27b0bb99 100644
--- a/drivers/media/platform/qcom/venus/hfi.c
+++ b/drivers/media/platform/qcom/venus/hfi.c
@@ -203,13 +203,12 @@ int hfi_session_init(struct venus_inst *inst, u32 pixfmt)
 {
 	struct venus_core *core = inst->core;
 	const struct hfi_ops *ops = core->ops;
-	u32 codec;
 	int ret;
 
-	codec = to_codec_type(pixfmt);
+	inst->hfi_codec = to_codec_type(pixfmt);
 	reinit_completion(&inst->done);
 
-	ret = ops->session_init(inst, inst->session_type, codec);
+	ret = ops->session_init(inst, inst->session_type, inst->hfi_codec);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/venus/hfi_helper.h
index 1bc5aab1ce6b..64cc2bc946ee 100644
--- a/drivers/media/platform/qcom/venus/hfi_helper.h
+++ b/drivers/media/platform/qcom/venus/hfi_helper.h
@@ -858,10 +858,23 @@ struct hfi_uncompressed_format_select {
 	u32 format;
 };
 
+struct hfi_uncompressed_plane_constraints {
+	u32 stride_multiples;
+	u32 max_stride;
+	u32 min_plane_buffer_height_multiple;
+	u32 buffer_alignment;
+};
+
+struct hfi_uncompressed_plane_info {
+	u32 format;
+	u32 num_planes;
+	struct hfi_uncompressed_plane_constraints plane_format[1];
+};
+
 struct hfi_uncompressed_format_supported {
 	u32 buffer_type;
 	u32 format_entries;
-	u32 format_info[1];
+	struct hfi_uncompressed_plane_info format_info[1];
 };
 
 struct hfi_uncompressed_plane_actual {
@@ -875,19 +888,6 @@ struct hfi_uncompressed_plane_actual_info {
 	struct hfi_uncompressed_plane_actual plane_format[1];
 };
 
-struct hfi_uncompressed_plane_constraints {
-	u32 stride_multiples;
-	u32 max_stride;
-	u32 min_plane_buffer_height_multiple;
-	u32 buffer_alignment;
-};
-
-struct hfi_uncompressed_plane_info {
-	u32 format;
-	u32 num_planes;
-	struct hfi_uncompressed_plane_constraints plane_format[1];
-};
-
 struct hfi_uncompressed_plane_actual_constraints_info {
 	u32 buffer_type;
 	u32 num_planes;
diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/venus/hfi_msgs.c
index 023802e62833..8a943f53a12b 100644
--- a/drivers/media/platform/qcom/venus/hfi_msgs.c
+++ b/drivers/media/platform/qcom/venus/hfi_msgs.c
@@ -21,6 +21,7 @@
 #include "hfi.h"
 #include "hfi_helper.h"
 #include "hfi_msgs.h"
+#include "hfi_parser.h"
 
 static void event_seq_changed(struct venus_core *core, struct venus_inst *inst,
 			      struct hfi_msg_event_notify_pkt *pkt)
@@ -219,81 +220,30 @@ static void hfi_sys_init_done(struct venus_core *core, struct venus_inst *inst,
 			      void *packet)
 {
 	struct hfi_msg_sys_init_done_pkt *pkt = packet;
-	u32 rem_bytes, read_bytes = 0, num_properties;
-	u32 error, ptype;
-	u8 *data;
+	u32 rem_bytes, num_properties, error;
 
 	error = pkt->error_type;
 	if (error != HFI_ERR_NONE)
-		goto err_no_prop;
+		goto done;
 
 	num_properties = pkt->num_properties;
 
 	if (!num_properties) {
 		error = HFI_ERR_SYS_INVALID_PARAMETER;
-		goto err_no_prop;
+		goto done;
 	}
 
 	rem_bytes = pkt->hdr.size - sizeof(*pkt) + sizeof(u32);
-
 	if (!rem_bytes) {
 		/* missing property data */
 		error = HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
-		goto err_no_prop;
+		goto done;
 	}
 
-	data = (u8 *)&pkt->data[0];
-
-	if (core->res->hfi_version == HFI_VERSION_3XX)
-		goto err_no_prop;
-
-	while (num_properties && rem_bytes >= sizeof(u32)) {
-		ptype = *((u32 *)data);
-		data += sizeof(u32);
-
-		switch (ptype) {
-		case HFI_PROPERTY_PARAM_CODEC_SUPPORTED: {
-			struct hfi_codec_supported *prop;
-
-			prop = (struct hfi_codec_supported *)data;
-
-			if (rem_bytes < sizeof(*prop)) {
-				error = HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
-				break;
-			}
-
-			read_bytes += sizeof(*prop) + sizeof(u32);
-			core->dec_codecs = prop->dec_codecs;
-			core->enc_codecs = prop->enc_codecs;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_MAX_SESSIONS_SUPPORTED: {
-			struct hfi_max_sessions_supported *prop;
-
-			if (rem_bytes < sizeof(*prop)) {
-				error = HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
-				break;
-			}
-
-			prop = (struct hfi_max_sessions_supported *)data;
-			read_bytes += sizeof(*prop) + sizeof(u32);
-			core->max_sessions_supported = prop->max_sessions;
-			break;
-		}
-		default:
-			error = HFI_ERR_SYS_INVALID_PARAMETER;
-			break;
-		}
-
-		if (error)
-			break;
-
-		rem_bytes -= read_bytes;
-		data += read_bytes;
-		num_properties--;
-	}
+	error = hfi_parser(core, inst, num_properties, &pkt->data[0],
+			   rem_bytes);
 
-err_no_prop:
+done:
 	core->error = error;
 	complete(&core->done);
 }
@@ -371,51 +321,6 @@ static void hfi_sys_pc_prepare_done(struct venus_core *core,
 	dev_dbg(core->dev, "pc prepare done (error %x)\n", pkt->error_type);
 }
 
-static void
-hfi_copy_cap_prop(struct hfi_capability *in, struct venus_inst *inst)
-{
-	if (!in || !inst)
-		return;
-
-	switch (in->capability_type) {
-	case HFI_CAPABILITY_FRAME_WIDTH:
-		inst->cap_width = *in;
-		break;
-	case HFI_CAPABILITY_FRAME_HEIGHT:
-		inst->cap_height = *in;
-		break;
-	case HFI_CAPABILITY_MBS_PER_FRAME:
-		inst->cap_mbs_per_frame = *in;
-		break;
-	case HFI_CAPABILITY_MBS_PER_SECOND:
-		inst->cap_mbs_per_sec = *in;
-		break;
-	case HFI_CAPABILITY_FRAMERATE:
-		inst->cap_framerate = *in;
-		break;
-	case HFI_CAPABILITY_SCALE_X:
-		inst->cap_scale_x = *in;
-		break;
-	case HFI_CAPABILITY_SCALE_Y:
-		inst->cap_scale_y = *in;
-		break;
-	case HFI_CAPABILITY_BITRATE:
-		inst->cap_bitrate = *in;
-		break;
-	case HFI_CAPABILITY_HIER_P_NUM_ENH_LAYERS:
-		inst->cap_hier_p = *in;
-		break;
-	case HFI_CAPABILITY_ENC_LTR_COUNT:
-		inst->cap_ltr_count = *in;
-		break;
-	case HFI_CAPABILITY_CP_OUTPUT2_THRESH:
-		inst->cap_secure_output2_threshold = *in;
-		break;
-	default:
-		break;
-	}
-}
-
 static unsigned int
 session_get_prop_profile_level(struct hfi_msg_session_property_info_pkt *pkt,
 			       struct hfi_profile_level *profile_level)
@@ -505,238 +410,11 @@ static void hfi_session_prop_info(struct venus_core *core,
 	complete(&inst->done);
 }
 
-static u32 init_done_read_prop(struct venus_core *core, struct venus_inst *inst,
-			       struct hfi_msg_session_init_done_pkt *pkt)
-{
-	struct device *dev = core->dev;
-	u32 rem_bytes, num_props;
-	u32 ptype, next_offset = 0;
-	u32 err;
-	u8 *data;
-
-	rem_bytes = pkt->shdr.hdr.size - sizeof(*pkt) + sizeof(u32);
-	if (!rem_bytes) {
-		dev_err(dev, "%s: missing property info\n", __func__);
-		return HFI_ERR_SESSION_INSUFFICIENT_RESOURCES;
-	}
-
-	err = pkt->error_type;
-	if (err)
-		return err;
-
-	data = (u8 *)&pkt->data[0];
-	num_props = pkt->num_properties;
-
-	while (err == HFI_ERR_NONE && num_props && rem_bytes >= sizeof(u32)) {
-		ptype = *((u32 *)data);
-		next_offset = sizeof(u32);
-
-		switch (ptype) {
-		case HFI_PROPERTY_PARAM_CODEC_MASK_SUPPORTED: {
-			struct hfi_codec_mask_supported *masks =
-				(struct hfi_codec_mask_supported *)
-				(data + next_offset);
-
-			next_offset += sizeof(*masks);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_CAPABILITY_SUPPORTED: {
-			struct hfi_capabilities *caps;
-			struct hfi_capability *cap;
-			u32 num_caps;
-
-			if ((rem_bytes - next_offset) < sizeof(*cap)) {
-				err = HFI_ERR_SESSION_INVALID_PARAMETER;
-				break;
-			}
-
-			caps = (struct hfi_capabilities *)(data + next_offset);
-
-			num_caps = caps->num_capabilities;
-			cap = &caps->data[0];
-			next_offset += sizeof(u32);
-
-			while (num_caps &&
-			       (rem_bytes - next_offset) >= sizeof(u32)) {
-				hfi_copy_cap_prop(cap, inst);
-				cap++;
-				next_offset += sizeof(*cap);
-				num_caps--;
-			}
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SUPPORTED: {
-			struct hfi_uncompressed_format_supported *prop =
-				(struct hfi_uncompressed_format_supported *)
-				(data + next_offset);
-			u32 num_fmt_entries;
-			u8 *fmt;
-			struct hfi_uncompressed_plane_info *inf;
-
-			if ((rem_bytes - next_offset) < sizeof(*prop)) {
-				err = HFI_ERR_SESSION_INVALID_PARAMETER;
-				break;
-			}
-
-			num_fmt_entries = prop->format_entries;
-			next_offset = sizeof(*prop) - sizeof(u32);
-			fmt = (u8 *)&prop->format_info[0];
-
-			dev_dbg(dev, "uncomm format support num entries:%u\n",
-				num_fmt_entries);
-
-			while (num_fmt_entries) {
-				struct hfi_uncompressed_plane_constraints *cnts;
-				u32 bytes_to_skip;
-
-				inf = (struct hfi_uncompressed_plane_info *)fmt;
-
-				if ((rem_bytes - next_offset) < sizeof(*inf)) {
-					err = HFI_ERR_SESSION_INVALID_PARAMETER;
-					break;
-				}
-
-				dev_dbg(dev, "plane info: fmt:%x, planes:%x\n",
-					inf->format, inf->num_planes);
-
-				cnts = &inf->plane_format[0];
-				dev_dbg(dev, "%u %u %u %u\n",
-					cnts->stride_multiples,
-					cnts->max_stride,
-					cnts->min_plane_buffer_height_multiple,
-					cnts->buffer_alignment);
-
-				bytes_to_skip = sizeof(*inf) - sizeof(*cnts) +
-						inf->num_planes * sizeof(*cnts);
-
-				fmt += bytes_to_skip;
-				next_offset += bytes_to_skip;
-				num_fmt_entries--;
-			}
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_PROPERTIES_SUPPORTED: {
-			struct hfi_properties_supported *prop =
-				(struct hfi_properties_supported *)
-				(data + next_offset);
-
-			next_offset += sizeof(*prop) - sizeof(u32)
-					+ prop->num_properties * sizeof(u32);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_PROFILE_LEVEL_SUPPORTED: {
-			struct hfi_profile_level_supported *prop =
-				(struct hfi_profile_level_supported *)
-				(data + next_offset);
-			struct hfi_profile_level *pl;
-			unsigned int prop_count = 0;
-			unsigned int count = 0;
-			u8 *ptr;
-
-			ptr = (u8 *)&prop->profile_level[0];
-			prop_count = prop->profile_count;
-
-			if (prop_count > HFI_MAX_PROFILE_COUNT)
-				prop_count = HFI_MAX_PROFILE_COUNT;
-
-			while (prop_count) {
-				ptr++;
-				pl = (struct hfi_profile_level *)ptr;
-
-				inst->pl[count].profile = pl->profile;
-				inst->pl[count].level = pl->level;
-				prop_count--;
-				count++;
-				ptr += sizeof(*pl) / sizeof(u32);
-			}
-
-			inst->pl_count = count;
-			next_offset += sizeof(*prop) - sizeof(*pl) +
-				       prop->profile_count * sizeof(*pl);
-
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_INTERLACE_FORMAT_SUPPORTED: {
-			next_offset +=
-				sizeof(struct hfi_interlace_format_supported);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_NAL_STREAM_FORMAT_SUPPORTED: {
-			struct hfi_nal_stream_format *nal =
-				(struct hfi_nal_stream_format *)
-				(data + next_offset);
-			dev_dbg(dev, "NAL format: %x\n", nal->format);
-			next_offset += sizeof(*nal);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_NAL_STREAM_FORMAT_SELECT: {
-			next_offset += sizeof(u32);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_MAX_SEQUENCE_HEADER_SIZE: {
-			u32 *max_seq_sz = (u32 *)(data + next_offset);
-
-			dev_dbg(dev, "max seq header sz: %x\n", *max_seq_sz);
-			next_offset += sizeof(u32);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH: {
-			next_offset += sizeof(struct hfi_intra_refresh);
-			num_props--;
-			break;
-		}
-		case HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE_SUPPORTED: {
-			struct hfi_buffer_alloc_mode_supported *prop =
-				(struct hfi_buffer_alloc_mode_supported *)
-				(data + next_offset);
-			unsigned int i;
-
-			for (i = 0; i < prop->num_entries; i++) {
-				if (prop->buffer_type == HFI_BUFFER_OUTPUT ||
-				    prop->buffer_type == HFI_BUFFER_OUTPUT2) {
-					switch (prop->data[i]) {
-					case HFI_BUFFER_MODE_STATIC:
-						inst->cap_bufs_mode_static = true;
-						break;
-					case HFI_BUFFER_MODE_DYNAMIC:
-						inst->cap_bufs_mode_dynamic = true;
-						break;
-					default:
-						break;
-					}
-				}
-			}
-			next_offset += sizeof(*prop) -
-				sizeof(u32) + prop->num_entries * sizeof(u32);
-			num_props--;
-			break;
-		}
-		default:
-			dev_dbg(dev, "%s: default case %#x\n", __func__, ptype);
-			break;
-		}
-
-		rem_bytes -= next_offset;
-		data += next_offset;
-	}
-
-	return err;
-}
-
 static void hfi_session_init_done(struct venus_core *core,
 				  struct venus_inst *inst, void *packet)
 {
 	struct hfi_msg_session_init_done_pkt *pkt = packet;
-	unsigned int error;
+	u32 rem_bytes, error;
 
 	error = pkt->error_type;
 	if (error != HFI_ERR_NONE)
@@ -745,8 +423,14 @@ static void hfi_session_init_done(struct venus_core *core,
 	if (core->res->hfi_version != HFI_VERSION_1XX)
 		goto done;
 
-	error = init_done_read_prop(core, inst, pkt);
+	rem_bytes = pkt->shdr.hdr.size - sizeof(*pkt) + sizeof(u32);
+	if (!rem_bytes) {
+		error = HFI_ERR_SESSION_INSUFFICIENT_RESOURCES;
+		goto done;
+	}
 
+	error = hfi_parser(core, inst, pkt->num_properties, &pkt->data[0],
+			   rem_bytes);
 done:
 	inst->error = error;
 	complete(&inst->done);
diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c b/drivers/media/platform/qcom/venus/hfi_parser.c
new file mode 100644
index 000000000000..4b155997abbb
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/hfi_parser.c
@@ -0,0 +1,294 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 Linaro Ltd.
+ *
+ * Author: Stanimir Varbanov <stanimir.varbanov@linaro.org>
+ */
+#include <linux/kernel.h>
+
+#include "core.h"
+#include "hfi_helper.h"
+
+typedef void (*func)(struct venus_caps *cap, void *data, unsigned int size);
+
+static void init_codecs_vcaps(struct venus_core *core)
+{
+	struct venus_caps *caps = core->caps;
+	struct venus_caps *cap;
+	unsigned int i;
+
+	for (i = 0; i < 8 * sizeof(core->dec_codecs); i++) {
+		if ((1 << i) & core->dec_codecs) {
+			cap = &caps[core->codecs_count++];
+			cap->codec = (1 << i) & core->dec_codecs;
+			cap->domain = VIDC_SESSION_TYPE_DEC;
+			cap->valid = false;
+		}
+	}
+
+	for (i = 0; i < 8 * sizeof(core->enc_codecs); i++) {
+		if ((1 << i) & core->enc_codecs) {
+			cap = &caps[core->codecs_count++];
+			cap->codec = (1 << i) & core->enc_codecs;
+			cap->domain = VIDC_SESSION_TYPE_ENC;
+			cap->valid = false;
+		}
+	}
+}
+
+static void for_each_codec(struct venus_caps *caps, unsigned int caps_num,
+			   u32 codecs, u32 domain, func cb, void *data,
+			   unsigned int size)
+{
+	struct venus_caps *cap;
+	unsigned int i;
+
+	for (i = 0; i < caps_num; i++) {
+		cap = &caps[i];
+		if (cap->valid && cap->domain == domain)
+			continue;
+		if (cap->codec & codecs && cap->domain == domain)
+			cb(cap, data, size);
+	}
+}
+
+static void fill_buf_mode(struct venus_caps *cap, void *data, unsigned int num)
+{
+	u32 *type = data;
+
+	if (*type == HFI_BUFFER_MODE_DYNAMIC)
+		cap->cap_bufs_mode_dynamic = true;
+}
+
+static void parse_alloc_mode(struct venus_core *core, struct venus_inst *inst,
+			     u32 codecs, u32 domain, void *data)
+{
+	struct hfi_buffer_alloc_mode_supported *mode = data;
+	u32 num_entries = mode->num_entries;
+	u32 *type;
+
+	if (num_entries > 16)
+		return;
+
+	type = mode->data;
+
+	while (num_entries--) {
+		if (mode->buffer_type == HFI_BUFFER_OUTPUT ||
+		    mode->buffer_type == HFI_BUFFER_OUTPUT2) {
+			if (*type == HFI_BUFFER_MODE_DYNAMIC && inst)
+				inst->cap_bufs_mode_dynamic = true;
+
+			for_each_codec(core->caps, ARRAY_SIZE(core->caps),
+				       codecs, domain, fill_buf_mode, type, 1);
+		}
+
+		type++;
+	}
+}
+
+static void parse_profile_level(u32 codecs, u32 domain, void *data)
+{
+	struct hfi_profile_level_supported *pl = data;
+	struct hfi_profile_level *proflevel = pl->profile_level;
+	u32 count = pl->profile_count;
+
+	if (count > HFI_MAX_PROFILE_COUNT)
+		return;
+
+	while (count) {
+		proflevel = (void *)proflevel + sizeof(*proflevel);
+		count--;
+	}
+}
+
+static void fill_caps(struct venus_caps *cap, void *data, unsigned int num)
+{
+	struct hfi_capability *caps = data;
+	unsigned int i;
+
+	for (i = 0; i < num; i++)
+		cap->caps[cap->num_caps++] = caps[i];
+}
+
+static void parse_caps(struct venus_core *core, struct venus_inst *inst,
+		       u32 codecs, u32 domain, void *data)
+{
+	struct hfi_capabilities *caps = data;
+	struct hfi_capability *cap = caps->data;
+	u32 num_caps = caps->num_capabilities;
+	struct hfi_capability caps_arr[MAX_CAP_ENTRIES] = {};
+	unsigned int i = 0;
+
+	if (num_caps > MAX_CAP_ENTRIES)
+		return;
+
+	while (num_caps) {
+		caps_arr[i++] = *cap;
+		cap = (void *)cap + sizeof(*cap);
+		num_caps--;
+	}
+
+	for_each_codec(core->caps, ARRAY_SIZE(core->caps), codecs, domain,
+		       fill_caps, caps_arr, i);
+}
+
+static void fill_raw_fmts(struct venus_caps *cap, void *fmts,
+			  unsigned int num_fmts)
+{
+	struct raw_formats *formats = fmts;
+	unsigned int i;
+
+	for (i = 0; i < num_fmts; i++)
+		cap->fmts[cap->num_fmts++] = formats[i];
+}
+
+static void parse_raw_formats(struct venus_core *core, struct venus_inst *inst,
+			      u32 codecs, u32 domain, void *data)
+{
+	struct hfi_uncompressed_format_supported *fmt = data;
+	struct hfi_uncompressed_plane_info *pinfo = fmt->format_info;
+	struct hfi_uncompressed_plane_constraints *constr;
+	u32 entries = fmt->format_entries;
+	u32 num_planes;
+	struct raw_formats rfmts[MAX_FMT_ENTRIES] = {};
+	unsigned int i = 0;
+
+	while (entries) {
+		num_planes = pinfo->num_planes;
+
+		rfmts[i].fmt = pinfo->format;
+		rfmts[i].buftype = fmt->buffer_type;
+		i++;
+
+		if (pinfo->num_planes > MAX_PLANES)
+			break;
+
+		constr = pinfo->plane_format;
+
+		while (pinfo->num_planes) {
+			constr = (void *)constr + sizeof(*constr);
+			pinfo->num_planes--;
+		}
+
+		pinfo = (void *)pinfo + sizeof(*constr) * num_planes +
+			2 * sizeof(u32);
+		entries--;
+	}
+
+	for_each_codec(core->caps, ARRAY_SIZE(core->caps), codecs, domain,
+		       fill_raw_fmts, rfmts, i);
+}
+
+static void parse_codecs(struct venus_core *core, void *data)
+{
+	struct hfi_codec_supported *codecs = data;
+
+	core->dec_codecs = codecs->dec_codecs;
+	core->enc_codecs = codecs->enc_codecs;
+
+	if (core->res->hfi_version == HFI_VERSION_1XX) {
+		core->dec_codecs &= ~HFI_VIDEO_CODEC_HEVC;
+		core->dec_codecs &= ~HFI_VIDEO_CODEC_SPARK;
+	}
+}
+
+static void parse_max_sessions(struct venus_core *core, void *data)
+{
+	struct hfi_max_sessions_supported *sessions = data;
+
+	core->max_sessions_supported = sessions->max_sessions;
+}
+
+static void parse_codecs_mask(u32 *codecs, u32 *domain, void *data)
+{
+	struct hfi_codec_mask_supported *mask = data;
+
+	*codecs = mask->codecs;
+	*domain = mask->video_domains;
+}
+
+static void parser_init(struct venus_core *core, struct venus_inst *inst,
+			u32 *codecs, u32 *domain)
+{
+	if (core->res->hfi_version != HFI_VERSION_1XX)
+		return;
+
+	if (!inst)
+		return;
+
+	*codecs = inst->hfi_codec;
+	*domain = inst->session_type;
+}
+
+static void parser_fini(struct venus_core *core, struct venus_inst *inst,
+			u32 codecs, u32 domain)
+{
+	struct venus_caps *caps = core->caps;
+	struct venus_caps *cap;
+	u32 dom;
+	unsigned int i;
+
+	if (core->res->hfi_version != HFI_VERSION_1XX)
+		return;
+
+	if (!inst)
+		return;
+
+	dom = inst->session_type;
+
+	for (i = 0; i < MAX_CODEC_NUM; i++) {
+		cap = &caps[i];
+		if (cap->codec & codecs && cap->domain == dom)
+			cap->valid = true;
+	}
+}
+
+u32 hfi_parser(struct venus_core *core, struct venus_inst *inst,
+	       u32 num_properties, void *buf, u32 size)
+{
+	unsigned int words_count = size >> 2;
+	u32 *word = buf, *data, codecs = 0, domain = 0;
+
+	if (size % 4)
+		return HFI_ERR_SYS_INSUFFICIENT_RESOURCES;
+
+	parser_init(core, inst, &codecs, &domain);
+
+	while (words_count) {
+		data = word + 1;
+
+		switch (*word) {
+		case HFI_PROPERTY_PARAM_CODEC_SUPPORTED:
+			parse_codecs(core, data);
+			init_codecs_vcaps(core);
+			break;
+		case HFI_PROPERTY_PARAM_MAX_SESSIONS_SUPPORTED:
+			parse_max_sessions(core, data);
+			break;
+		case HFI_PROPERTY_PARAM_CODEC_MASK_SUPPORTED:
+			parse_codecs_mask(&codecs, &domain, data);
+			break;
+		case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SUPPORTED:
+			parse_raw_formats(core, inst, codecs, domain, data);
+			break;
+		case HFI_PROPERTY_PARAM_CAPABILITY_SUPPORTED:
+			parse_caps(core, inst, codecs, domain, data);
+			break;
+		case HFI_PROPERTY_PARAM_PROFILE_LEVEL_SUPPORTED:
+			parse_profile_level(codecs, domain, data);
+			break;
+		case HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE_SUPPORTED:
+			parse_alloc_mode(core, inst, codecs, domain, data);
+			break;
+		default:
+			break;
+		}
+
+		word++;
+		words_count--;
+	}
+
+	parser_fini(core, inst, codecs, domain);
+
+	return HFI_ERR_NONE;
+}
diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h b/drivers/media/platform/qcom/venus/hfi_parser.h
new file mode 100644
index 000000000000..c484ac91a8e2
--- /dev/null
+++ b/drivers/media/platform/qcom/venus/hfi_parser.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2018 Linaro Ltd. */
+#ifndef __VENUS_HFI_PARSER_H__
+#define __VENUS_HFI_PARSER_H__
+
+#include "core.h"
+
+u32 hfi_parser(struct venus_core *core, struct venus_inst *inst,
+	       u32 num_properties, void *buf, u32 size);
+
+static inline struct hfi_capability *get_cap(struct venus_inst *inst, u32 type)
+{
+	struct venus_core *core = inst->core;
+	struct venus_caps *caps;
+	unsigned int i;
+
+	caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
+	if (!caps)
+		return ERR_PTR(-EINVAL);
+
+	for (i = 0; i < MAX_CAP_ENTRIES; i++) {
+		if (caps->caps[i].capability_type == type)
+			return &caps->caps[i];
+	}
+
+	return ERR_PTR(-EINVAL);
+}
+
+#define CAP_MIN(inst, type)	((get_cap(inst, type))->min)
+#define CAP_MAX(inst, type)	((get_cap(inst, type))->max)
+#define CAP_STEP(inst, type)	((get_cap(inst, type))->step_size)
+
+#define FRAME_WIDTH_MIN(inst)	CAP_MIN(inst, HFI_CAPABILITY_FRAME_WIDTH)
+#define FRAME_WIDTH_MAX(inst)	CAP_MAX(inst, HFI_CAPABILITY_FRAME_WIDTH)
+#define FRAME_WIDTH_STEP(inst)	CAP_STEP(inst, HFI_CAPABILITY_FRAME_WIDTH)
+
+#define FRAME_HEIGHT_MIN(inst)	CAP_MIN(inst, HFI_CAPABILITY_FRAME_HEIGHT)
+#define FRAME_HEIGHT_MAX(inst)	CAP_MAX(inst, HFI_CAPABILITY_FRAME_HEIGHT)
+#define FRAME_HEIGHT_STEP(inst)	CAP_STEP(inst, HFI_CAPABILITY_FRAME_HEIGHT)
+
+#define FRATE_MIN(inst)		CAP_MIN(inst, HFI_CAPABILITY_FRAMERATE)
+#define FRATE_MAX(inst)		CAP_MAX(inst, HFI_CAPABILITY_FRAMERATE)
+#define FRATE_STEP(inst)	CAP_STEP(inst, HFI_CAPABILITY_FRAMERATE)
+
+#endif
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 91c7384ff9c8..cd278a695899 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -24,6 +24,7 @@
 #include <media/videobuf2-dma-sg.h>
 
 #include "hfi_venus_io.h"
+#include "hfi_parser.h"
 #include "core.h"
 #include "helpers.h"
 #include "vdec.h"
@@ -177,10 +178,10 @@ vdec_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 		pixmp->height = 720;
 	}
 
-	pixmp->width = clamp(pixmp->width, inst->cap_width.min,
-			     inst->cap_width.max);
-	pixmp->height = clamp(pixmp->height, inst->cap_height.min,
-			      inst->cap_height.max);
+	pixmp->width = clamp(pixmp->width, FRAME_WIDTH_MIN(inst),
+			     FRAME_WIDTH_MAX(inst));
+	pixmp->height = clamp(pixmp->height, FRAME_HEIGHT_MIN(inst),
+			      FRAME_HEIGHT_MAX(inst));
 
 	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
 		pixmp->height = ALIGN(pixmp->height, 32);
@@ -442,12 +443,12 @@ static int vdec_enum_framesizes(struct file *file, void *fh,
 
 	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
 
-	fsize->stepwise.min_width = inst->cap_width.min;
-	fsize->stepwise.max_width = inst->cap_width.max;
-	fsize->stepwise.step_width = inst->cap_width.step_size;
-	fsize->stepwise.min_height = inst->cap_height.min;
-	fsize->stepwise.max_height = inst->cap_height.max;
-	fsize->stepwise.step_height = inst->cap_height.step_size;
+	fsize->stepwise.min_width = FRAME_WIDTH_MIN(inst);
+	fsize->stepwise.max_width = FRAME_WIDTH_MAX(inst);
+	fsize->stepwise.step_width = FRAME_WIDTH_STEP(inst);
+	fsize->stepwise.min_height = FRAME_HEIGHT_MIN(inst);
+	fsize->stepwise.max_height = FRAME_HEIGHT_MAX(inst);
+	fsize->stepwise.step_height = FRAME_HEIGHT_STEP(inst);
 
 	return 0;
 }
@@ -910,22 +911,7 @@ static void vdec_inst_init(struct venus_inst *inst)
 	inst->fps = 30;
 	inst->timeperframe.numerator = 1;
 	inst->timeperframe.denominator = 30;
-
-	inst->cap_width.min = 64;
-	inst->cap_width.max = 1920;
-	if (inst->core->res->hfi_version == HFI_VERSION_3XX)
-		inst->cap_width.max = 3840;
-	inst->cap_width.step_size = 1;
-	inst->cap_height.min = 64;
-	inst->cap_height.max = ALIGN(1080, 32);
-	if (inst->core->res->hfi_version == HFI_VERSION_3XX)
-		inst->cap_height.max = ALIGN(2160, 32);
-	inst->cap_height.step_size = 1;
-	inst->cap_framerate.min = 1;
-	inst->cap_framerate.max = 30;
-	inst->cap_framerate.step_size = 1;
-	inst->cap_mbs_per_frame.min = 16;
-	inst->cap_mbs_per_frame.max = 8160;
+	inst->hfi_codec = HFI_VIDEO_CODEC_H264;
 }
 
 static const struct v4l2_m2m_ops vdec_m2m_ops = {
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index bc8c2e7a8d2c..be8ea3326386 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -24,6 +24,7 @@
 #include <media/v4l2-ctrls.h>
 
 #include "hfi_venus_io.h"
+#include "hfi_parser.h"
 #include "core.h"
 #include "helpers.h"
 #include "venc.h"
@@ -301,10 +302,10 @@ venc_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 		pixmp->height = 720;
 	}
 
-	pixmp->width = clamp(pixmp->width, inst->cap_width.min,
-			     inst->cap_width.max);
-	pixmp->height = clamp(pixmp->height, inst->cap_height.min,
-			      inst->cap_height.max);
+	pixmp->width = clamp(pixmp->width, FRAME_WIDTH_MIN(inst),
+			     FRAME_WIDTH_MAX(inst));
+	pixmp->height = clamp(pixmp->height, FRAME_HEIGHT_MIN(inst),
+			      FRAME_HEIGHT_MAX(inst));
 
 	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
 		pixmp->height = ALIGN(pixmp->height, 32);
@@ -553,12 +554,12 @@ static int venc_enum_framesizes(struct file *file, void *fh,
 	if (fsize->index)
 		return -EINVAL;
 
-	fsize->stepwise.min_width = inst->cap_width.min;
-	fsize->stepwise.max_width = inst->cap_width.max;
-	fsize->stepwise.step_width = inst->cap_width.step_size;
-	fsize->stepwise.min_height = inst->cap_height.min;
-	fsize->stepwise.max_height = inst->cap_height.max;
-	fsize->stepwise.step_height = inst->cap_height.step_size;
+	fsize->stepwise.min_width = FRAME_WIDTH_MIN(inst);
+	fsize->stepwise.max_width = FRAME_WIDTH_MAX(inst);
+	fsize->stepwise.step_width = FRAME_WIDTH_STEP(inst);
+	fsize->stepwise.min_height = FRAME_HEIGHT_MIN(inst);
+	fsize->stepwise.max_height = FRAME_HEIGHT_MAX(inst);
+	fsize->stepwise.step_height = FRAME_HEIGHT_STEP(inst);
 
 	return 0;
 }
@@ -586,18 +587,18 @@ static int venc_enum_frameintervals(struct file *file, void *fh,
 	if (!fival->width || !fival->height)
 		return -EINVAL;
 
-	if (fival->width > inst->cap_width.max ||
-	    fival->width < inst->cap_width.min ||
-	    fival->height > inst->cap_height.max ||
-	    fival->height < inst->cap_height.min)
+	if (fival->width > FRAME_WIDTH_MAX(inst) ||
+	    fival->width < FRAME_WIDTH_MIN(inst) ||
+	    fival->height > FRAME_HEIGHT_MAX(inst) ||
+	    fival->height < FRAME_HEIGHT_MIN(inst))
 		return -EINVAL;
 
 	fival->stepwise.min.numerator = 1;
-	fival->stepwise.min.denominator = inst->cap_framerate.max;
+	fival->stepwise.min.denominator = FRATE_MAX(inst);
 	fival->stepwise.max.numerator = 1;
-	fival->stepwise.max.denominator = inst->cap_framerate.min;
+	fival->stepwise.max.denominator = FRATE_MIN(inst);
 	fival->stepwise.step.numerator = 1;
-	fival->stepwise.step.denominator = inst->cap_framerate.max;
+	fival->stepwise.step.denominator = FRATE_MAX(inst);
 
 	return 0;
 }
@@ -1091,22 +1092,7 @@ static void venc_inst_init(struct venus_inst *inst)
 	inst->fps = 15;
 	inst->timeperframe.numerator = 1;
 	inst->timeperframe.denominator = 15;
-
-	inst->cap_width.min = 96;
-	inst->cap_width.max = 1920;
-	if (inst->core->res->hfi_version == HFI_VERSION_3XX)
-		inst->cap_width.max = 3840;
-	inst->cap_width.step_size = 2;
-	inst->cap_height.min = 64;
-	inst->cap_height.max = ALIGN(1080, 32);
-	if (inst->core->res->hfi_version == HFI_VERSION_3XX)
-		inst->cap_height.max = ALIGN(2160, 32);
-	inst->cap_height.step_size = 2;
-	inst->cap_framerate.min = 1;
-	inst->cap_framerate.max = 30;
-	inst->cap_framerate.step_size = 1;
-	inst->cap_mbs_per_frame.min = 24;
-	inst->cap_mbs_per_frame.max = 8160;
+	inst->hfi_codec = HFI_VIDEO_CODEC_H264;
 }
 
 static int venc_open(struct file *file)
-- 
2.14.1

Re: [PATCH 11/28] venus: add common capability parser

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 2252 bytes --]

Hi Stanimir,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linuxtv-media/master]
[also build test WARNING on v4.17-rc2 next-20180424]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Stanimir-Varbanov/Venus-updates/20180426-030344
base:   git://linuxtv.org/media_tree.git master
config: i386-allmodconfig (attached as .config)
compiler: gcc-7 (Debian 7.3.0-16) 7.3.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All warnings (new ones prefixed by >>):

   drivers/media/platform/qcom/venus/core.c: In function 'venus_enumerate_codecs':
>> drivers/media/platform/qcom/venus/core.c:225:1: warning: the frame size of 1040 bytes is larger than 1024 bytes [-Wframe-larger-than=]
    }
    ^

vim +225 drivers/media/platform/qcom/venus/core.c

   181	
   182	static int venus_enumerate_codecs(struct venus_core *core, u32 type)
   183	{
   184		const struct hfi_inst_ops dummy_ops = {};
   185		struct venus_inst inst;
   186		unsigned int i;
   187		u32 codec, codecs;
   188		int ret;
   189	
   190		if (core->res->hfi_version != HFI_VERSION_1XX)
   191			return 0;
   192	
   193		memset(&inst, 0, sizeof(inst));
   194		mutex_init(&inst.lock);
   195		inst.core = core;
   196		inst.session_type = type;
   197		if (type == VIDC_SESSION_TYPE_DEC)
   198			codecs = core->dec_codecs;
   199		else
   200			codecs = core->enc_codecs;
   201	
   202		ret = hfi_session_create(&inst, &dummy_ops);
   203		if (ret)
   204			return ret;
   205	
   206		for (i = 0; i < MAX_CODEC_NUM; i++) {
   207			codec = (1 << i) & codecs;
   208			if (!codec)
   209				continue;
   210	
   211			ret = hfi_session_init(&inst, to_v4l2_codec_type(codec));
   212			if (ret)
   213				goto done;
   214	
   215			ret = hfi_session_deinit(&inst);
   216			if (ret)
   217				goto done;
   218		}
   219	
   220	done:
   221		hfi_session_destroy(&inst);
   222		mutex_destroy(&inst.lock);
   223	
   224		return ret;
 > 225	}
   226	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 63277 bytes --]

Re: [PATCH 11/28] venus: add common capability parser

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 2252 bytes --]

Hi Stanimir,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linuxtv-media/master]
[also build test WARNING on v4.17-rc2 next-20180424]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Stanimir-Varbanov/Venus-updates/20180426-030344
base:   git://linuxtv.org/media_tree.git master
config: i386-allmodconfig (attached as .config)
compiler: gcc-7 (Debian 7.3.0-16) 7.3.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

All warnings (new ones prefixed by >>):

   drivers/media/platform/qcom/venus/core.c: In function 'venus_enumerate_codecs':
>> drivers/media/platform/qcom/venus/core.c:225:1: warning: the frame size of 1040 bytes is larger than 1024 bytes [-Wframe-larger-than=]
    }
    ^

vim +225 drivers/media/platform/qcom/venus/core.c

   181	
   182	static int venus_enumerate_codecs(struct venus_core *core, u32 type)
   183	{
   184		const struct hfi_inst_ops dummy_ops = {};
   185		struct venus_inst inst;
   186		unsigned int i;
   187		u32 codec, codecs;
   188		int ret;
   189	
   190		if (core->res->hfi_version != HFI_VERSION_1XX)
   191			return 0;
   192	
   193		memset(&inst, 0, sizeof(inst));
   194		mutex_init(&inst.lock);
   195		inst.core = core;
   196		inst.session_type = type;
   197		if (type == VIDC_SESSION_TYPE_DEC)
   198			codecs = core->dec_codecs;
   199		else
   200			codecs = core->enc_codecs;
   201	
   202		ret = hfi_session_create(&inst, &dummy_ops);
   203		if (ret)
   204			return ret;
   205	
   206		for (i = 0; i < MAX_CODEC_NUM; i++) {
   207			codec = (1 << i) & codecs;
   208			if (!codec)
   209				continue;
   210	
   211			ret = hfi_session_init(&inst, to_v4l2_codec_type(codec));
   212			if (ret)
   213				goto done;
   214	
   215			ret = hfi_session_deinit(&inst);
   216			if (ret)
   217				goto done;
   218		}
   219	
   220	done:
   221		hfi_session_destroy(&inst);
   222		mutex_destroy(&inst.lock);
   223	
   224		return ret;
 > 225	}
   226	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 63277 bytes --]

[PATCH 12/28] venus: helpers: make a commmon function for power_enable

[Stanimir Varbanov]

Make common function which will enable power when enabling/disabling
clocks and also covers Venus 3xx/4xx versions.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 51 +++++++++++++++++++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  2 ++
 drivers/media/platform/qcom/venus/vdec.c    | 25 ++++----------
 drivers/media/platform/qcom/venus/venc.c    | 25 ++++----------
 4 files changed, 67 insertions(+), 36 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index d9065cc8a7d3..2b21f6ed7502 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -13,6 +13,7 @@
  *
  */
 #include <linux/clk.h>
+#include <linux/iopoll.h>
 #include <linux/list.h>
 #include <linux/mutex.h>
 #include <linux/pm_runtime.h>
@@ -24,6 +25,7 @@
 #include "core.h"
 #include "helpers.h"
 #include "hfi_helper.h"
+#include "hfi_venus_io.h"
 
 struct intbuf {
 	struct list_head list;
@@ -781,3 +783,52 @@ void venus_helper_init_instance(struct venus_inst *inst)
 	}
 }
 EXPORT_SYMBOL_GPL(venus_helper_init_instance);
+
+int venus_helper_power_enable(struct venus_core *core, u32 session_type,
+			      bool enable)
+{
+	void __iomem *ctrl, *stat;
+	u32 val;
+	int ret;
+
+	if (!IS_V3(core) && !IS_V4(core))
+		return -EINVAL;
+
+	if (IS_V3(core)) {
+		if (session_type == VIDC_SESSION_TYPE_DEC)
+			ctrl = core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL;
+		else
+			ctrl = core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL;
+		if (enable)
+			writel(0, ctrl);
+		else
+			writel(1, ctrl);
+
+		return 0;
+	}
+
+	if (session_type == VIDC_SESSION_TYPE_DEC) {
+		ctrl = core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL;
+		stat = core->base + WRAPPER_VCODEC0_MMCC_POWER_STATUS;
+	} else {
+		ctrl = core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL;
+		stat = core->base + WRAPPER_VCODEC1_MMCC_POWER_STATUS;
+	}
+
+	if (enable) {
+		writel(0, ctrl);
+
+		ret = readl_poll_timeout(stat, val, val & BIT(1), 1, 100);
+		if (ret)
+			return ret;
+	} else {
+		writel(1, ctrl);
+
+		ret = readl_poll_timeout(stat, val, !(val & BIT(1)), 1, 100);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(venus_helper_power_enable);
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 971392be5df5..0e64aa95624a 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -43,4 +43,6 @@ int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
 void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
 void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
 void venus_helper_init_instance(struct venus_inst *inst);
+int venus_helper_power_enable(struct venus_core *core, u32 session_type,
+			      bool enable);
 #endif
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index cd278a695899..0ddc2c4df934 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -1131,26 +1131,21 @@ static int vdec_remove(struct platform_device *pdev)
 static __maybe_unused int vdec_runtime_suspend(struct device *dev)
 {
 	struct venus_core *core = dev_get_drvdata(dev);
+	int ret;
 
 	if (IS_V1(core))
 		return 0;
 
-	if (IS_V3(core))
-		writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(0, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+	ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, true);
 
 	if (IS_V4(core))
 		clk_disable_unprepare(core->core0_bus_clk);
 
 	clk_disable_unprepare(core->core0_clk);
 
-	if (IS_V3(core))
-		writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(1, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+	ret |= venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, false);
 
-	return 0;
+	return ret;
 }
 
 static __maybe_unused int vdec_runtime_resume(struct device *dev)
@@ -1161,20 +1156,14 @@ static __maybe_unused int vdec_runtime_resume(struct device *dev)
 	if (IS_V1(core))
 		return 0;
 
-	if (IS_V3(core))
-		writel(0, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(0, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+	ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, true);
 
-	ret = clk_prepare_enable(core->core0_clk);
+	ret |= clk_prepare_enable(core->core0_clk);
 
 	if (IS_V4(core))
 		ret |= clk_prepare_enable(core->core0_bus_clk);
 
-	if (IS_V3(core))
-		writel(1, core->base + WRAPPER_VDEC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(1, core->base + WRAPPER_VCODEC0_MMCC_POWER_CONTROL);
+	ret |= venus_helper_power_enable(core, VIDC_SESSION_TYPE_DEC, false);
 
 	return ret;
 }
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index be8ea3326386..f87d891325ea 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -1267,26 +1267,21 @@ static int venc_remove(struct platform_device *pdev)
 static __maybe_unused int venc_runtime_suspend(struct device *dev)
 {
 	struct venus_core *core = dev_get_drvdata(dev);
+	int ret;
 
 	if (IS_V1(core))
 		return 0;
 
-	if (IS_V3(core))
-		writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(0, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+	ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, true);
 
 	if (IS_V4(core))
 		clk_disable_unprepare(core->core1_bus_clk);
 
 	clk_disable_unprepare(core->core1_clk);
 
-	if (IS_V3(core))
-		writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(1, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+	ret |= venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, false);
 
-	return 0;
+	return ret;
 }
 
 static __maybe_unused int venc_runtime_resume(struct device *dev)
@@ -1297,20 +1292,14 @@ static __maybe_unused int venc_runtime_resume(struct device *dev)
 	if (IS_V1(core))
 		return 0;
 
-	if (IS_V3(core))
-		writel(0, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(0, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+	ret = venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, true);
 
-	ret = clk_prepare_enable(core->core1_clk);
+	ret |= clk_prepare_enable(core->core1_clk);
 
 	if (IS_V4(core))
 		ret |= clk_prepare_enable(core->core1_bus_clk);
 
-	if (IS_V3(core))
-		writel(1, core->base + WRAPPER_VENC_VCODEC_POWER_CONTROL);
-	else if (IS_V4(core))
-		writel(1, core->base + WRAPPER_VCODEC1_MMCC_POWER_CONTROL);
+	ret |= venus_helper_power_enable(core, VIDC_SESSION_TYPE_ENC, false);
 
 	return ret;
 }
-- 
2.14.1

[PATCH 13/28] venus: core: delete not used flag for buffer mode

[Stanimir Varbanov]

Delete not used flag for capture buffer allocation mode.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index fe2d2b9e8af8..c46334454cd9 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -249,7 +249,6 @@ struct venus_buffer {
  * @priv:	a private for HFI operations callbacks
  * @session_type:	the type of the session (decoder or encoder)
  * @hprop:	a union used as a holder by get property
- * @cap_bufs_mode_static:	buffers allocation mode capability
  * @cap_bufs_mode_dynamic:	buffers allocation mode capability
  */
 struct venus_inst {
@@ -299,7 +298,6 @@ struct venus_inst {
 	const struct hfi_inst_ops *ops;
 	u32 session_type;
 	union hfi_get_property hprop;
-	bool cap_bufs_mode_static;
 	bool cap_bufs_mode_dynamic;
 };
 
-- 
2.14.1

[PATCH 14/28] venus: helpers: rename a helper function and use buffer mode from caps

[Stanimir Varbanov]

Rename is_reg_unreg_needed() to better name is_dynamic_bufmode() and
use buffer mode from enumerated per codec capabilities.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 2b21f6ed7502..1eda19adbf28 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -354,18 +354,19 @@ session_process_buf(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf)
 	return 0;
 }
 
-static inline int is_reg_unreg_needed(struct venus_inst *inst)
+static inline int is_dynamic_bufmode(struct venus_inst *inst)
 {
-	if (inst->session_type == VIDC_SESSION_TYPE_DEC &&
-	    inst->core->res->hfi_version == HFI_VERSION_3XX)
-		return 0;
+	struct venus_core *core = inst->core;
+	struct venus_caps *caps;
 
-	if (inst->session_type == VIDC_SESSION_TYPE_DEC &&
-	    inst->cap_bufs_mode_dynamic &&
-	    inst->core->res->hfi_version == HFI_VERSION_1XX)
+	caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
+	if (!caps)
 		return 0;
 
-	return 1;
+	if (caps->cap_bufs_mode_dynamic)
+		return 1;
+
+	return 0;
 }
 
 static int session_unregister_bufs(struct venus_inst *inst)
@@ -374,7 +375,7 @@ static int session_unregister_bufs(struct venus_inst *inst)
 	struct hfi_buffer_desc bd;
 	int ret = 0;
 
-	if (!is_reg_unreg_needed(inst))
+	if (is_dynamic_bufmode(inst))
 		return 0;
 
 	list_for_each_entry_safe(buf, n, &inst->registeredbufs, reg_list) {
@@ -394,7 +395,7 @@ static int session_register_bufs(struct venus_inst *inst)
 	struct venus_buffer *buf;
 	int ret = 0;
 
-	if (!is_reg_unreg_needed(inst))
+	if (is_dynamic_bufmode(inst))
 		return 0;
 
 	list_for_each_entry(buf, &inst->registeredbufs, reg_list) {
-- 
2.14.1

[PATCH 15/28] venus: add a helper function to set dynamic buffer mode

[Stanimir Varbanov]

Adds a new helper function to set dymaic buffer mode if it is
supported by current HFI version.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 22 ++++++++++++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  1 +
 drivers/media/platform/qcom/venus/vdec.c    | 15 +++------------
 3 files changed, 26 insertions(+), 12 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 1eda19adbf28..824ad4d2d064 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -522,6 +522,28 @@ int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
 
+int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
+{
+	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
+	struct hfi_buffer_alloc_mode mode;
+	int ret;
+
+	if (!is_dynamic_bufmode(inst))
+		return 0;
+
+	mode.type = HFI_BUFFER_OUTPUT;
+	mode.mode = HFI_BUFFER_MODE_DYNAMIC;
+
+	ret = hfi_session_set_property(inst, ptype, &mode);
+	if (ret)
+		return ret;
+
+	mode.type = HFI_BUFFER_OUTPUT2;
+
+	return hfi_session_set_property(inst, ptype, &mode);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_dyn_bufmode);
+
 static void delayed_process_buf_func(struct work_struct *work)
 {
 	struct venus_buffer *buf, *n;
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 0e64aa95624a..52b961ed491e 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -40,6 +40,7 @@ int venus_helper_set_output_resolution(struct venus_inst *inst,
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs);
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
+int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
 void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
 void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
 void venus_helper_init_instance(struct venus_inst *inst);
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 0ddc2c4df934..1de9cc64cf2f 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -557,18 +557,9 @@ static int vdec_set_properties(struct venus_inst *inst)
 			return ret;
 	}
 
-	if (core->res->hfi_version == HFI_VERSION_3XX ||
-	    inst->cap_bufs_mode_dynamic) {
-		struct hfi_buffer_alloc_mode mode;
-
-		ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
-		mode.type = HFI_BUFFER_OUTPUT;
-		mode.mode = HFI_BUFFER_MODE_DYNAMIC;
-
-		ret = hfi_session_set_property(inst, ptype, &mode);
-		if (ret)
-			return ret;
-	}
+	ret = venus_helper_set_dyn_bufmode(inst);
+	if (ret)
+		return ret;
 
 	if (ctr->post_loop_deb_mode) {
 		ptype = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
-- 
2.14.1

Re: [PATCH 15/28] venus: add a helper function to set dynamic buffer mode

[Hans Verkuil]

On 24/04/18 14:44, Stanimir Varbanov wrote:
> Adds a new helper function to set dymaic buffer mode if it is

dymaic -> dynamic

Regards,

	Hans

> supported by current HFI version.
> 
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/helpers.c | 22 ++++++++++++++++++++++
>  drivers/media/platform/qcom/venus/helpers.h |  1 +
>  drivers/media/platform/qcom/venus/vdec.c    | 15 +++------------
>  3 files changed, 26 insertions(+), 12 deletions(-)
> 
> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
> index 1eda19adbf28..824ad4d2d064 100644
> --- a/drivers/media/platform/qcom/venus/helpers.c
> +++ b/drivers/media/platform/qcom/venus/helpers.c
> @@ -522,6 +522,28 @@ int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
>  }
>  EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
>  
> +int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
> +{
> +	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
> +	struct hfi_buffer_alloc_mode mode;
> +	int ret;
> +
> +	if (!is_dynamic_bufmode(inst))
> +		return 0;
> +
> +	mode.type = HFI_BUFFER_OUTPUT;
> +	mode.mode = HFI_BUFFER_MODE_DYNAMIC;
> +
> +	ret = hfi_session_set_property(inst, ptype, &mode);
> +	if (ret)
> +		return ret;
> +
> +	mode.type = HFI_BUFFER_OUTPUT2;
> +
> +	return hfi_session_set_property(inst, ptype, &mode);
> +}
> +EXPORT_SYMBOL_GPL(venus_helper_set_dyn_bufmode);
> +
>  static void delayed_process_buf_func(struct work_struct *work)
>  {
>  	struct venus_buffer *buf, *n;
> diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
> index 0e64aa95624a..52b961ed491e 100644
> --- a/drivers/media/platform/qcom/venus/helpers.h
> +++ b/drivers/media/platform/qcom/venus/helpers.h
> @@ -40,6 +40,7 @@ int venus_helper_set_output_resolution(struct venus_inst *inst,
>  int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
>  			      unsigned int output_bufs);
>  int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
> +int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
>  void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
>  void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
>  void venus_helper_init_instance(struct venus_inst *inst);
> diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
> index 0ddc2c4df934..1de9cc64cf2f 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -557,18 +557,9 @@ static int vdec_set_properties(struct venus_inst *inst)
>  			return ret;
>  	}
>  
> -	if (core->res->hfi_version == HFI_VERSION_3XX ||
> -	    inst->cap_bufs_mode_dynamic) {
> -		struct hfi_buffer_alloc_mode mode;
> -
> -		ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
> -		mode.type = HFI_BUFFER_OUTPUT;
> -		mode.mode = HFI_BUFFER_MODE_DYNAMIC;
> -
> -		ret = hfi_session_set_property(inst, ptype, &mode);
> -		if (ret)
> -			return ret;
> -	}
> +	ret = venus_helper_set_dyn_bufmode(inst);
> +	if (ret)
> +		return ret;
>  
>  	if (ctr->post_loop_deb_mode) {
>  		ptype = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
> 

[PATCH 16/28] venus: add helper function to set actual buffer size

[Stanimir Varbanov]

Add and use a helper function to set actual buffer size for
particular buffer type. This is also preparation to use
the second decoder output.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 12 ++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  1 +
 drivers/media/platform/qcom/venus/vdec.c    | 10 ++--------
 3 files changed, 15 insertions(+), 8 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 824ad4d2d064..94664a3ce3e2 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -544,6 +544,18 @@ int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_dyn_bufmode);
 
+int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype)
+{
+	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL;
+	struct hfi_buffer_size_actual bufsz;
+
+	bufsz.type = buftype;
+	bufsz.size = bufsize;
+
+	return hfi_session_set_property(inst, ptype, &bufsz);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_bufsize);
+
 static void delayed_process_buf_func(struct work_struct *work)
 {
 	struct venus_buffer *buf, *n;
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 52b961ed491e..cd306bd8978f 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -41,6 +41,7 @@ int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs);
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
+int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype);
 void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
 void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
 void venus_helper_init_instance(struct venus_inst *inst);
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 1de9cc64cf2f..b43607dee4fe 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -710,7 +710,6 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 {
 	struct venus_inst *inst = vb2_get_drv_priv(q);
 	struct venus_core *core = inst->core;
-	u32 ptype;
 	int ret;
 
 	mutex_lock(&inst->lock);
@@ -740,13 +739,8 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 		goto deinit_sess;
 
 	if (core->res->hfi_version == HFI_VERSION_3XX) {
-		struct hfi_buffer_size_actual buf_sz;
-
-		ptype = HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL;
-		buf_sz.type = HFI_BUFFER_OUTPUT;
-		buf_sz.size = inst->output_buf_size;
-
-		ret = hfi_session_set_property(inst, ptype, &buf_sz);
+		ret = venus_helper_set_bufsize(inst, inst->output_buf_size,
+					       HFI_BUFFER_OUTPUT);
 		if (ret)
 			goto deinit_sess;
 	}
-- 
2.14.1

[PATCH 17/28] venus: delete no longer used bufmode flag from instance

[Stanimir Varbanov]

Delete no longer used flag cap_bufs_mode_dynamic from instance
structure.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h       | 2 --
 drivers/media/platform/qcom/venus/hfi_parser.c | 6 +-----
 2 files changed, 1 insertion(+), 7 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index c46334454cd9..255292899204 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -249,7 +249,6 @@ struct venus_buffer {
  * @priv:	a private for HFI operations callbacks
  * @session_type:	the type of the session (decoder or encoder)
  * @hprop:	a union used as a holder by get property
- * @cap_bufs_mode_dynamic:	buffers allocation mode capability
  */
 struct venus_inst {
 	struct list_head list;
@@ -298,7 +297,6 @@ struct venus_inst {
 	const struct hfi_inst_ops *ops;
 	u32 session_type;
 	union hfi_get_property hprop;
-	bool cap_bufs_mode_dynamic;
 };
 
 #define IS_V1(core)	((core)->res->hfi_version == HFI_VERSION_1XX)
diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c b/drivers/media/platform/qcom/venus/hfi_parser.c
index 4b155997abbb..ac92fd347ce1 100644
--- a/drivers/media/platform/qcom/venus/hfi_parser.c
+++ b/drivers/media/platform/qcom/venus/hfi_parser.c
@@ -74,13 +74,9 @@ static void parse_alloc_mode(struct venus_core *core, struct venus_inst *inst,
 
 	while (num_entries--) {
 		if (mode->buffer_type == HFI_BUFFER_OUTPUT ||
-		    mode->buffer_type == HFI_BUFFER_OUTPUT2) {
-			if (*type == HFI_BUFFER_MODE_DYNAMIC && inst)
-				inst->cap_bufs_mode_dynamic = true;
-
+		    mode->buffer_type == HFI_BUFFER_OUTPUT2)
 			for_each_codec(core->caps, ARRAY_SIZE(core->caps),
 				       codecs, domain, fill_buf_mode, type, 1);
-		}
 
 		type++;
 	}
-- 
2.14.1

[PATCH 18/28] venus: helpers: add buffer type argument to a helper

[Stanimir Varbanov]

This adds one more function argument to pass buffer type to
set_output_resolution() helper function. That is a preparation
to support secondary decoder output.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 5 +++--
 drivers/media/platform/qcom/venus/helpers.h | 3 ++-
 drivers/media/platform/qcom/venus/venc.c    | 3 ++-
 3 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 94664a3ce3e2..5512fbfdebb9 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -456,12 +456,13 @@ int venus_helper_set_input_resolution(struct venus_inst *inst,
 EXPORT_SYMBOL_GPL(venus_helper_set_input_resolution);
 
 int venus_helper_set_output_resolution(struct venus_inst *inst,
-				       unsigned int width, unsigned int height)
+				       unsigned int width, unsigned int height,
+				       u32 buftype)
 {
 	u32 ptype = HFI_PROPERTY_PARAM_FRAME_SIZE;
 	struct hfi_framesize fs;
 
-	fs.buffer_type = HFI_BUFFER_OUTPUT;
+	fs.buffer_type = buftype;
 	fs.width = width;
 	fs.height = height;
 
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index cd306bd8978f..0de9989adcdb 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -36,7 +36,8 @@ int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
 int venus_helper_set_input_resolution(struct venus_inst *inst,
 				      unsigned int width, unsigned int height);
 int venus_helper_set_output_resolution(struct venus_inst *inst,
-				       unsigned int width, unsigned int height);
+				       unsigned int width, unsigned int height,
+				       u32 buftype);
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs);
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index f87d891325ea..8970f14b3a82 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -795,7 +795,8 @@ static int venc_init_session(struct venus_inst *inst)
 		goto deinit;
 
 	ret = venus_helper_set_output_resolution(inst, inst->width,
-						 inst->height);
+						 inst->height,
+						 HFI_BUFFER_OUTPUT);
 	if (ret)
 		goto deinit;
 
-- 
2.14.1

[PATCH 19/28] venus: helpers: add a new helper to set raw format

[Stanimir Varbanov]

The new helper will has one more argument for buffer type, that
way the decoder can configure the format on it's secondary
output.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 52 ++++++++++++++++++-----------
 drivers/media/platform/qcom/venus/helpers.h |  2 ++
 2 files changed, 35 insertions(+), 19 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 5512fbfdebb9..0d55604f7484 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -410,6 +410,20 @@ static int session_register_bufs(struct venus_inst *inst)
 	return ret;
 }
 
+static u32 to_hfi_raw_fmt(u32 v4l2_fmt)
+{
+	switch (v4l2_fmt) {
+	case V4L2_PIX_FMT_NV12:
+		return HFI_COLOR_FORMAT_NV12;
+	case V4L2_PIX_FMT_NV21:
+		return HFI_COLOR_FORMAT_NV21;
+	default:
+		break;
+	}
+
+	return 0;
+}
+
 int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
 			    struct hfi_buffer_requirements *req)
 {
@@ -491,35 +505,35 @@ int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_num_bufs);
 
-int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
+int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
+				u32 buftype)
 {
-	struct hfi_uncompressed_format_select fmt;
 	u32 ptype = HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT;
-	int ret;
+	struct hfi_uncompressed_format_select fmt;
+
+	fmt.buffer_type = buftype;
+	fmt.format = hfi_format;
+
+	return hfi_session_set_property(inst, ptype, &fmt);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_raw_format);
+
+int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
+{
+	u32 hfi_format, buftype;
 
 	if (inst->session_type == VIDC_SESSION_TYPE_DEC)
-		fmt.buffer_type = HFI_BUFFER_OUTPUT;
+		buftype = HFI_BUFFER_OUTPUT;
 	else if (inst->session_type == VIDC_SESSION_TYPE_ENC)
-		fmt.buffer_type = HFI_BUFFER_INPUT;
+		buftype = HFI_BUFFER_INPUT;
 	else
 		return -EINVAL;
 
-	switch (pixfmt) {
-	case V4L2_PIX_FMT_NV12:
-		fmt.format = HFI_COLOR_FORMAT_NV12;
-		break;
-	case V4L2_PIX_FMT_NV21:
-		fmt.format = HFI_COLOR_FORMAT_NV21;
-		break;
-	default:
+	hfi_format = to_hfi_raw_fmt(pixfmt);
+	if (!hfi_format)
 		return -EINVAL;
-	}
 
-	ret = hfi_session_set_property(inst, ptype, &fmt);
-	if (ret)
-		return ret;
-
-	return 0;
+	return venus_helper_set_raw_format(inst, hfi_format, buftype);
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
 
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 0de9989adcdb..79af7845efbd 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -40,6 +40,8 @@ int venus_helper_set_output_resolution(struct venus_inst *inst,
 				       u32 buftype);
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs);
+int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
+				u32 buftype);
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
 int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype);
-- 
2.14.1

[PATCH 20/28] venus: helpers,vdec,venc: add helpers to set work mode and core usage

[Stanimir Varbanov]

These are new properties applicable to Venus version 4xx. Add the
helpers and call them from decoder and encoder drivers.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 28 ++++++++++++++++++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  2 ++
 drivers/media/platform/qcom/venus/vdec.c    |  8 ++++++++
 drivers/media/platform/qcom/venus/venc.c    |  8 ++++++++
 4 files changed, 46 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 0d55604f7484..adf8701a64bb 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -484,6 +484,34 @@ int venus_helper_set_output_resolution(struct venus_inst *inst,
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_output_resolution);
 
+int venus_helper_set_work_mode(struct venus_inst *inst, u32 mode)
+{
+	u32 ptype = HFI_PROPERTY_PARAM_WORK_MODE;
+	struct hfi_video_work_mode wm;
+
+	if (!IS_V4(inst->core))
+		return 0;
+
+	wm.video_work_mode = mode;
+
+	return hfi_session_set_property(inst, ptype, &wm);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_work_mode);
+
+int venus_helper_set_core_usage(struct venus_inst *inst, u32 usage)
+{
+	u32 ptype = HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE;
+	struct hfi_videocores_usage_type cu;
+
+	if (!IS_V4(inst->core))
+		return 0;
+
+	cu.video_core_enable_mask = usage;
+
+	return hfi_session_set_property(inst, ptype, &cu);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_core_usage);
+
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs)
 {
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 79af7845efbd..d5e727e1ecab 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -38,6 +38,8 @@ int venus_helper_set_input_resolution(struct venus_inst *inst,
 int venus_helper_set_output_resolution(struct venus_inst *inst,
 				       unsigned int width, unsigned int height,
 				       u32 buftype);
+int venus_helper_set_work_mode(struct venus_inst *inst, u32 mode);
+int venus_helper_set_core_usage(struct venus_inst *inst, u32 usage);
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 			      unsigned int output_bufs);
 int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index b43607dee4fe..ceaf1a338eb3 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -550,6 +550,14 @@ static int vdec_set_properties(struct venus_inst *inst)
 	u32 ptype;
 	int ret;
 
+	ret = venus_helper_set_work_mode(inst, VIDC_WORK_MODE_2);
+	if (ret)
+		return ret;
+
+	ret = venus_helper_set_core_usage(inst, VIDC_CORE_ID_1);
+	if (ret)
+		return ret;
+
 	if (core->res->hfi_version == HFI_VERSION_1XX) {
 		ptype = HFI_PROPERTY_PARAM_VDEC_CONTINUE_DATA_TRANSFER;
 		ret = hfi_session_set_property(inst, ptype, &en);
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 8970f14b3a82..3b3299bff1cd 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -643,6 +643,14 @@ static int venc_set_properties(struct venus_inst *inst)
 	u32 ptype, rate_control, bitrate, profile = 0, level = 0;
 	int ret;
 
+	ret = venus_helper_set_work_mode(inst, VIDC_WORK_MODE_2);
+	if (ret)
+		return ret;
+
+	ret = venus_helper_set_core_usage(inst, VIDC_CORE_ID_2);
+	if (ret)
+		return ret;
+
 	ptype = HFI_PROPERTY_CONFIG_FRAME_RATE;
 	frate.buffer_type = HFI_BUFFER_OUTPUT;
 	frate.framerate = inst->fps * (1 << 16);
-- 
2.14.1

[PATCH 21/28] venus: helpers: extend set_num_bufs helper with one more argument

[Stanimir Varbanov]

Extend venus_helper_set_num_bufs() helper function with one more
argument to set number of output buffers for the secondary decoder
output.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 16 ++++++++++++++--
 drivers/media/platform/qcom/venus/helpers.h |  3 ++-
 drivers/media/platform/qcom/venus/vdec.c    |  2 +-
 drivers/media/platform/qcom/venus/venc.c    |  2 +-
 4 files changed, 18 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index adf8701a64bb..f04d16953b3a 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -513,7 +513,8 @@ int venus_helper_set_core_usage(struct venus_inst *inst, u32 usage)
 EXPORT_SYMBOL_GPL(venus_helper_set_core_usage);
 
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
-			      unsigned int output_bufs)
+			      unsigned int output_bufs,
+			      unsigned int output2_bufs)
 {
 	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL;
 	struct hfi_buffer_count_actual buf_count;
@@ -529,7 +530,18 @@ int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
 	buf_count.type = HFI_BUFFER_OUTPUT;
 	buf_count.count_actual = output_bufs;
 
-	return hfi_session_set_property(inst, ptype, &buf_count);
+	ret = hfi_session_set_property(inst, ptype, &buf_count);
+	if (ret)
+		return ret;
+
+	if (output2_bufs) {
+		buf_count.type = HFI_BUFFER_OUTPUT2;
+		buf_count.count_actual = output2_bufs;
+
+		ret = hfi_session_set_property(inst, ptype, &buf_count);
+	}
+
+	return ret;
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_num_bufs);
 
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index d5e727e1ecab..8ff4bd3ef958 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -41,7 +41,8 @@ int venus_helper_set_output_resolution(struct venus_inst *inst,
 int venus_helper_set_work_mode(struct venus_inst *inst, u32 mode);
 int venus_helper_set_core_usage(struct venus_inst *inst, u32 usage);
 int venus_helper_set_num_bufs(struct venus_inst *inst, unsigned int input_bufs,
-			      unsigned int output_bufs);
+			      unsigned int output_bufs,
+			      unsigned int output2_bufs);
 int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
 				u32 buftype);
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index ceaf1a338eb3..8d188b11b85a 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -758,7 +758,7 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 		goto deinit_sess;
 
 	ret = venus_helper_set_num_bufs(inst, inst->num_input_bufs,
-					VB2_MAX_FRAME);
+					VB2_MAX_FRAME, VB2_MAX_FRAME);
 	if (ret)
 		goto deinit_sess;
 
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 3b3299bff1cd..c9c40d1ce7c6 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -963,7 +963,7 @@ static int venc_start_streaming(struct vb2_queue *q, unsigned int count)
 		goto deinit_sess;
 
 	ret = venus_helper_set_num_bufs(inst, inst->num_input_bufs,
-					inst->num_output_bufs);
+					inst->num_output_bufs, 0);
 	if (ret)
 		goto deinit_sess;
 
-- 
2.14.1

[PATCH 22/28] venus: helpers: add a helper to return opb buffer sizes

[Stanimir Varbanov]

Add a helper function to return current output picture buffer size.
OPB sizes can vary depending on the selected decoder output(s).

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h    | 10 ++++++++++
 drivers/media/platform/qcom/venus/helpers.c | 15 +++++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  1 +
 3 files changed, 26 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 255292899204..4d6c05f156c4 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -234,6 +234,11 @@ struct venus_buffer {
  * @num_output_bufs:	holds number of output buffers
  * @input_buf_size	holds input buffer size
  * @output_buf_size:	holds output buffer size
+ * @output2_buf_size:	holds secondary decoder output buffer size
+ * @dpb_buftype:	decoded picture buffer type
+ * @dpb_fmt:		decodec picture buffre raw format
+ * @opb_buftype:	output picture buffer type
+ * @opb_fmt:		output picture buffer raw format
  * @reconfig:	a flag raised by decoder when the stream resolution changed
  * @reconfig_width:	holds the new width
  * @reconfig_height:	holds the new height
@@ -282,6 +287,11 @@ struct venus_inst {
 	unsigned int num_output_bufs;
 	unsigned int input_buf_size;
 	unsigned int output_buf_size;
+	unsigned int output2_buf_size;
+	u32 dpb_buftype;
+	u32 dpb_fmt;
+	u32 opb_buftype;
+	u32 opb_fmt;
 	bool reconfig;
 	u32 reconfig_width;
 	u32 reconfig_height;
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index f04d16953b3a..f0a0fca60c76 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -611,6 +611,21 @@ int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype)
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_bufsize);
 
+unsigned int venus_helper_get_opb_size(struct venus_inst *inst)
+{
+	/* the encoder has only one output */
+	if (inst->session_type == VIDC_SESSION_TYPE_ENC)
+		return inst->output_buf_size;
+
+	if (inst->opb_buftype == HFI_BUFFER_OUTPUT)
+		return inst->output_buf_size;
+	else if (inst->opb_buftype == HFI_BUFFER_OUTPUT2)
+		return inst->output2_buf_size;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(venus_helper_get_opb_size);
+
 static void delayed_process_buf_func(struct work_struct *work)
 {
 	struct venus_buffer *buf, *n;
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 8ff4bd3ef958..92be45894a69 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -48,6 +48,7 @@ int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
 int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype);
+unsigned int venus_helper_get_opb_size(struct venus_inst *inst);
 void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
 void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
 void venus_helper_init_instance(struct venus_inst *inst);
-- 
2.14.1

[PATCH 23/28] venus: vdec: get required input buffers as well

[Stanimir Varbanov]

Rework and rename vdec_cap_num_buffers() to get the number of
input buffers too.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/vdec.c | 41 +++++++++++++++++++-------------
 1 file changed, 24 insertions(+), 17 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 8d188b11b85a..6ed9b7c4bd6e 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -603,19 +603,32 @@ static int vdec_init_session(struct venus_inst *inst)
 	return ret;
 }
 
-static int vdec_cap_num_buffers(struct venus_inst *inst, unsigned int *num)
+static int vdec_num_buffers(struct venus_inst *inst, unsigned int *in_num,
+			    unsigned int *out_num)
 {
+	enum hfi_version ver = inst->core->res->hfi_version;
 	struct hfi_buffer_requirements bufreq;
 	int ret;
 
+	*in_num = *out_num = 0;
+
 	ret = vdec_init_session(inst);
 	if (ret)
 		return ret;
 
+	ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);
+	if (ret)
+		goto deinit;
+
+	*in_num = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
+
 	ret = venus_helper_get_bufreq(inst, HFI_BUFFER_OUTPUT, &bufreq);
+	if (ret)
+		goto deinit;
 
-	*num = bufreq.count_actual;
+	*out_num = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
 
+deinit:
 	hfi_session_deinit(inst);
 
 	return ret;
@@ -626,7 +639,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
 			    unsigned int sizes[], struct device *alloc_devs[])
 {
 	struct venus_inst *inst = vb2_get_drv_priv(q);
-	unsigned int p, num;
+	unsigned int p, in_num, out_num;
 	int ret = 0;
 
 	if (*num_planes) {
@@ -649,35 +662,29 @@ static int vdec_queue_setup(struct vb2_queue *q,
 		return 0;
 	}
 
+	ret = vdec_num_buffers(inst, &in_num, &out_num);
+	if (ret)
+		return ret;
+
 	switch (q->type) {
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
 		*num_planes = inst->fmt_out->num_planes;
 		sizes[0] = get_framesize_compressed(inst->out_width,
 						    inst->out_height);
 		inst->input_buf_size = sizes[0];
+		*num_buffers = max(*num_buffers, in_num);
 		inst->num_input_bufs = *num_buffers;
-
-		ret = vdec_cap_num_buffers(inst, &num);
-		if (ret)
-			break;
-
-		inst->num_output_bufs = num;
+		inst->num_output_bufs = out_num;
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
 		*num_planes = inst->fmt_cap->num_planes;
 
-		ret = vdec_cap_num_buffers(inst, &num);
-		if (ret)
-			break;
-
-		*num_buffers = max(*num_buffers, num);
-
 		for (p = 0; p < *num_planes; p++)
 			sizes[p] = get_framesize_uncompressed(p, inst->width,
 							      inst->height);
-
-		inst->num_output_bufs = *num_buffers;
 		inst->output_buf_size = sizes[0];
+		*num_buffers = max(*num_buffers, out_num);
+		inst->num_output_bufs = *num_buffers;
 		break;
 	default:
 		ret = -EINVAL;
-- 
2.14.1

[PATCH 24/28] venus: vdec: new function for output configuration

[Stanimir Varbanov]

Make a new function vdec_output_conf() for decoder output
configuration. vdec_output_conf() will set properties via
HFI interface related to the output configuration, and
keep vdec_set_properties() which will set properties
related to decoding parameters.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/vdec.c | 35 ++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 15 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 6ed9b7c4bd6e..ced3330c396a 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -545,6 +545,23 @@ static const struct v4l2_ioctl_ops vdec_ioctl_ops = {
 static int vdec_set_properties(struct venus_inst *inst)
 {
 	struct vdec_controls *ctr = &inst->controls.dec;
+	struct hfi_enable en = { .enable = 1 };
+	u32 ptype;
+	int ret;
+
+	if (ctr->post_loop_deb_mode) {
+		ptype = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
+		en.enable = 1;
+		ret = hfi_session_set_property(inst, ptype, &en);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+static int vdec_output_conf(struct venus_inst *inst)
+{
 	struct venus_core *core = inst->core;
 	struct hfi_enable en = { .enable = 1 };
 	u32 ptype;
@@ -569,14 +586,6 @@ static int vdec_set_properties(struct venus_inst *inst)
 	if (ret)
 		return ret;
 
-	if (ctr->post_loop_deb_mode) {
-		ptype = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER;
-		en.enable = 1;
-		ret = hfi_session_set_property(inst, ptype, &en);
-		if (ret)
-			return ret;
-	}
-
 	return 0;
 }
 
@@ -724,7 +733,6 @@ static int vdec_verify_conf(struct venus_inst *inst)
 static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 {
 	struct venus_inst *inst = vb2_get_drv_priv(q);
-	struct venus_core *core = inst->core;
 	int ret;
 
 	mutex_lock(&inst->lock);
@@ -753,12 +761,9 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 	if (ret)
 		goto deinit_sess;
 
-	if (core->res->hfi_version == HFI_VERSION_3XX) {
-		ret = venus_helper_set_bufsize(inst, inst->output_buf_size,
-					       HFI_BUFFER_OUTPUT);
-		if (ret)
-			goto deinit_sess;
-	}
+	ret = vdec_output_conf(inst);
+	if (ret)
+		goto deinit_sess;
 
 	ret = vdec_verify_conf(inst);
 	if (ret)
-- 
2.14.1

[PATCH 25/28] venus: move frame size calculations in common place

[Stanimir Varbanov]

move calculations of raw and compressed in a common helper
and make it identical for encoder and decoder.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 98 +++++++++++++++++++++++++++++
 drivers/media/platform/qcom/venus/helpers.h |  2 +
 drivers/media/platform/qcom/venus/vdec.c    | 54 ++++------------
 drivers/media/platform/qcom/venus/venc.c    | 56 ++++-------------
 4 files changed, 126 insertions(+), 84 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index f0a0fca60c76..ed569705ecac 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -455,6 +455,104 @@ int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
 }
 EXPORT_SYMBOL_GPL(venus_helper_get_bufreq);
 
+static u32 get_framesize_raw_nv12(u32 width, u32 height)
+{
+	u32 y_stride, uv_stride, y_plane;
+	u32 y_sclines, uv_sclines, uv_plane;
+	u32 size;
+
+	y_stride = ALIGN(width, 128);
+	uv_stride = ALIGN(width, 128);
+	y_sclines = ALIGN(height, 32);
+	uv_sclines = ALIGN(((height + 1) >> 1), 16);
+
+	y_plane = y_stride * y_sclines;
+	uv_plane = uv_stride * uv_sclines + SZ_4K;
+	size = y_plane + uv_plane + SZ_8K;
+
+	return ALIGN(size, SZ_4K);
+}
+
+static u32 get_framesize_raw_nv12_ubwc(u32 width, u32 height)
+{
+	u32 y_meta_stride, y_meta_plane;
+	u32 y_stride, y_plane;
+	u32 uv_meta_stride, uv_meta_plane;
+	u32 uv_stride, uv_plane;
+	u32 extradata = SZ_16K;
+
+	y_meta_stride = ALIGN(DIV_ROUND_UP(width, 32), 64);
+	y_meta_plane = y_meta_stride * ALIGN(DIV_ROUND_UP(height, 8), 16);
+	y_meta_plane = ALIGN(y_meta_plane, SZ_4K);
+
+	y_stride = ALIGN(width, 128);
+	y_plane = ALIGN(y_stride * ALIGN(height, 32), SZ_4K);
+
+	uv_meta_stride = ALIGN(DIV_ROUND_UP(width / 2, 16), 64);
+	uv_meta_plane = uv_meta_stride * ALIGN(DIV_ROUND_UP(height / 2, 8), 16);
+	uv_meta_plane = ALIGN(uv_meta_plane, SZ_4K);
+
+	uv_stride = ALIGN(width, 128);
+	uv_plane = ALIGN(uv_stride * ALIGN(height / 2, 32), SZ_4K);
+
+	return ALIGN(y_meta_plane + y_plane + uv_meta_plane + uv_plane +
+		     max(extradata, y_stride * 48), SZ_4K);
+}
+
+u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height)
+{
+	switch (hfi_fmt) {
+	case HFI_COLOR_FORMAT_NV12:
+	case HFI_COLOR_FORMAT_NV21:
+		return get_framesize_raw_nv12(width, height);
+	case HFI_COLOR_FORMAT_NV12_UBWC:
+		return get_framesize_raw_nv12_ubwc(width, height);
+	default:
+		return 0;
+	}
+}
+EXPORT_SYMBOL_GPL(venus_helper_get_framesz_raw);
+
+u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height)
+{
+	u32 hfi_fmt, sz;
+	bool compressed;
+
+	switch (v4l2_fmt) {
+	case V4L2_PIX_FMT_MPEG:
+	case V4L2_PIX_FMT_H264:
+	case V4L2_PIX_FMT_H264_NO_SC:
+	case V4L2_PIX_FMT_H264_MVC:
+	case V4L2_PIX_FMT_H263:
+	case V4L2_PIX_FMT_MPEG1:
+	case V4L2_PIX_FMT_MPEG2:
+	case V4L2_PIX_FMT_MPEG4:
+	case V4L2_PIX_FMT_XVID:
+	case V4L2_PIX_FMT_VC1_ANNEX_G:
+	case V4L2_PIX_FMT_VC1_ANNEX_L:
+	case V4L2_PIX_FMT_VP8:
+	case V4L2_PIX_FMT_VP9:
+	case V4L2_PIX_FMT_HEVC:
+		compressed = true;
+		break;
+	default:
+		compressed = false;
+		break;
+	}
+
+	if (compressed) {
+		sz = ALIGN(height, 32) * ALIGN(width, 32) * 3 / 2 / 2;
+		return ALIGN(sz, SZ_4K);
+	}
+
+	hfi_fmt = to_hfi_raw_fmt(v4l2_fmt);
+	if (!hfi_fmt)
+		return 0;
+
+	return venus_helper_get_framesz_raw(hfi_fmt, width, height);
+}
+EXPORT_SYMBOL_GPL(venus_helper_get_framesz);
+
 int venus_helper_set_input_resolution(struct venus_inst *inst,
 				      unsigned int width, unsigned int height)
 {
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 92be45894a69..92b167a47166 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -33,6 +33,8 @@ void venus_helper_m2m_device_run(void *priv);
 void venus_helper_m2m_job_abort(void *priv);
 int venus_helper_get_bufreq(struct venus_inst *inst, u32 type,
 			    struct hfi_buffer_requirements *req);
+u32 venus_helper_get_framesz_raw(u32 hfi_fmt, u32 width, u32 height);
+u32 venus_helper_get_framesz(u32 v4l2_fmt, u32 width, u32 height);
 int venus_helper_set_input_resolution(struct venus_inst *inst,
 				      unsigned int width, unsigned int height);
 int venus_helper_set_output_resolution(struct venus_inst *inst,
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index ced3330c396a..589fc13b84bc 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -29,29 +29,6 @@
 #include "helpers.h"
 #include "vdec.h"
 
-static u32 get_framesize_uncompressed(unsigned int plane, u32 width, u32 height)
-{
-	u32 y_stride, uv_stride, y_plane;
-	u32 y_sclines, uv_sclines, uv_plane;
-	u32 size;
-
-	y_stride = ALIGN(width, 128);
-	uv_stride = ALIGN(width, 128);
-	y_sclines = ALIGN(height, 32);
-	uv_sclines = ALIGN(((height + 1) >> 1), 16);
-
-	y_plane = y_stride * y_sclines;
-	uv_plane = uv_stride * uv_sclines + SZ_4K;
-	size = y_plane + uv_plane + SZ_8K;
-
-	return ALIGN(size, SZ_4K);
-}
-
-static u32 get_framesize_compressed(unsigned int width, unsigned int height)
-{
-	return ((width * height * 3 / 2) / 2) + 128;
-}
-
 /*
  * Three resons to keep MPLANE formats (despite that the number of planes
  * currently is one):
@@ -160,7 +137,6 @@ vdec_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
 	struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
 	const struct venus_format *fmt;
-	unsigned int p;
 
 	memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
 	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
@@ -191,18 +167,14 @@ vdec_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 	pixmp->num_planes = fmt->num_planes;
 	pixmp->flags = 0;
 
-	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
-		for (p = 0; p < pixmp->num_planes; p++) {
-			pfmt[p].sizeimage =
-				get_framesize_uncompressed(p, pixmp->width,
-							   pixmp->height);
-			pfmt[p].bytesperline = ALIGN(pixmp->width, 128);
-		}
-	} else {
-		pfmt[0].sizeimage = get_framesize_compressed(pixmp->width,
-							     pixmp->height);
+	pfmt[0].sizeimage = venus_helper_get_framesz(pixmp->pixelformat,
+						     pixmp->width,
+						     pixmp->height);
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		pfmt[0].bytesperline = ALIGN(pixmp->width, 128);
+	else
 		pfmt[0].bytesperline = 0;
-	}
 
 	return fmt;
 }
@@ -648,7 +620,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
 			    unsigned int sizes[], struct device *alloc_devs[])
 {
 	struct venus_inst *inst = vb2_get_drv_priv(q);
-	unsigned int p, in_num, out_num;
+	unsigned int in_num, out_num;
 	int ret = 0;
 
 	if (*num_planes) {
@@ -678,7 +650,8 @@ static int vdec_queue_setup(struct vb2_queue *q,
 	switch (q->type) {
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
 		*num_planes = inst->fmt_out->num_planes;
-		sizes[0] = get_framesize_compressed(inst->out_width,
+		sizes[0] = venus_helper_get_framesz(inst->fmt_out->pixfmt,
+						    inst->out_width,
 						    inst->out_height);
 		inst->input_buf_size = sizes[0];
 		*num_buffers = max(*num_buffers, in_num);
@@ -687,10 +660,9 @@ static int vdec_queue_setup(struct vb2_queue *q,
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
 		*num_planes = inst->fmt_cap->num_planes;
-
-		for (p = 0; p < *num_planes; p++)
-			sizes[p] = get_framesize_uncompressed(p, inst->width,
-							      inst->height);
+		sizes[0] = venus_helper_get_framesz(inst->fmt_cap->pixfmt,
+						    inst->width,
+						    inst->height);
 		inst->output_buf_size = sizes[0];
 		*num_buffers = max(*num_buffers, out_num);
 		inst->num_output_bufs = *num_buffers;
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index c9c40d1ce7c6..54f253b98b24 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -31,32 +31,6 @@
 
 #define NUM_B_FRAMES_MAX	4
 
-static u32 get_framesize_uncompressed(unsigned int plane, u32 width, u32 height)
-{
-	u32 y_stride, uv_stride, y_plane;
-	u32 y_sclines, uv_sclines, uv_plane;
-	u32 size;
-
-	y_stride = ALIGN(width, 128);
-	uv_stride = ALIGN(width, 128);
-	y_sclines = ALIGN(height, 32);
-	uv_sclines = ALIGN(((height + 1) >> 1), 16);
-
-	y_plane = y_stride * y_sclines;
-	uv_plane = uv_stride * uv_sclines + SZ_4K;
-	size = y_plane + uv_plane + SZ_8K;
-	size = ALIGN(size, SZ_4K);
-
-	return size;
-}
-
-static u32 get_framesize_compressed(u32 width, u32 height)
-{
-	u32 sz = ALIGN(height, 32) * ALIGN(width, 32) * 3 / 2 / 2;
-
-	return ALIGN(sz, SZ_4K);
-}
-
 /*
  * Three resons to keep MPLANE formats (despite that the number of planes
  * currently is one):
@@ -284,7 +258,6 @@ venc_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
 	struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
 	const struct venus_format *fmt;
-	unsigned int p;
 
 	memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
 	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
@@ -318,19 +291,14 @@ venc_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
 	pixmp->num_planes = fmt->num_planes;
 	pixmp->flags = 0;
 
-	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
-		for (p = 0; p < pixmp->num_planes; p++) {
-			pfmt[p].sizeimage =
-				get_framesize_uncompressed(p, pixmp->width,
-							   pixmp->height);
+	pfmt[0].sizeimage = venus_helper_get_framesz(pixmp->pixelformat,
+						     pixmp->width,
+						     pixmp->height);
 
-			pfmt[p].bytesperline = ALIGN(pixmp->width, 128);
-		}
-	} else {
-		pfmt[0].sizeimage = get_framesize_compressed(pixmp->width,
-							     pixmp->height);
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+		pfmt[0].bytesperline = ALIGN(pixmp->width, 128);
+	else
 		pfmt[0].bytesperline = 0;
-	}
 
 	return fmt;
 }
@@ -845,7 +813,7 @@ static int venc_queue_setup(struct vb2_queue *q,
 			    unsigned int sizes[], struct device *alloc_devs[])
 {
 	struct venus_inst *inst = vb2_get_drv_priv(q);
-	unsigned int p, num, min = 4;
+	unsigned int num, min = 4;
 	int ret = 0;
 
 	if (*num_planes) {
@@ -880,16 +848,18 @@ static int venc_queue_setup(struct vb2_queue *q,
 		*num_buffers = max(*num_buffers, num);
 		inst->num_input_bufs = *num_buffers;
 
-		for (p = 0; p < *num_planes; ++p)
-			sizes[p] = get_framesize_uncompressed(p, inst->width,
-							      inst->height);
+		sizes[0] = venus_helper_get_framesz(inst->fmt_out->pixfmt,
+						    inst->width,
+						    inst->height);
 		inst->input_buf_size = sizes[0];
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
 		*num_planes = inst->fmt_cap->num_planes;
 		*num_buffers = max(*num_buffers, min);
 		inst->num_output_bufs = *num_buffers;
-		sizes[0] = get_framesize_compressed(inst->width, inst->height);
+		sizes[0] = venus_helper_get_framesz(inst->fmt_cap->pixfmt,
+						    inst->width,
+						    inst->height);
 		inst->output_buf_size = sizes[0];
 		break;
 	default:
-- 
2.14.1

[PATCH 26/28] venus: implementing multi-stream support

[Stanimir Varbanov]

This is implementing a multi-stream decoder support. The multi
stream gives an option to use the secondary decoder output
with different raw format (or the same in case of crop).

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/core.h    |   1 +
 drivers/media/platform/qcom/venus/helpers.c | 204 +++++++++++++++++++++++++++-
 drivers/media/platform/qcom/venus/helpers.h |   6 +
 drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
 drivers/media/platform/qcom/venus/venc.c    |   1 +
 5 files changed, 299 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 4d6c05f156c4..85e66e2dd672 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -259,6 +259,7 @@ struct venus_inst {
 	struct list_head list;
 	struct mutex lock;
 	struct venus_core *core;
+	struct list_head dpbbufs;
 	struct list_head internalbufs;
 	struct list_head registeredbufs;
 	struct list_head delayed_process;
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index ed569705ecac..87dcf9973e6f 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -85,6 +85,112 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
 }
 EXPORT_SYMBOL_GPL(venus_helper_check_codec);
 
+static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
+{
+	struct intbuf *buf;
+	int ret = 0;
+
+	if (list_empty(&inst->dpbbufs))
+		return 0;
+
+	list_for_each_entry(buf, &inst->dpbbufs, list) {
+		struct hfi_frame_data fdata;
+
+		memset(&fdata, 0, sizeof(fdata));
+		fdata.alloc_len = buf->size;
+		fdata.device_addr = buf->da;
+		fdata.buffer_type = buf->type;
+
+		ret = hfi_session_process_buf(inst, &fdata);
+		if (ret)
+			goto fail;
+	}
+
+fail:
+	return ret;
+}
+
+int venus_helper_free_dpb_bufs(struct venus_inst *inst)
+{
+	struct intbuf *buf, *n;
+
+	if (list_empty(&inst->dpbbufs))
+		return 0;
+
+	list_for_each_entry_safe(buf, n, &inst->dpbbufs, list) {
+		list_del_init(&buf->list);
+		dma_free_attrs(inst->core->dev, buf->size, buf->va, buf->da,
+			       buf->attrs);
+		kfree(buf);
+	}
+
+	INIT_LIST_HEAD(&inst->dpbbufs);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(venus_helper_free_dpb_bufs);
+
+int venus_helper_alloc_dpb_bufs(struct venus_inst *inst)
+{
+	struct venus_core *core = inst->core;
+	struct device *dev = core->dev;
+	enum hfi_version ver = core->res->hfi_version;
+	struct hfi_buffer_requirements bufreq;
+	u32 buftype = inst->dpb_buftype;
+	unsigned int dpb_size = 0;
+	struct intbuf *buf;
+	unsigned int i;
+	u32 count;
+	int ret;
+
+	/* no need to allocate dpb buffers */
+	if (!inst->dpb_fmt)
+		return 0;
+
+	if (inst->dpb_buftype == HFI_BUFFER_OUTPUT)
+		dpb_size = inst->output_buf_size;
+	else if (inst->dpb_buftype == HFI_BUFFER_OUTPUT2)
+		dpb_size = inst->output2_buf_size;
+
+	if (!dpb_size)
+		return 0;
+
+	ret = venus_helper_get_bufreq(inst, buftype, &bufreq);
+	if (ret)
+		return ret;
+
+	count = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
+
+	for (i = 0; i < count; i++) {
+		buf = kzalloc(sizeof(*buf), GFP_KERNEL);
+		if (!buf) {
+			ret = -ENOMEM;
+			goto fail;
+		}
+
+		buf->type = buftype;
+		buf->size = dpb_size;
+		buf->attrs = DMA_ATTR_WRITE_COMBINE |
+			     DMA_ATTR_NO_KERNEL_MAPPING;
+		buf->va = dma_alloc_attrs(dev, buf->size, &buf->da, GFP_KERNEL,
+					  buf->attrs);
+		if (!buf->va) {
+			kfree(buf);
+			ret = -ENOMEM;
+			goto fail;
+		}
+
+		list_add_tail(&buf->list, &inst->dpbbufs);
+	}
+
+	return 0;
+
+fail:
+	venus_helper_free_dpb_bufs(inst);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(venus_helper_alloc_dpb_bufs);
+
 static int intbufs_set_buffer(struct venus_inst *inst, u32 type)
 {
 	struct venus_core *core = inst->core;
@@ -342,7 +448,10 @@ session_process_buf(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf)
 		if (vbuf->flags & V4L2_BUF_FLAG_LAST || !fdata.filled_len)
 			fdata.flags |= HFI_BUFFERFLAG_EOS;
 	} else if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
-		fdata.buffer_type = HFI_BUFFER_OUTPUT;
+		if (inst->session_type == VIDC_SESSION_TYPE_ENC)
+			fdata.buffer_type = HFI_BUFFER_OUTPUT;
+		else
+			fdata.buffer_type = inst->opb_buftype;
 		fdata.filled_len = 0;
 		fdata.offset = 0;
 	}
@@ -675,6 +784,27 @@ int venus_helper_set_color_format(struct venus_inst *inst, u32 pixfmt)
 }
 EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
 
+int venus_helper_set_multistream(struct venus_inst *inst, bool out_en,
+				 bool out2_en)
+{
+	struct hfi_multi_stream multi = {0};
+	u32 ptype = HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM;
+	int ret;
+
+	multi.buffer_type = HFI_BUFFER_OUTPUT;
+	multi.enable = out_en;
+
+	ret = hfi_session_set_property(inst, ptype, &multi);
+	if (ret)
+		return ret;
+
+	multi.buffer_type = HFI_BUFFER_OUTPUT2;
+	multi.enable = out2_en;
+
+	return hfi_session_set_property(inst, ptype, &multi);
+}
+EXPORT_SYMBOL_GPL(venus_helper_set_multistream);
+
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
 {
 	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
@@ -822,9 +952,10 @@ EXPORT_SYMBOL_GPL(venus_helper_vb2_buf_init);
 int venus_helper_vb2_buf_prepare(struct vb2_buffer *vb)
 {
 	struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+	unsigned int out_buf_size = venus_helper_get_opb_size(inst);
 
 	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
-	    vb2_plane_size(vb, 0) < inst->output_buf_size)
+	    vb2_plane_size(vb, 0) < out_buf_size)
 		return -EINVAL;
 	if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
 	    vb2_plane_size(vb, 0) < inst->input_buf_size)
@@ -894,6 +1025,8 @@ void venus_helper_vb2_stop_streaming(struct vb2_queue *q)
 		if (ret)
 			hfi_session_abort(inst);
 
+		venus_helper_free_dpb_bufs(inst);
+
 		load_scale_clocks(core);
 		INIT_LIST_HEAD(&inst->registeredbufs);
 	}
@@ -932,8 +1065,14 @@ int venus_helper_vb2_start_streaming(struct venus_inst *inst)
 	if (ret)
 		goto err_unload_res;
 
+	ret = venus_helper_queue_dpb_bufs(inst);
+	if (ret)
+		goto err_session_stop;
+
 	return 0;
 
+err_session_stop:
+	hfi_session_stop(inst);
 err_unload_res:
 	hfi_session_unload_res(inst);
 err_unreg_bufs:
@@ -987,6 +1126,67 @@ void venus_helper_init_instance(struct venus_inst *inst)
 }
 EXPORT_SYMBOL_GPL(venus_helper_init_instance);
 
+static bool find_fmt_from_caps(struct venus_caps *caps, u32 buftype, u32 fmt)
+{
+	unsigned int i;
+
+	for (i = 0; i < caps->num_fmts; i++) {
+		if (caps->fmts[i].buftype == buftype &&
+		    caps->fmts[i].fmt == fmt)
+			return true;
+	}
+
+	return false;
+}
+
+int venus_helper_get_out_fmts(struct venus_inst *inst, u32 v4l2_fmt,
+			      u32 *out_fmt, u32 *out2_fmt, bool ubwc)
+{
+	struct venus_core *core = inst->core;
+	struct venus_caps *caps;
+	u32 ubwc_fmt, fmt = to_hfi_raw_fmt(v4l2_fmt);
+	bool found, found_ubwc;
+
+	*out_fmt = *out2_fmt = 0;
+
+	if (!fmt)
+		return -EINVAL;
+
+	caps = venus_caps_by_codec(core, inst->hfi_codec, inst->session_type);
+	if (!caps)
+		return -EINVAL;
+
+	if (ubwc) {
+		ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE;
+		found_ubwc = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT,
+						ubwc_fmt);
+		found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
+
+		if (found_ubwc && found) {
+			*out_fmt = ubwc_fmt;
+			*out2_fmt = fmt;
+			return 0;
+		}
+	}
+
+	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT, fmt);
+	if (found) {
+		*out_fmt = fmt;
+		*out2_fmt = 0;
+		return 0;
+	}
+
+	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
+	if (found) {
+		*out_fmt = 0;
+		*out2_fmt = fmt;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+EXPORT_SYMBOL_GPL(venus_helper_get_out_fmts);
+
 int venus_helper_power_enable(struct venus_core *core, u32 session_type,
 			      bool enable)
 {
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 92b167a47166..2475f284f396 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -50,10 +50,16 @@ int venus_helper_set_raw_format(struct venus_inst *inst, u32 hfi_format,
 int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
 int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
 int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize, u32 buftype);
+int venus_helper_set_multistream(struct venus_inst *inst, bool out_en,
+				 bool out2_en);
 unsigned int venus_helper_get_opb_size(struct venus_inst *inst);
 void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
 void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned int idx);
 void venus_helper_init_instance(struct venus_inst *inst);
+int venus_helper_get_out_fmts(struct venus_inst *inst, u32 fmt, u32 *out_fmt,
+			      u32 *out2_fmt, bool ubwc);
+int venus_helper_alloc_dpb_bufs(struct venus_inst *inst);
+int venus_helper_free_dpb_bufs(struct venus_inst *inst);
 int venus_helper_power_enable(struct venus_core *core, u32 session_type,
 			      bool enable);
 #endif
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 589fc13b84bc..7deee104ac56 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -532,10 +532,16 @@ static int vdec_set_properties(struct venus_inst *inst)
 	return 0;
 }
 
+#define is_ubwc_fmt(fmt) (!!((fmt) & HFI_COLOR_FORMAT_UBWC_BASE))
+
 static int vdec_output_conf(struct venus_inst *inst)
 {
 	struct venus_core *core = inst->core;
 	struct hfi_enable en = { .enable = 1 };
+	u32 width = inst->out_width;
+	u32 height = inst->out_height;
+	u32 out_fmt, out2_fmt;
+	bool ubwc = false;
 	u32 ptype;
 	int ret;
 
@@ -554,6 +560,78 @@ static int vdec_output_conf(struct venus_inst *inst)
 			return ret;
 	}
 
+	if (width > 1920 && height > ALIGN(1080, 32))
+		ubwc = true;
+
+	if (IS_V4(core))
+		ubwc = true;
+
+	ret = venus_helper_get_out_fmts(inst, inst->fmt_cap->pixfmt, &out_fmt,
+					&out2_fmt, ubwc);
+	if (ret)
+		return ret;
+
+	inst->output_buf_size =
+			venus_helper_get_framesz_raw(out_fmt, width, height);
+	inst->output2_buf_size =
+			venus_helper_get_framesz_raw(out2_fmt, width, height);
+
+	if (is_ubwc_fmt(out_fmt)) {
+		inst->opb_buftype = HFI_BUFFER_OUTPUT2;
+		inst->opb_fmt = out2_fmt;
+		inst->dpb_buftype = HFI_BUFFER_OUTPUT;
+		inst->dpb_fmt = out_fmt;
+	} else if (is_ubwc_fmt(out2_fmt)) {
+		inst->opb_buftype = HFI_BUFFER_OUTPUT;
+		inst->opb_fmt = out_fmt;
+		inst->dpb_buftype = HFI_BUFFER_OUTPUT2;
+		inst->dpb_fmt = out2_fmt;
+	} else {
+		inst->opb_buftype = HFI_BUFFER_OUTPUT;
+		inst->opb_fmt = out_fmt;
+		inst->dpb_buftype = 0;
+		inst->dpb_fmt = 0;
+	}
+
+	ret = venus_helper_set_raw_format(inst, inst->opb_fmt,
+					  inst->opb_buftype);
+	if (ret)
+		return ret;
+
+	if (inst->dpb_fmt) {
+		ret = venus_helper_set_multistream(inst, false, true);
+		if (ret)
+			return ret;
+
+		ret = venus_helper_set_raw_format(inst, inst->dpb_fmt,
+						  inst->dpb_buftype);
+		if (ret)
+			return ret;
+
+		ret = venus_helper_set_output_resolution(inst, width, height,
+							 HFI_BUFFER_OUTPUT2);
+		if (ret)
+			return ret;
+	}
+
+	if (IS_V3(core) || IS_V4(core)) {
+		if (inst->output2_buf_size) {
+			ret = venus_helper_set_bufsize(inst,
+						       inst->output2_buf_size,
+						       HFI_BUFFER_OUTPUT2);
+			if (ret)
+				return ret;
+		}
+
+		if (inst->output_buf_size) {
+			ret = venus_helper_set_bufsize(inst,
+						       inst->output_buf_size,
+						       HFI_BUFFER_OUTPUT);
+			if (ret)
+				return ret;
+		}
+	}
+
 	ret = venus_helper_set_dyn_bufmode(inst);
 	if (ret)
 		return ret;
@@ -624,6 +702,8 @@ static int vdec_queue_setup(struct vb2_queue *q,
 	int ret = 0;
 
 	if (*num_planes) {
+		unsigned int output_buf_size = venus_helper_get_opb_size(inst);
+
 		if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
 		    *num_planes != inst->fmt_out->num_planes)
 			return -EINVAL;
@@ -637,7 +717,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
 			return -EINVAL;
 
 		if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
-		    sizes[0] < inst->output_buf_size)
+		    sizes[0] < output_buf_size)
 			return -EINVAL;
 
 		return 0;
@@ -746,6 +826,10 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 	if (ret)
 		goto deinit_sess;
 
+	ret = venus_helper_alloc_dpb_bufs(inst);
+	if (ret)
+		goto deinit_sess;
+
 	ret = venus_helper_vb2_start_streaming(inst);
 	if (ret)
 		goto deinit_sess;
@@ -797,9 +881,11 @@ static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type,
 	vbuf->field = V4L2_FIELD_NONE;
 
 	if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+		unsigned int opb_sz = venus_helper_get_opb_size(inst);
+
 		vb = &vbuf->vb2_buf;
 		vb->planes[0].bytesused =
-			max_t(unsigned int, inst->output_buf_size, bytesused);
+			max_t(unsigned int, opb_sz, bytesused);
 		vb->planes[0].data_offset = data_offset;
 		vb->timestamp = timestamp_us * NSEC_PER_USEC;
 		vbuf->sequence = inst->sequence_cap++;
@@ -945,6 +1031,7 @@ static int vdec_open(struct file *file)
 	if (!inst)
 		return -ENOMEM;
 
+	INIT_LIST_HEAD(&inst->dpbbufs);
 	INIT_LIST_HEAD(&inst->registeredbufs);
 	INIT_LIST_HEAD(&inst->internalbufs);
 	INIT_LIST_HEAD(&inst->list);
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 54f253b98b24..a703bce78abc 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -1084,6 +1084,7 @@ static int venc_open(struct file *file)
 	if (!inst)
 		return -ENOMEM;
 
+	INIT_LIST_HEAD(&inst->dpbbufs);
 	INIT_LIST_HEAD(&inst->registeredbufs);
 	INIT_LIST_HEAD(&inst->internalbufs);
 	INIT_LIST_HEAD(&inst->list);
-- 
2.14.1

Re: [PATCH 26/28] venus: implementing multi-stream support

[Vikash Garodia]

Hello Stanimir,

On 2018-04-24 18:14, Stanimir Varbanov wrote:
> This is implementing a multi-stream decoder support. The multi
> stream gives an option to use the secondary decoder output
> with different raw format (or the same in case of crop).
> 
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/core.h    |   1 +
>  drivers/media/platform/qcom/venus/helpers.c | 204 
> +++++++++++++++++++++++++++-
>  drivers/media/platform/qcom/venus/helpers.h |   6 +
>  drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
>  drivers/media/platform/qcom/venus/venc.c    |   1 +
>  5 files changed, 299 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/qcom/venus/core.h
> b/drivers/media/platform/qcom/venus/core.h
> index 4d6c05f156c4..85e66e2dd672 100644
> --- a/drivers/media/platform/qcom/venus/core.h
> +++ b/drivers/media/platform/qcom/venus/core.h
> @@ -259,6 +259,7 @@ struct venus_inst {
>  	struct list_head list;
>  	struct mutex lock;
>  	struct venus_core *core;
> +	struct list_head dpbbufs;
>  	struct list_head internalbufs;
>  	struct list_head registeredbufs;
>  	struct list_head delayed_process;
> diff --git a/drivers/media/platform/qcom/venus/helpers.c
> b/drivers/media/platform/qcom/venus/helpers.c
> index ed569705ecac..87dcf9973e6f 100644
> --- a/drivers/media/platform/qcom/venus/helpers.c
> +++ b/drivers/media/platform/qcom/venus/helpers.c
> @@ -85,6 +85,112 @@ bool venus_helper_check_codec(struct venus_inst
> *inst, u32 v4l2_pixfmt)
>  }
>  EXPORT_SYMBOL_GPL(venus_helper_check_codec);
> 
> +static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
> +{
> +	struct intbuf *buf;
> +	int ret = 0;
> +
> +	if (list_empty(&inst->dpbbufs))
> +		return 0;
> +
> +	list_for_each_entry(buf, &inst->dpbbufs, list) {
> +		struct hfi_frame_data fdata;
> +
> +		memset(&fdata, 0, sizeof(fdata));
> +		fdata.alloc_len = buf->size;
> +		fdata.device_addr = buf->da;
> +		fdata.buffer_type = buf->type;
> +
> +		ret = hfi_session_process_buf(inst, &fdata);
> +		if (ret)
> +			goto fail;
> +	}
> +
> +fail:
> +	return ret;
> +}
> +
> +int venus_helper_free_dpb_bufs(struct venus_inst *inst)
> +{
> +	struct intbuf *buf, *n;
> +
> +	if (list_empty(&inst->dpbbufs))
> +		return 0;
> +
> +	list_for_each_entry_safe(buf, n, &inst->dpbbufs, list) {
> +		list_del_init(&buf->list);
> +		dma_free_attrs(inst->core->dev, buf->size, buf->va, buf->da,
> +			       buf->attrs);
> +		kfree(buf);
> +	}
> +
> +	INIT_LIST_HEAD(&inst->dpbbufs);
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(venus_helper_free_dpb_bufs);
> +
> +int venus_helper_alloc_dpb_bufs(struct venus_inst *inst)
> +{
> +	struct venus_core *core = inst->core;
> +	struct device *dev = core->dev;
> +	enum hfi_version ver = core->res->hfi_version;
> +	struct hfi_buffer_requirements bufreq;
> +	u32 buftype = inst->dpb_buftype;
> +	unsigned int dpb_size = 0;
> +	struct intbuf *buf;
> +	unsigned int i;
> +	u32 count;
> +	int ret;
> +
> +	/* no need to allocate dpb buffers */
> +	if (!inst->dpb_fmt)
> +		return 0;
> +
> +	if (inst->dpb_buftype == HFI_BUFFER_OUTPUT)
> +		dpb_size = inst->output_buf_size;
> +	else if (inst->dpb_buftype == HFI_BUFFER_OUTPUT2)
> +		dpb_size = inst->output2_buf_size;
> +
> +	if (!dpb_size)
> +		return 0;
> +
> +	ret = venus_helper_get_bufreq(inst, buftype, &bufreq);
> +	if (ret)
> +		return ret;
> +
> +	count = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
> +
> +	for (i = 0; i < count; i++) {
> +		buf = kzalloc(sizeof(*buf), GFP_KERNEL);
> +		if (!buf) {
> +			ret = -ENOMEM;
> +			goto fail;
> +		}
> +
> +		buf->type = buftype;
> +		buf->size = dpb_size;
> +		buf->attrs = DMA_ATTR_WRITE_COMBINE |
> +			     DMA_ATTR_NO_KERNEL_MAPPING;
> +		buf->va = dma_alloc_attrs(dev, buf->size, &buf->da, GFP_KERNEL,
> +					  buf->attrs);
> +		if (!buf->va) {
> +			kfree(buf);
> +			ret = -ENOMEM;
> +			goto fail;
> +		}
> +
> +		list_add_tail(&buf->list, &inst->dpbbufs);
> +	}
> +
> +	return 0;
> +
> +fail:
> +	venus_helper_free_dpb_bufs(inst);
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(venus_helper_alloc_dpb_bufs);
> +
>  static int intbufs_set_buffer(struct venus_inst *inst, u32 type)
>  {
>  	struct venus_core *core = inst->core;
> @@ -342,7 +448,10 @@ session_process_buf(struct venus_inst *inst,
> struct vb2_v4l2_buffer *vbuf)
>  		if (vbuf->flags & V4L2_BUF_FLAG_LAST || !fdata.filled_len)
>  			fdata.flags |= HFI_BUFFERFLAG_EOS;
>  	} else if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> -		fdata.buffer_type = HFI_BUFFER_OUTPUT;
> +		if (inst->session_type == VIDC_SESSION_TYPE_ENC)
> +			fdata.buffer_type = HFI_BUFFER_OUTPUT;
> +		else
> +			fdata.buffer_type = inst->opb_buftype;
>  		fdata.filled_len = 0;
>  		fdata.offset = 0;
>  	}
> @@ -675,6 +784,27 @@ int venus_helper_set_color_format(struct
> venus_inst *inst, u32 pixfmt)
>  }
>  EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
> 
> +int venus_helper_set_multistream(struct venus_inst *inst, bool out_en,
> +				 bool out2_en)
> +{
> +	struct hfi_multi_stream multi = {0};
> +	u32 ptype = HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM;
> +	int ret;
> +
> +	multi.buffer_type = HFI_BUFFER_OUTPUT;
> +	multi.enable = out_en;
> +
> +	ret = hfi_session_set_property(inst, ptype, &multi);
> +	if (ret)
> +		return ret;
> +
> +	multi.buffer_type = HFI_BUFFER_OUTPUT2;
> +	multi.enable = out2_en;
> +
> +	return hfi_session_set_property(inst, ptype, &multi);
> +}
> +EXPORT_SYMBOL_GPL(venus_helper_set_multistream);
> +
>  int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
>  {
>  	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
> @@ -822,9 +952,10 @@ EXPORT_SYMBOL_GPL(venus_helper_vb2_buf_init);
>  int venus_helper_vb2_buf_prepare(struct vb2_buffer *vb)
>  {
>  	struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
> +	unsigned int out_buf_size = venus_helper_get_opb_size(inst);
> 
>  	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> -	    vb2_plane_size(vb, 0) < inst->output_buf_size)
> +	    vb2_plane_size(vb, 0) < out_buf_size)
>  		return -EINVAL;
>  	if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
>  	    vb2_plane_size(vb, 0) < inst->input_buf_size)
> @@ -894,6 +1025,8 @@ void venus_helper_vb2_stop_streaming(struct 
> vb2_queue *q)
>  		if (ret)
>  			hfi_session_abort(inst);
> 
> +		venus_helper_free_dpb_bufs(inst);
> +
>  		load_scale_clocks(core);
>  		INIT_LIST_HEAD(&inst->registeredbufs);
>  	}
> @@ -932,8 +1065,14 @@ int venus_helper_vb2_start_streaming(struct
> venus_inst *inst)
>  	if (ret)
>  		goto err_unload_res;
> 
> +	ret = venus_helper_queue_dpb_bufs(inst);
> +	if (ret)
> +		goto err_session_stop;
> +
>  	return 0;
> 
> +err_session_stop:
> +	hfi_session_stop(inst);
>  err_unload_res:
>  	hfi_session_unload_res(inst);
>  err_unreg_bufs:
> @@ -987,6 +1126,67 @@ void venus_helper_init_instance(struct venus_inst 
> *inst)
>  }
>  EXPORT_SYMBOL_GPL(venus_helper_init_instance);
> 
> +static bool find_fmt_from_caps(struct venus_caps *caps, u32 buftype, 
> u32 fmt)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < caps->num_fmts; i++) {
> +		if (caps->fmts[i].buftype == buftype &&
> +		    caps->fmts[i].fmt == fmt)
> +			return true;
> +	}
> +
> +	return false;
> +}
> +
> +int venus_helper_get_out_fmts(struct venus_inst *inst, u32 v4l2_fmt,
> +			      u32 *out_fmt, u32 *out2_fmt, bool ubwc)
> +{
> +	struct venus_core *core = inst->core;
> +	struct venus_caps *caps;
> +	u32 ubwc_fmt, fmt = to_hfi_raw_fmt(v4l2_fmt);
> +	bool found, found_ubwc;
> +
> +	*out_fmt = *out2_fmt = 0;
> +
> +	if (!fmt)
> +		return -EINVAL;
> +
> +	caps = venus_caps_by_codec(core, inst->hfi_codec, 
> inst->session_type);
> +	if (!caps)
> +		return -EINVAL;
> +
> +	if (ubwc) {
> +		ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE;
> +		found_ubwc = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT,
> +						ubwc_fmt);
> +		found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
> +
> +		if (found_ubwc && found) {
> +			*out_fmt = ubwc_fmt;
> +			*out2_fmt = fmt;
> +			return 0;
> +		}
> +	}
> +
> +	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT, fmt);
> +	if (found) {
> +		*out_fmt = fmt;
> +		*out2_fmt = 0;
> +		return 0;
> +	}
> +
> +	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
> +	if (found) {
> +		*out_fmt = 0;
> +		*out2_fmt = fmt;
> +		return 0;
> +	}
> +
> +	return -EINVAL;
> +}
> +EXPORT_SYMBOL_GPL(venus_helper_get_out_fmts);
> +
>  int venus_helper_power_enable(struct venus_core *core, u32 
> session_type,
>  			      bool enable)
>  {
> diff --git a/drivers/media/platform/qcom/venus/helpers.h
> b/drivers/media/platform/qcom/venus/helpers.h
> index 92b167a47166..2475f284f396 100644
> --- a/drivers/media/platform/qcom/venus/helpers.h
> +++ b/drivers/media/platform/qcom/venus/helpers.h
> @@ -50,10 +50,16 @@ int venus_helper_set_raw_format(struct venus_inst
> *inst, u32 hfi_format,
>  int venus_helper_set_color_format(struct venus_inst *inst, u32 fmt);
>  int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
>  int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize,
> u32 buftype);
> +int venus_helper_set_multistream(struct venus_inst *inst, bool out_en,
> +				 bool out2_en);
>  unsigned int venus_helper_get_opb_size(struct venus_inst *inst);
>  void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
>  void venus_helper_release_buf_ref(struct venus_inst *inst, unsigned 
> int idx);
>  void venus_helper_init_instance(struct venus_inst *inst);
> +int venus_helper_get_out_fmts(struct venus_inst *inst, u32 fmt, u32 
> *out_fmt,
> +			      u32 *out2_fmt, bool ubwc);
> +int venus_helper_alloc_dpb_bufs(struct venus_inst *inst);
> +int venus_helper_free_dpb_bufs(struct venus_inst *inst);
>  int venus_helper_power_enable(struct venus_core *core, u32 
> session_type,
>  			      bool enable);
>  #endif
> diff --git a/drivers/media/platform/qcom/venus/vdec.c
> b/drivers/media/platform/qcom/venus/vdec.c
> index 589fc13b84bc..7deee104ac56 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -532,10 +532,16 @@ static int vdec_set_properties(struct venus_inst 
> *inst)
>  	return 0;
>  }
> 
> +#define is_ubwc_fmt(fmt) (!!((fmt) & HFI_COLOR_FORMAT_UBWC_BASE))
> +
>  static int vdec_output_conf(struct venus_inst *inst)
>  {
>  	struct venus_core *core = inst->core;
>  	struct hfi_enable en = { .enable = 1 };
> +	u32 width = inst->out_width;
> +	u32 height = inst->out_height;
> +	u32 out_fmt, out2_fmt;
> +	bool ubwc = false;
>  	u32 ptype;
>  	int ret;
> 
> @@ -554,6 +560,78 @@ static int vdec_output_conf(struct venus_inst 
> *inst)
>  			return ret;
>  	}
> 
> +	if (width > 1920 && height > ALIGN(1080, 32))
> +		ubwc = true;
> +
> +	if (IS_V4(core))
> +		ubwc = true;
> +
> +	ret = venus_helper_get_out_fmts(inst, inst->fmt_cap->pixfmt, 
> &out_fmt,
> +					&out2_fmt, ubwc);
> +	if (ret)
> +		return ret;
> +
> +	inst->output_buf_size =
> +			venus_helper_get_framesz_raw(out_fmt, width, height);
> +	inst->output2_buf_size =
> +			venus_helper_get_framesz_raw(out2_fmt, width, height);
> +
> +	if (is_ubwc_fmt(out_fmt)) {
> +		inst->opb_buftype = HFI_BUFFER_OUTPUT2;
> +		inst->opb_fmt = out2_fmt;
> +		inst->dpb_buftype = HFI_BUFFER_OUTPUT;
> +		inst->dpb_fmt = out_fmt;
> +	} else if (is_ubwc_fmt(out2_fmt)) {
> +		inst->opb_buftype = HFI_BUFFER_OUTPUT;
> +		inst->opb_fmt = out_fmt;
> +		inst->dpb_buftype = HFI_BUFFER_OUTPUT2;
> +		inst->dpb_fmt = out2_fmt;
> +	} else {
> +		inst->opb_buftype = HFI_BUFFER_OUTPUT;
> +		inst->opb_fmt = out_fmt;
> +		inst->dpb_buftype = 0;
> +		inst->dpb_fmt = 0;
> +	}
> +
> +	ret = venus_helper_set_raw_format(inst, inst->opb_fmt,
> +					  inst->opb_buftype);
> +	if (ret)
> +		return ret;
> +
> +	if (inst->dpb_fmt) {
> +		ret = venus_helper_set_multistream(inst, false, true);
> +		if (ret)
> +			return ret;
> +
> +		ret = venus_helper_set_raw_format(inst, inst->dpb_fmt,
> +						  inst->dpb_buftype);
> +		if (ret)
> +			return ret;
> +
> +		ret = venus_helper_set_output_resolution(inst, width, height,
> +							 HFI_BUFFER_OUTPUT2);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	if (IS_V3(core) || IS_V4(core)) {
> +		if (inst->output2_buf_size) {
> +			ret = venus_helper_set_bufsize(inst,
> +						       inst->output2_buf_size,
> +						       HFI_BUFFER_OUTPUT2);
> +			if (ret)
> +				return ret;
> +		}
> +
> +		if (inst->output_buf_size) {
> +			ret = venus_helper_set_bufsize(inst,
> +						       inst->output_buf_size,
> +						       HFI_BUFFER_OUTPUT);
> +			if (ret)
> +				return ret;
> +		}
> +	}
> +
>  	ret = venus_helper_set_dyn_bufmode(inst);
>  	if (ret)
>  		return ret;
> @@ -624,6 +702,8 @@ static int vdec_queue_setup(struct vb2_queue *q,
>  	int ret = 0;
> 
>  	if (*num_planes) {
> +		unsigned int output_buf_size = venus_helper_get_opb_size(inst);
> +
>  		if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
>  		    *num_planes != inst->fmt_out->num_planes)
>  			return -EINVAL;
> @@ -637,7 +717,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
>  			return -EINVAL;
> 
>  		if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> -		    sizes[0] < inst->output_buf_size)
> +		    sizes[0] < output_buf_size)
>  			return -EINVAL;
> 
>  		return 0;
> @@ -746,6 +826,10 @@ static int vdec_start_streaming(struct vb2_queue
> *q, unsigned int count)
>  	if (ret)
>  		goto deinit_sess;
> 
> +	ret = venus_helper_alloc_dpb_bufs(inst);
> +	if (ret)
> +		goto deinit_sess;
> +
>  	ret = venus_helper_vb2_start_streaming(inst);
>  	if (ret)
>  		goto deinit_sess;
> @@ -797,9 +881,11 @@ static void vdec_buf_done(struct venus_inst
> *inst, unsigned int buf_type,
>  	vbuf->field = V4L2_FIELD_NONE;
> 
>  	if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +		unsigned int opb_sz = venus_helper_get_opb_size(inst);
> +
>  		vb = &vbuf->vb2_buf;
>  		vb->planes[0].bytesused =
> -			max_t(unsigned int, inst->output_buf_size, bytesused);
> +			max_t(unsigned int, opb_sz, bytesused);
>  		vb->planes[0].data_offset = data_offset;
>  		vb->timestamp = timestamp_us * NSEC_PER_USEC;
>  		vbuf->sequence = inst->sequence_cap++;
> @@ -945,6 +1031,7 @@ static int vdec_open(struct file *file)
>  	if (!inst)
>  		return -ENOMEM;
> 
> +	INIT_LIST_HEAD(&inst->dpbbufs);
>  	INIT_LIST_HEAD(&inst->registeredbufs);
>  	INIT_LIST_HEAD(&inst->internalbufs);
>  	INIT_LIST_HEAD(&inst->list);
> diff --git a/drivers/media/platform/qcom/venus/venc.c
> b/drivers/media/platform/qcom/venus/venc.c
> index 54f253b98b24..a703bce78abc 100644
> --- a/drivers/media/platform/qcom/venus/venc.c
> +++ b/drivers/media/platform/qcom/venus/venc.c
> @@ -1084,6 +1084,7 @@ static int venc_open(struct file *file)
>  	if (!inst)
>  		return -ENOMEM;
> 
> +	INIT_LIST_HEAD(&inst->dpbbufs);
>  	INIT_LIST_HEAD(&inst->registeredbufs);
>  	INIT_LIST_HEAD(&inst->internalbufs);
>  	INIT_LIST_HEAD(&inst->list);

The dpb buffers queued to hardware will be returned back to host either 
during flush
or when the session is stopped. Host should not send these buffers to 
client.
vdec_buf_done should be handling in a way to drop dpb buffers from 
sending to client.

Thanks,
Vikash

Re: [PATCH 26/28] venus: implementing multi-stream support

[Nicolas Dufresne]

Le mercredi 02 mai 2018 à 13:10 +0530, Vikash Garodia a écrit :
> Hello Stanimir,
> 
> On 2018-04-24 18:14, Stanimir Varbanov wrote:
> > This is implementing a multi-stream decoder support. The multi
> > stream gives an option to use the secondary decoder output
> > with different raw format (or the same in case of crop).
> > 
> > Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> > ---
> >  drivers/media/platform/qcom/venus/core.h    |   1 +
> >  drivers/media/platform/qcom/venus/helpers.c | 204 
> > +++++++++++++++++++++++++++-
> >  drivers/media/platform/qcom/venus/helpers.h |   6 +
> >  drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
> >  drivers/media/platform/qcom/venus/venc.c    |   1 +
> >  5 files changed, 299 insertions(+), 4 deletions(-)
> > 
> > diff --git a/drivers/media/platform/qcom/venus/core.h
> > b/drivers/media/platform/qcom/venus/core.h
> > index 4d6c05f156c4..85e66e2dd672 100644
> > --- a/drivers/media/platform/qcom/venus/core.h
> > +++ b/drivers/media/platform/qcom/venus/core.h
> > @@ -259,6 +259,7 @@ struct venus_inst {
> >  	struct list_head list;
> >  	struct mutex lock;
> >  	struct venus_core *core;
> > +	struct list_head dpbbufs;
> >  	struct list_head internalbufs;
> >  	struct list_head registeredbufs;
> >  	struct list_head delayed_process;
> > diff --git a/drivers/media/platform/qcom/venus/helpers.c
> > b/drivers/media/platform/qcom/venus/helpers.c
> > index ed569705ecac..87dcf9973e6f 100644
> > --- a/drivers/media/platform/qcom/venus/helpers.c
> > +++ b/drivers/media/platform/qcom/venus/helpers.c
> > @@ -85,6 +85,112 @@ bool venus_helper_check_codec(struct venus_inst
> > *inst, u32 v4l2_pixfmt)
> >  }
> >  EXPORT_SYMBOL_GPL(venus_helper_check_codec);
> > 
> > +static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
> > +{
> > +	struct intbuf *buf;
> > +	int ret = 0;
> > +
> > +	if (list_empty(&inst->dpbbufs))
> > +		return 0;
> > +
> > +	list_for_each_entry(buf, &inst->dpbbufs, list) {
> > +		struct hfi_frame_data fdata;
> > +
> > +		memset(&fdata, 0, sizeof(fdata));
> > +		fdata.alloc_len = buf->size;
> > +		fdata.device_addr = buf->da;
> > +		fdata.buffer_type = buf->type;
> > +
> > +		ret = hfi_session_process_buf(inst, &fdata);
> > +		if (ret)
> > +			goto fail;
> > +	}
> > +
> > +fail:
> > +	return ret;
> > +}
> > +
> > +int venus_helper_free_dpb_bufs(struct venus_inst *inst)
> > +{
> > +	struct intbuf *buf, *n;
> > +
> > +	if (list_empty(&inst->dpbbufs))
> > +		return 0;
> > +
> > +	list_for_each_entry_safe(buf, n, &inst->dpbbufs, list) {
> > +		list_del_init(&buf->list);
> > +		dma_free_attrs(inst->core->dev, buf->size, buf-
> > >va, buf->da,
> > +			       buf->attrs);
> > +		kfree(buf);
> > +	}
> > +
> > +	INIT_LIST_HEAD(&inst->dpbbufs);
> > +
> > +	return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(venus_helper_free_dpb_bufs);
> > +
> > +int venus_helper_alloc_dpb_bufs(struct venus_inst *inst)
> > +{
> > +	struct venus_core *core = inst->core;
> > +	struct device *dev = core->dev;
> > +	enum hfi_version ver = core->res->hfi_version;
> > +	struct hfi_buffer_requirements bufreq;
> > +	u32 buftype = inst->dpb_buftype;
> > +	unsigned int dpb_size = 0;
> > +	struct intbuf *buf;
> > +	unsigned int i;
> > +	u32 count;
> > +	int ret;
> > +
> > +	/* no need to allocate dpb buffers */
> > +	if (!inst->dpb_fmt)
> > +		return 0;
> > +
> > +	if (inst->dpb_buftype == HFI_BUFFER_OUTPUT)
> > +		dpb_size = inst->output_buf_size;
> > +	else if (inst->dpb_buftype == HFI_BUFFER_OUTPUT2)
> > +		dpb_size = inst->output2_buf_size;
> > +
> > +	if (!dpb_size)
> > +		return 0;
> > +
> > +	ret = venus_helper_get_bufreq(inst, buftype, &bufreq);
> > +	if (ret)
> > +		return ret;
> > +
> > +	count = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
> > +
> > +	for (i = 0; i < count; i++) {
> > +		buf = kzalloc(sizeof(*buf), GFP_KERNEL);
> > +		if (!buf) {
> > +			ret = -ENOMEM;
> > +			goto fail;
> > +		}
> > +
> > +		buf->type = buftype;
> > +		buf->size = dpb_size;
> > +		buf->attrs = DMA_ATTR_WRITE_COMBINE |
> > +			     DMA_ATTR_NO_KERNEL_MAPPING;
> > +		buf->va = dma_alloc_attrs(dev, buf->size, &buf-
> > >da, GFP_KERNEL,
> > +					  buf->attrs);
> > +		if (!buf->va) {
> > +			kfree(buf);
> > +			ret = -ENOMEM;
> > +			goto fail;
> > +		}
> > +
> > +		list_add_tail(&buf->list, &inst->dpbbufs);
> > +	}
> > +
> > +	return 0;
> > +
> > +fail:
> > +	venus_helper_free_dpb_bufs(inst);
> > +	return ret;
> > +}
> > +EXPORT_SYMBOL_GPL(venus_helper_alloc_dpb_bufs);
> > +
> >  static int intbufs_set_buffer(struct venus_inst *inst, u32 type)
> >  {
> >  	struct venus_core *core = inst->core;
> > @@ -342,7 +448,10 @@ session_process_buf(struct venus_inst *inst,
> > struct vb2_v4l2_buffer *vbuf)
> >  		if (vbuf->flags & V4L2_BUF_FLAG_LAST ||
> > !fdata.filled_len)
> >  			fdata.flags |= HFI_BUFFERFLAG_EOS;
> >  	} else if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > -		fdata.buffer_type = HFI_BUFFER_OUTPUT;
> > +		if (inst->session_type == VIDC_SESSION_TYPE_ENC)
> > +			fdata.buffer_type = HFI_BUFFER_OUTPUT;
> > +		else
> > +			fdata.buffer_type = inst->opb_buftype;
> >  		fdata.filled_len = 0;
> >  		fdata.offset = 0;
> >  	}
> > @@ -675,6 +784,27 @@ int venus_helper_set_color_format(struct
> > venus_inst *inst, u32 pixfmt)
> >  }
> >  EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
> > 
> > +int venus_helper_set_multistream(struct venus_inst *inst, bool
> > out_en,
> > +				 bool out2_en)
> > +{
> > +	struct hfi_multi_stream multi = {0};
> > +	u32 ptype = HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM;
> > +	int ret;
> > +
> > +	multi.buffer_type = HFI_BUFFER_OUTPUT;
> > +	multi.enable = out_en;
> > +
> > +	ret = hfi_session_set_property(inst, ptype, &multi);
> > +	if (ret)
> > +		return ret;
> > +
> > +	multi.buffer_type = HFI_BUFFER_OUTPUT2;
> > +	multi.enable = out2_en;
> > +
> > +	return hfi_session_set_property(inst, ptype, &multi);
> > +}
> > +EXPORT_SYMBOL_GPL(venus_helper_set_multistream);
> > +
> >  int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
> >  {
> >  	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
> > @@ -822,9 +952,10 @@ EXPORT_SYMBOL_GPL(venus_helper_vb2_buf_init);
> >  int venus_helper_vb2_buf_prepare(struct vb2_buffer *vb)
> >  {
> >  	struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
> > +	unsigned int out_buf_size =
> > venus_helper_get_opb_size(inst);
> > 
> >  	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> > -	    vb2_plane_size(vb, 0) < inst->output_buf_size)
> > +	    vb2_plane_size(vb, 0) < out_buf_size)
> >  		return -EINVAL;
> >  	if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
> >  	    vb2_plane_size(vb, 0) < inst->input_buf_size)
> > @@ -894,6 +1025,8 @@ void venus_helper_vb2_stop_streaming(struct 
> > vb2_queue *q)
> >  		if (ret)
> >  			hfi_session_abort(inst);
> > 
> > +		venus_helper_free_dpb_bufs(inst);
> > +
> >  		load_scale_clocks(core);
> >  		INIT_LIST_HEAD(&inst->registeredbufs);
> >  	}
> > @@ -932,8 +1065,14 @@ int venus_helper_vb2_start_streaming(struct
> > venus_inst *inst)
> >  	if (ret)
> >  		goto err_unload_res;
> > 
> > +	ret = venus_helper_queue_dpb_bufs(inst);
> > +	if (ret)
> > +		goto err_session_stop;
> > +
> >  	return 0;
> > 
> > +err_session_stop:
> > +	hfi_session_stop(inst);
> >  err_unload_res:
> >  	hfi_session_unload_res(inst);
> >  err_unreg_bufs:
> > @@ -987,6 +1126,67 @@ void venus_helper_init_instance(struct
> > venus_inst 
> > *inst)
> >  }
> >  EXPORT_SYMBOL_GPL(venus_helper_init_instance);
> > 
> > +static bool find_fmt_from_caps(struct venus_caps *caps, u32
> > buftype, 
> > u32 fmt)
> > +{
> > +	unsigned int i;
> > +
> > +	for (i = 0; i < caps->num_fmts; i++) {
> > +		if (caps->fmts[i].buftype == buftype &&
> > +		    caps->fmts[i].fmt == fmt)
> > +			return true;
> > +	}
> > +
> > +	return false;
> > +}
> > +
> > +int venus_helper_get_out_fmts(struct venus_inst *inst, u32
> > v4l2_fmt,
> > +			      u32 *out_fmt, u32 *out2_fmt, bool
> > ubwc)
> > +{
> > +	struct venus_core *core = inst->core;
> > +	struct venus_caps *caps;
> > +	u32 ubwc_fmt, fmt = to_hfi_raw_fmt(v4l2_fmt);
> > +	bool found, found_ubwc;
> > +
> > +	*out_fmt = *out2_fmt = 0;
> > +
> > +	if (!fmt)
> > +		return -EINVAL;
> > +
> > +	caps = venus_caps_by_codec(core, inst->hfi_codec, 
> > inst->session_type);
> > +	if (!caps)
> > +		return -EINVAL;
> > +
> > +	if (ubwc) {
> > +		ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE;
> > +		found_ubwc = find_fmt_from_caps(caps,
> > HFI_BUFFER_OUTPUT,
> > +						ubwc_fmt);
> > +		found = find_fmt_from_caps(caps,
> > HFI_BUFFER_OUTPUT2, fmt);
> > +
> > +		if (found_ubwc && found) {
> > +			*out_fmt = ubwc_fmt;
> > +			*out2_fmt = fmt;
> > +			return 0;
> > +		}
> > +	}
> > +
> > +	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT, fmt);
> > +	if (found) {
> > +		*out_fmt = fmt;
> > +		*out2_fmt = 0;
> > +		return 0;
> > +	}
> > +
> > +	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
> > +	if (found) {
> > +		*out_fmt = 0;
> > +		*out2_fmt = fmt;
> > +		return 0;
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +EXPORT_SYMBOL_GPL(venus_helper_get_out_fmts);
> > +
> >  int venus_helper_power_enable(struct venus_core *core, u32 
> > session_type,
> >  			      bool enable)
> >  {
> > diff --git a/drivers/media/platform/qcom/venus/helpers.h
> > b/drivers/media/platform/qcom/venus/helpers.h
> > index 92b167a47166..2475f284f396 100644
> > --- a/drivers/media/platform/qcom/venus/helpers.h
> > +++ b/drivers/media/platform/qcom/venus/helpers.h
> > @@ -50,10 +50,16 @@ int venus_helper_set_raw_format(struct
> > venus_inst
> > *inst, u32 hfi_format,
> >  int venus_helper_set_color_format(struct venus_inst *inst, u32
> > fmt);
> >  int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
> >  int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize,
> > u32 buftype);
> > +int venus_helper_set_multistream(struct venus_inst *inst, bool
> > out_en,
> > +				 bool out2_en);
> >  unsigned int venus_helper_get_opb_size(struct venus_inst *inst);
> >  void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
> >  void venus_helper_release_buf_ref(struct venus_inst *inst,
> > unsigned 
> > int idx);
> >  void venus_helper_init_instance(struct venus_inst *inst);
> > +int venus_helper_get_out_fmts(struct venus_inst *inst, u32 fmt,
> > u32 
> > *out_fmt,
> > +			      u32 *out2_fmt, bool ubwc);
> > +int venus_helper_alloc_dpb_bufs(struct venus_inst *inst);
> > +int venus_helper_free_dpb_bufs(struct venus_inst *inst);
> >  int venus_helper_power_enable(struct venus_core *core, u32 
> > session_type,
> >  			      bool enable);
> >  #endif
> > diff --git a/drivers/media/platform/qcom/venus/vdec.c
> > b/drivers/media/platform/qcom/venus/vdec.c
> > index 589fc13b84bc..7deee104ac56 100644
> > --- a/drivers/media/platform/qcom/venus/vdec.c
> > +++ b/drivers/media/platform/qcom/venus/vdec.c
> > @@ -532,10 +532,16 @@ static int vdec_set_properties(struct
> > venus_inst 
> > *inst)
> >  	return 0;
> >  }
> > 
> > +#define is_ubwc_fmt(fmt) (!!((fmt) & HFI_COLOR_FORMAT_UBWC_BASE))
> > +
> >  static int vdec_output_conf(struct venus_inst *inst)
> >  {
> >  	struct venus_core *core = inst->core;
> >  	struct hfi_enable en = { .enable = 1 };
> > +	u32 width = inst->out_width;
> > +	u32 height = inst->out_height;
> > +	u32 out_fmt, out2_fmt;
> > +	bool ubwc = false;
> >  	u32 ptype;
> >  	int ret;
> > 
> > @@ -554,6 +560,78 @@ static int vdec_output_conf(struct venus_inst 
> > *inst)
> >  			return ret;
> >  	}
> > 
> > +	if (width > 1920 && height > ALIGN(1080, 32))
> > +		ubwc = true;
> > +
> > +	if (IS_V4(core))
> > +		ubwc = true;
> > +
> > +	ret = venus_helper_get_out_fmts(inst, inst->fmt_cap-
> > >pixfmt, 
> > &out_fmt,
> > +					&out2_fmt, ubwc);
> > +	if (ret)
> > +		return ret;
> > +
> > +	inst->output_buf_size =
> > +			venus_helper_get_framesz_raw(out_fmt,
> > width, height);
> > +	inst->output2_buf_size =
> > +			venus_helper_get_framesz_raw(out2_fmt,
> > width, height);
> > +
> > +	if (is_ubwc_fmt(out_fmt)) {
> > +		inst->opb_buftype = HFI_BUFFER_OUTPUT2;
> > +		inst->opb_fmt = out2_fmt;
> > +		inst->dpb_buftype = HFI_BUFFER_OUTPUT;
> > +		inst->dpb_fmt = out_fmt;
> > +	} else if (is_ubwc_fmt(out2_fmt)) {
> > +		inst->opb_buftype = HFI_BUFFER_OUTPUT;
> > +		inst->opb_fmt = out_fmt;
> > +		inst->dpb_buftype = HFI_BUFFER_OUTPUT2;
> > +		inst->dpb_fmt = out2_fmt;
> > +	} else {
> > +		inst->opb_buftype = HFI_BUFFER_OUTPUT;
> > +		inst->opb_fmt = out_fmt;
> > +		inst->dpb_buftype = 0;
> > +		inst->dpb_fmt = 0;
> > +	}
> > +
> > +	ret = venus_helper_set_raw_format(inst, inst->opb_fmt,
> > +					  inst->opb_buftype);
> > +	if (ret)
> > +		return ret;
> > +
> > +	if (inst->dpb_fmt) {
> > +		ret = venus_helper_set_multistream(inst, false,
> > true);
> > +		if (ret)
> > +			return ret;
> > +
> > +		ret = venus_helper_set_raw_format(inst, inst-
> > >dpb_fmt,
> > +						  inst-
> > >dpb_buftype);
> > +		if (ret)
> > +			return ret;
> > +
> > +		ret = venus_helper_set_output_resolution(inst,
> > width, height,
> > +							 HFI_BUFFE
> > R_OUTPUT2);
> > +		if (ret)
> > +			return ret;
> > +	}
> > +
> > +	if (IS_V3(core) || IS_V4(core)) {
> > +		if (inst->output2_buf_size) {
> > +			ret = venus_helper_set_bufsize(inst,
> > +						       inst-
> > >output2_buf_size,
> > +						       HFI_BUFFER_
> > OUTPUT2);
> > +			if (ret)
> > +				return ret;
> > +		}
> > +
> > +		if (inst->output_buf_size) {
> > +			ret = venus_helper_set_bufsize(inst,
> > +						       inst-
> > >output_buf_size,
> > +						       HFI_BUFFER_
> > OUTPUT);
> > +			if (ret)
> > +				return ret;
> > +		}
> > +	}
> > +
> >  	ret = venus_helper_set_dyn_bufmode(inst);
> >  	if (ret)
> >  		return ret;
> > @@ -624,6 +702,8 @@ static int vdec_queue_setup(struct vb2_queue
> > *q,
> >  	int ret = 0;
> > 
> >  	if (*num_planes) {
> > +		unsigned int output_buf_size =
> > venus_helper_get_opb_size(inst);
> > +
> >  		if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE
> > &&
> >  		    *num_planes != inst->fmt_out->num_planes)
> >  			return -EINVAL;
> > @@ -637,7 +717,7 @@ static int vdec_queue_setup(struct vb2_queue
> > *q,
> >  			return -EINVAL;
> > 
> >  		if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
> > &&
> > -		    sizes[0] < inst->output_buf_size)
> > +		    sizes[0] < output_buf_size)
> >  			return -EINVAL;
> > 
> >  		return 0;
> > @@ -746,6 +826,10 @@ static int vdec_start_streaming(struct
> > vb2_queue
> > *q, unsigned int count)
> >  	if (ret)
> >  		goto deinit_sess;
> > 
> > +	ret = venus_helper_alloc_dpb_bufs(inst);
> > +	if (ret)
> > +		goto deinit_sess;
> > +
> >  	ret = venus_helper_vb2_start_streaming(inst);
> >  	if (ret)
> >  		goto deinit_sess;
> > @@ -797,9 +881,11 @@ static void vdec_buf_done(struct venus_inst
> > *inst, unsigned int buf_type,
> >  	vbuf->field = V4L2_FIELD_NONE;
> > 
> >  	if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> > +		unsigned int opb_sz =
> > venus_helper_get_opb_size(inst);
> > +
> >  		vb = &vbuf->vb2_buf;
> >  		vb->planes[0].bytesused =
> > -			max_t(unsigned int, inst->output_buf_size, 
> > bytesused);
> > +			max_t(unsigned int, opb_sz, bytesused);
> >  		vb->planes[0].data_offset = data_offset;
> >  		vb->timestamp = timestamp_us * NSEC_PER_USEC;
> >  		vbuf->sequence = inst->sequence_cap++;
> > @@ -945,6 +1031,7 @@ static int vdec_open(struct file *file)
> >  	if (!inst)
> >  		return -ENOMEM;
> > 
> > +	INIT_LIST_HEAD(&inst->dpbbufs);
> >  	INIT_LIST_HEAD(&inst->registeredbufs);
> >  	INIT_LIST_HEAD(&inst->internalbufs);
> >  	INIT_LIST_HEAD(&inst->list);
> > diff --git a/drivers/media/platform/qcom/venus/venc.c
> > b/drivers/media/platform/qcom/venus/venc.c
> > index 54f253b98b24..a703bce78abc 100644
> > --- a/drivers/media/platform/qcom/venus/venc.c
> > +++ b/drivers/media/platform/qcom/venus/venc.c
> > @@ -1084,6 +1084,7 @@ static int venc_open(struct file *file)
> >  	if (!inst)
> >  		return -ENOMEM;
> > 
> > +	INIT_LIST_HEAD(&inst->dpbbufs);
> >  	INIT_LIST_HEAD(&inst->registeredbufs);
> >  	INIT_LIST_HEAD(&inst->internalbufs);
> >  	INIT_LIST_HEAD(&inst->list);
> 
> The dpb buffers queued to hardware will be returned back to host
> either 
> during flush
> or when the session is stopped. Host should not send these buffers
> to 
> client.
> vdec_buf_done should be handling in a way to drop dpb buffers from 
> sending to client.

Are you sure ? In V4L2 the CODEC is only ever flushed or stopped
through userspace actions (STREAMOFF). In which case, userspace expects
all buffers to be dequeued by the driver. Userspace will requeue them
ignoring their content.

> 
> Thanks,
> Vikash

Re: [PATCH 26/28] venus: implementing multi-stream support

[Vikash Garodia]

On 2018-05-02 18:58, Nicolas Dufresne wrote:
> Le mercredi 02 mai 2018 à 13:10 +0530, Vikash Garodia a écrit :
>> Hello Stanimir,
>> 
>> On 2018-04-24 18:14, Stanimir Varbanov wrote:
>> > This is implementing a multi-stream decoder support. The multi
>> > stream gives an option to use the secondary decoder output
>> > with different raw format (or the same in case of crop).
>> >
>> > Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> > ---
>> >  drivers/media/platform/qcom/venus/core.h    |   1 +
>> >  drivers/media/platform/qcom/venus/helpers.c | 204
>> > +++++++++++++++++++++++++++-
>> >  drivers/media/platform/qcom/venus/helpers.h |   6 +
>> >  drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
>> >  drivers/media/platform/qcom/venus/venc.c    |   1 +
>> >  5 files changed, 299 insertions(+), 4 deletions(-)
>> >
>> > diff --git a/drivers/media/platform/qcom/venus/core.h
>> > b/drivers/media/platform/qcom/venus/core.h
>> > index 4d6c05f156c4..85e66e2dd672 100644
>> > --- a/drivers/media/platform/qcom/venus/core.h
>> > +++ b/drivers/media/platform/qcom/venus/core.h
>> > @@ -259,6 +259,7 @@ struct venus_inst {
>> >  	struct list_head list;
>> >  	struct mutex lock;
>> >  	struct venus_core *core;
>> > +	struct list_head dpbbufs;
>> >  	struct list_head internalbufs;
>> >  	struct list_head registeredbufs;
>> >  	struct list_head delayed_process;
>> > diff --git a/drivers/media/platform/qcom/venus/helpers.c
>> > b/drivers/media/platform/qcom/venus/helpers.c
>> > index ed569705ecac..87dcf9973e6f 100644
>> > --- a/drivers/media/platform/qcom/venus/helpers.c
>> > +++ b/drivers/media/platform/qcom/venus/helpers.c
>> > @@ -85,6 +85,112 @@ bool venus_helper_check_codec(struct venus_inst
>> > *inst, u32 v4l2_pixfmt)
>> >  }
>> >  EXPORT_SYMBOL_GPL(venus_helper_check_codec);
>> >
>> > +static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
>> > +{
>> > +	struct intbuf *buf;
>> > +	int ret = 0;
>> > +
>> > +	if (list_empty(&inst->dpbbufs))
>> > +		return 0;
>> > +
>> > +	list_for_each_entry(buf, &inst->dpbbufs, list) {
>> > +		struct hfi_frame_data fdata;
>> > +
>> > +		memset(&fdata, 0, sizeof(fdata));
>> > +		fdata.alloc_len = buf->size;
>> > +		fdata.device_addr = buf->da;
>> > +		fdata.buffer_type = buf->type;
>> > +
>> > +		ret = hfi_session_process_buf(inst, &fdata);
>> > +		if (ret)
>> > +			goto fail;
>> > +	}
>> > +
>> > +fail:
>> > +	return ret;
>> > +}
>> > +
>> > +int venus_helper_free_dpb_bufs(struct venus_inst *inst)
>> > +{
>> > +	struct intbuf *buf, *n;
>> > +
>> > +	if (list_empty(&inst->dpbbufs))
>> > +		return 0;
>> > +
>> > +	list_for_each_entry_safe(buf, n, &inst->dpbbufs, list) {
>> > +		list_del_init(&buf->list);
>> > +		dma_free_attrs(inst->core->dev, buf->size, buf-
>> > >va, buf->da,
>> > +			       buf->attrs);
>> > +		kfree(buf);
>> > +	}
>> > +
>> > +	INIT_LIST_HEAD(&inst->dpbbufs);
>> > +
>> > +	return 0;
>> > +}
>> > +EXPORT_SYMBOL_GPL(venus_helper_free_dpb_bufs);
>> > +
>> > +int venus_helper_alloc_dpb_bufs(struct venus_inst *inst)
>> > +{
>> > +	struct venus_core *core = inst->core;
>> > +	struct device *dev = core->dev;
>> > +	enum hfi_version ver = core->res->hfi_version;
>> > +	struct hfi_buffer_requirements bufreq;
>> > +	u32 buftype = inst->dpb_buftype;
>> > +	unsigned int dpb_size = 0;
>> > +	struct intbuf *buf;
>> > +	unsigned int i;
>> > +	u32 count;
>> > +	int ret;
>> > +
>> > +	/* no need to allocate dpb buffers */
>> > +	if (!inst->dpb_fmt)
>> > +		return 0;
>> > +
>> > +	if (inst->dpb_buftype == HFI_BUFFER_OUTPUT)
>> > +		dpb_size = inst->output_buf_size;
>> > +	else if (inst->dpb_buftype == HFI_BUFFER_OUTPUT2)
>> > +		dpb_size = inst->output2_buf_size;
>> > +
>> > +	if (!dpb_size)
>> > +		return 0;
>> > +
>> > +	ret = venus_helper_get_bufreq(inst, buftype, &bufreq);
>> > +	if (ret)
>> > +		return ret;
>> > +
>> > +	count = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
>> > +
>> > +	for (i = 0; i < count; i++) {
>> > +		buf = kzalloc(sizeof(*buf), GFP_KERNEL);
>> > +		if (!buf) {
>> > +			ret = -ENOMEM;
>> > +			goto fail;
>> > +		}
>> > +
>> > +		buf->type = buftype;
>> > +		buf->size = dpb_size;
>> > +		buf->attrs = DMA_ATTR_WRITE_COMBINE |
>> > +			     DMA_ATTR_NO_KERNEL_MAPPING;
>> > +		buf->va = dma_alloc_attrs(dev, buf->size, &buf-
>> > >da, GFP_KERNEL,
>> > +					  buf->attrs);
>> > +		if (!buf->va) {
>> > +			kfree(buf);
>> > +			ret = -ENOMEM;
>> > +			goto fail;
>> > +		}
>> > +
>> > +		list_add_tail(&buf->list, &inst->dpbbufs);
>> > +	}
>> > +
>> > +	return 0;
>> > +
>> > +fail:
>> > +	venus_helper_free_dpb_bufs(inst);
>> > +	return ret;
>> > +}
>> > +EXPORT_SYMBOL_GPL(venus_helper_alloc_dpb_bufs);
>> > +
>> >  static int intbufs_set_buffer(struct venus_inst *inst, u32 type)
>> >  {
>> >  	struct venus_core *core = inst->core;
>> > @@ -342,7 +448,10 @@ session_process_buf(struct venus_inst *inst,
>> > struct vb2_v4l2_buffer *vbuf)
>> >  		if (vbuf->flags & V4L2_BUF_FLAG_LAST ||
>> > !fdata.filled_len)
>> >  			fdata.flags |= HFI_BUFFERFLAG_EOS;
>> >  	} else if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
>> > -		fdata.buffer_type = HFI_BUFFER_OUTPUT;
>> > +		if (inst->session_type == VIDC_SESSION_TYPE_ENC)
>> > +			fdata.buffer_type = HFI_BUFFER_OUTPUT;
>> > +		else
>> > +			fdata.buffer_type = inst->opb_buftype;
>> >  		fdata.filled_len = 0;
>> >  		fdata.offset = 0;
>> >  	}
>> > @@ -675,6 +784,27 @@ int venus_helper_set_color_format(struct
>> > venus_inst *inst, u32 pixfmt)
>> >  }
>> >  EXPORT_SYMBOL_GPL(venus_helper_set_color_format);
>> >
>> > +int venus_helper_set_multistream(struct venus_inst *inst, bool
>> > out_en,
>> > +				 bool out2_en)
>> > +{
>> > +	struct hfi_multi_stream multi = {0};
>> > +	u32 ptype = HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM;
>> > +	int ret;
>> > +
>> > +	multi.buffer_type = HFI_BUFFER_OUTPUT;
>> > +	multi.enable = out_en;
>> > +
>> > +	ret = hfi_session_set_property(inst, ptype, &multi);
>> > +	if (ret)
>> > +		return ret;
>> > +
>> > +	multi.buffer_type = HFI_BUFFER_OUTPUT2;
>> > +	multi.enable = out2_en;
>> > +
>> > +	return hfi_session_set_property(inst, ptype, &multi);
>> > +}
>> > +EXPORT_SYMBOL_GPL(venus_helper_set_multistream);
>> > +
>> >  int venus_helper_set_dyn_bufmode(struct venus_inst *inst)
>> >  {
>> >  	u32 ptype = HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE;
>> > @@ -822,9 +952,10 @@ EXPORT_SYMBOL_GPL(venus_helper_vb2_buf_init);
>> >  int venus_helper_vb2_buf_prepare(struct vb2_buffer *vb)
>> >  {
>> >  	struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
>> > +	unsigned int out_buf_size =
>> > venus_helper_get_opb_size(inst);
>> >
>> >  	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
>> > -	    vb2_plane_size(vb, 0) < inst->output_buf_size)
>> > +	    vb2_plane_size(vb, 0) < out_buf_size)
>> >  		return -EINVAL;
>> >  	if (vb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE &&
>> >  	    vb2_plane_size(vb, 0) < inst->input_buf_size)
>> > @@ -894,6 +1025,8 @@ void venus_helper_vb2_stop_streaming(struct
>> > vb2_queue *q)
>> >  		if (ret)
>> >  			hfi_session_abort(inst);
>> >
>> > +		venus_helper_free_dpb_bufs(inst);
>> > +
>> >  		load_scale_clocks(core);
>> >  		INIT_LIST_HEAD(&inst->registeredbufs);
>> >  	}
>> > @@ -932,8 +1065,14 @@ int venus_helper_vb2_start_streaming(struct
>> > venus_inst *inst)
>> >  	if (ret)
>> >  		goto err_unload_res;
>> >
>> > +	ret = venus_helper_queue_dpb_bufs(inst);
>> > +	if (ret)
>> > +		goto err_session_stop;
>> > +
>> >  	return 0;
>> >
>> > +err_session_stop:
>> > +	hfi_session_stop(inst);
>> >  err_unload_res:
>> >  	hfi_session_unload_res(inst);
>> >  err_unreg_bufs:
>> > @@ -987,6 +1126,67 @@ void venus_helper_init_instance(struct
>> > venus_inst
>> > *inst)
>> >  }
>> >  EXPORT_SYMBOL_GPL(venus_helper_init_instance);
>> >
>> > +static bool find_fmt_from_caps(struct venus_caps *caps, u32
>> > buftype,
>> > u32 fmt)
>> > +{
>> > +	unsigned int i;
>> > +
>> > +	for (i = 0; i < caps->num_fmts; i++) {
>> > +		if (caps->fmts[i].buftype == buftype &&
>> > +		    caps->fmts[i].fmt == fmt)
>> > +			return true;
>> > +	}
>> > +
>> > +	return false;
>> > +}
>> > +
>> > +int venus_helper_get_out_fmts(struct venus_inst *inst, u32
>> > v4l2_fmt,
>> > +			      u32 *out_fmt, u32 *out2_fmt, bool
>> > ubwc)
>> > +{
>> > +	struct venus_core *core = inst->core;
>> > +	struct venus_caps *caps;
>> > +	u32 ubwc_fmt, fmt = to_hfi_raw_fmt(v4l2_fmt);
>> > +	bool found, found_ubwc;
>> > +
>> > +	*out_fmt = *out2_fmt = 0;
>> > +
>> > +	if (!fmt)
>> > +		return -EINVAL;
>> > +
>> > +	caps = venus_caps_by_codec(core, inst->hfi_codec,
>> > inst->session_type);
>> > +	if (!caps)
>> > +		return -EINVAL;
>> > +
>> > +	if (ubwc) {
>> > +		ubwc_fmt = fmt | HFI_COLOR_FORMAT_UBWC_BASE;
>> > +		found_ubwc = find_fmt_from_caps(caps,
>> > HFI_BUFFER_OUTPUT,
>> > +						ubwc_fmt);
>> > +		found = find_fmt_from_caps(caps,
>> > HFI_BUFFER_OUTPUT2, fmt);
>> > +
>> > +		if (found_ubwc && found) {
>> > +			*out_fmt = ubwc_fmt;
>> > +			*out2_fmt = fmt;
>> > +			return 0;
>> > +		}
>> > +	}
>> > +
>> > +	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT, fmt);
>> > +	if (found) {
>> > +		*out_fmt = fmt;
>> > +		*out2_fmt = 0;
>> > +		return 0;
>> > +	}
>> > +
>> > +	found = find_fmt_from_caps(caps, HFI_BUFFER_OUTPUT2, fmt);
>> > +	if (found) {
>> > +		*out_fmt = 0;
>> > +		*out2_fmt = fmt;
>> > +		return 0;
>> > +	}
>> > +
>> > +	return -EINVAL;
>> > +}
>> > +EXPORT_SYMBOL_GPL(venus_helper_get_out_fmts);
>> > +
>> >  int venus_helper_power_enable(struct venus_core *core, u32
>> > session_type,
>> >  			      bool enable)
>> >  {
>> > diff --git a/drivers/media/platform/qcom/venus/helpers.h
>> > b/drivers/media/platform/qcom/venus/helpers.h
>> > index 92b167a47166..2475f284f396 100644
>> > --- a/drivers/media/platform/qcom/venus/helpers.h
>> > +++ b/drivers/media/platform/qcom/venus/helpers.h
>> > @@ -50,10 +50,16 @@ int venus_helper_set_raw_format(struct
>> > venus_inst
>> > *inst, u32 hfi_format,
>> >  int venus_helper_set_color_format(struct venus_inst *inst, u32
>> > fmt);
>> >  int venus_helper_set_dyn_bufmode(struct venus_inst *inst);
>> >  int venus_helper_set_bufsize(struct venus_inst *inst, u32 bufsize,
>> > u32 buftype);
>> > +int venus_helper_set_multistream(struct venus_inst *inst, bool
>> > out_en,
>> > +				 bool out2_en);
>> >  unsigned int venus_helper_get_opb_size(struct venus_inst *inst);
>> >  void venus_helper_acquire_buf_ref(struct vb2_v4l2_buffer *vbuf);
>> >  void venus_helper_release_buf_ref(struct venus_inst *inst,
>> > unsigned
>> > int idx);
>> >  void venus_helper_init_instance(struct venus_inst *inst);
>> > +int venus_helper_get_out_fmts(struct venus_inst *inst, u32 fmt,
>> > u32
>> > *out_fmt,
>> > +			      u32 *out2_fmt, bool ubwc);
>> > +int venus_helper_alloc_dpb_bufs(struct venus_inst *inst);
>> > +int venus_helper_free_dpb_bufs(struct venus_inst *inst);
>> >  int venus_helper_power_enable(struct venus_core *core, u32
>> > session_type,
>> >  			      bool enable);
>> >  #endif
>> > diff --git a/drivers/media/platform/qcom/venus/vdec.c
>> > b/drivers/media/platform/qcom/venus/vdec.c
>> > index 589fc13b84bc..7deee104ac56 100644
>> > --- a/drivers/media/platform/qcom/venus/vdec.c
>> > +++ b/drivers/media/platform/qcom/venus/vdec.c
>> > @@ -532,10 +532,16 @@ static int vdec_set_properties(struct
>> > venus_inst
>> > *inst)
>> >  	return 0;
>> >  }
>> >
>> > +#define is_ubwc_fmt(fmt) (!!((fmt) & HFI_COLOR_FORMAT_UBWC_BASE))
>> > +
>> >  static int vdec_output_conf(struct venus_inst *inst)
>> >  {
>> >  	struct venus_core *core = inst->core;
>> >  	struct hfi_enable en = { .enable = 1 };
>> > +	u32 width = inst->out_width;
>> > +	u32 height = inst->out_height;
>> > +	u32 out_fmt, out2_fmt;
>> > +	bool ubwc = false;
>> >  	u32 ptype;
>> >  	int ret;
>> >
>> > @@ -554,6 +560,78 @@ static int vdec_output_conf(struct venus_inst
>> > *inst)
>> >  			return ret;
>> >  	}
>> >
>> > +	if (width > 1920 && height > ALIGN(1080, 32))
>> > +		ubwc = true;
>> > +
>> > +	if (IS_V4(core))
>> > +		ubwc = true;
>> > +
>> > +	ret = venus_helper_get_out_fmts(inst, inst->fmt_cap-
>> > >pixfmt,
>> > &out_fmt,
>> > +					&out2_fmt, ubwc);
>> > +	if (ret)
>> > +		return ret;
>> > +
>> > +	inst->output_buf_size =
>> > +			venus_helper_get_framesz_raw(out_fmt,
>> > width, height);
>> > +	inst->output2_buf_size =
>> > +			venus_helper_get_framesz_raw(out2_fmt,
>> > width, height);
>> > +
>> > +	if (is_ubwc_fmt(out_fmt)) {
>> > +		inst->opb_buftype = HFI_BUFFER_OUTPUT2;
>> > +		inst->opb_fmt = out2_fmt;
>> > +		inst->dpb_buftype = HFI_BUFFER_OUTPUT;
>> > +		inst->dpb_fmt = out_fmt;
>> > +	} else if (is_ubwc_fmt(out2_fmt)) {
>> > +		inst->opb_buftype = HFI_BUFFER_OUTPUT;
>> > +		inst->opb_fmt = out_fmt;
>> > +		inst->dpb_buftype = HFI_BUFFER_OUTPUT2;
>> > +		inst->dpb_fmt = out2_fmt;
>> > +	} else {
>> > +		inst->opb_buftype = HFI_BUFFER_OUTPUT;
>> > +		inst->opb_fmt = out_fmt;
>> > +		inst->dpb_buftype = 0;
>> > +		inst->dpb_fmt = 0;
>> > +	}
>> > +
>> > +	ret = venus_helper_set_raw_format(inst, inst->opb_fmt,
>> > +					  inst->opb_buftype);
>> > +	if (ret)
>> > +		return ret;
>> > +
>> > +	if (inst->dpb_fmt) {
>> > +		ret = venus_helper_set_multistream(inst, false,
>> > true);
>> > +		if (ret)
>> > +			return ret;
>> > +
>> > +		ret = venus_helper_set_raw_format(inst, inst-
>> > >dpb_fmt,
>> > +						  inst-
>> > >dpb_buftype);
>> > +		if (ret)
>> > +			return ret;
>> > +
>> > +		ret = venus_helper_set_output_resolution(inst,
>> > width, height,
>> > +							 HFI_BUFFE
>> > R_OUTPUT2);
>> > +		if (ret)
>> > +			return ret;
>> > +	}
>> > +
>> > +	if (IS_V3(core) || IS_V4(core)) {
>> > +		if (inst->output2_buf_size) {
>> > +			ret = venus_helper_set_bufsize(inst,
>> > +						       inst-
>> > >output2_buf_size,
>> > +						       HFI_BUFFER_
>> > OUTPUT2);
>> > +			if (ret)
>> > +				return ret;
>> > +		}
>> > +
>> > +		if (inst->output_buf_size) {
>> > +			ret = venus_helper_set_bufsize(inst,
>> > +						       inst-
>> > >output_buf_size,
>> > +						       HFI_BUFFER_
>> > OUTPUT);
>> > +			if (ret)
>> > +				return ret;
>> > +		}
>> > +	}
>> > +
>> >  	ret = venus_helper_set_dyn_bufmode(inst);
>> >  	if (ret)
>> >  		return ret;
>> > @@ -624,6 +702,8 @@ static int vdec_queue_setup(struct vb2_queue
>> > *q,
>> >  	int ret = 0;
>> >
>> >  	if (*num_planes) {
>> > +		unsigned int output_buf_size =
>> > venus_helper_get_opb_size(inst);
>> > +
>> >  		if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE
>> > &&
>> >  		    *num_planes != inst->fmt_out->num_planes)
>> >  			return -EINVAL;
>> > @@ -637,7 +717,7 @@ static int vdec_queue_setup(struct vb2_queue
>> > *q,
>> >  			return -EINVAL;
>> >
>> >  		if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
>> > &&
>> > -		    sizes[0] < inst->output_buf_size)
>> > +		    sizes[0] < output_buf_size)
>> >  			return -EINVAL;
>> >
>> >  		return 0;
>> > @@ -746,6 +826,10 @@ static int vdec_start_streaming(struct
>> > vb2_queue
>> > *q, unsigned int count)
>> >  	if (ret)
>> >  		goto deinit_sess;
>> >
>> > +	ret = venus_helper_alloc_dpb_bufs(inst);
>> > +	if (ret)
>> > +		goto deinit_sess;
>> > +
>> >  	ret = venus_helper_vb2_start_streaming(inst);
>> >  	if (ret)
>> >  		goto deinit_sess;
>> > @@ -797,9 +881,11 @@ static void vdec_buf_done(struct venus_inst
>> > *inst, unsigned int buf_type,
>> >  	vbuf->field = V4L2_FIELD_NONE;
>> >
>> >  	if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
>> > +		unsigned int opb_sz =
>> > venus_helper_get_opb_size(inst);
>> > +
>> >  		vb = &vbuf->vb2_buf;
>> >  		vb->planes[0].bytesused =
>> > -			max_t(unsigned int, inst->output_buf_size,
>> > bytesused);
>> > +			max_t(unsigned int, opb_sz, bytesused);
>> >  		vb->planes[0].data_offset = data_offset;
>> >  		vb->timestamp = timestamp_us * NSEC_PER_USEC;
>> >  		vbuf->sequence = inst->sequence_cap++;
>> > @@ -945,6 +1031,7 @@ static int vdec_open(struct file *file)
>> >  	if (!inst)
>> >  		return -ENOMEM;
>> >
>> > +	INIT_LIST_HEAD(&inst->dpbbufs);
>> >  	INIT_LIST_HEAD(&inst->registeredbufs);
>> >  	INIT_LIST_HEAD(&inst->internalbufs);
>> >  	INIT_LIST_HEAD(&inst->list);
>> > diff --git a/drivers/media/platform/qcom/venus/venc.c
>> > b/drivers/media/platform/qcom/venus/venc.c
>> > index 54f253b98b24..a703bce78abc 100644
>> > --- a/drivers/media/platform/qcom/venus/venc.c
>> > +++ b/drivers/media/platform/qcom/venus/venc.c
>> > @@ -1084,6 +1084,7 @@ static int venc_open(struct file *file)
>> >  	if (!inst)
>> >  		return -ENOMEM;
>> >
>> > +	INIT_LIST_HEAD(&inst->dpbbufs);
>> >  	INIT_LIST_HEAD(&inst->registeredbufs);
>> >  	INIT_LIST_HEAD(&inst->internalbufs);
>> >  	INIT_LIST_HEAD(&inst->list);
>> 
>> The dpb buffers queued to hardware will be returned back to host
>> either
>> during flush
>> or when the session is stopped. Host should not send these buffers
>> to
>> client.
>> vdec_buf_done should be handling in a way to drop dpb buffers from
>> sending to client.
> 
> Are you sure ? In V4L2 the CODEC is only ever flushed or stopped
> through userspace actions (STREAMOFF). In which case, userspace expects
> all buffers to be dequeued by the driver. Userspace will requeue them
> ignoring their content.

DPB buffers, which i mentioned earlier, are managed 
(allocated/queued/freed)
internally in video driver. Client is nowhere aware of it and does not 
expects
the same to be dequeued/requeue.

>> 
>> Thanks,
>> Vikash

Re: [PATCH 26/28] venus: implementing multi-stream support

[Stanimir Varbanov]

Hi Vikash,

Please write the comments for the chunk of code for which they are refer to.

On  2.05.2018 10:40, Vikash Garodia wrote:
> Hello Stanimir,
> 
> On 2018-04-24 18:14, Stanimir Varbanov wrote:
>> This is implementing a multi-stream decoder support. The multi
>> stream gives an option to use the secondary decoder output
>> with different raw format (or the same in case of crop).
>>
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>  drivers/media/platform/qcom/venus/core.h    |   1 +
>>  drivers/media/platform/qcom/venus/helpers.c | 204 
>> +++++++++++++++++++++++++++-
>>  drivers/media/platform/qcom/venus/helpers.h |   6 +
>>  drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
>>  drivers/media/platform/qcom/venus/venc.c    |   1 +
>>  5 files changed, 299 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/media/platform/qcom/venus/core.h
>> b/drivers/media/platform/qcom/venus/core.h
>> index 4d6c05f156c4..85e66e2dd672 100644
>> --- a/drivers/media/platform/qcom/venus/core.h
>> +++ b/drivers/media/platform/qcom/venus/core.h
>> @@ -259,6 +259,7 @@ struct venus_inst {
>>      struct list_head list;
>>      struct mutex lock;
>>      struct venus_core *core;
>> +    struct list_head dpbbufs;
>>      struct list_head internalbufs;
>>      struct list_head registeredbufs;
>>      struct list_head delayed_process;

<snip>

> 
> The dpb buffers queued to hardware will be returned back to host either 
> during flush
> or when the session is stopped. Host should not send these buffers to 
> client.

That's correct.

> vdec_buf_done should be handling in a way to drop dpb buffers from 
> sending to client.

That is also correct, vdec_buf_done is trying to find the buffer by 
index from a list of queued buffers from v4l2 clients. See 
venus_helper_vb2_buf_queue where it is calling v4l2_m2m_buf_queue.

So for the dpb buffers venus_helper_find_buf() will return NULL.

regards,
Stan

Re: [PATCH 26/28] venus: implementing multi-stream support

[Vikash Garodia]

Hi Stanimir,

On 2018-05-03 12:42, Stanimir Varbanov wrote:
> Hi Vikash,
> 
> Please write the comments for the chunk of code for which they are 
> refer to.
I see that the patch was about handling multistream mode, but the code 
to handle
the dpb buffer response is missing. My comment is basically to add the 
required
code.

> On  2.05.2018 10:40, Vikash Garodia wrote:
>> Hello Stanimir,
>> 
>> On 2018-04-24 18:14, Stanimir Varbanov wrote:
>>> This is implementing a multi-stream decoder support. The multi
>>> stream gives an option to use the secondary decoder output
>>> with different raw format (or the same in case of crop).
>>> 
>>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>>> ---
>>>  drivers/media/platform/qcom/venus/core.h    |   1 +
>>>  drivers/media/platform/qcom/venus/helpers.c | 204 
>>> +++++++++++++++++++++++++++-
>>>  drivers/media/platform/qcom/venus/helpers.h |   6 +
>>>  drivers/media/platform/qcom/venus/vdec.c    |  91 ++++++++++++-
>>>  drivers/media/platform/qcom/venus/venc.c    |   1 +
>>>  5 files changed, 299 insertions(+), 4 deletions(-)
>>> 
>>> diff --git a/drivers/media/platform/qcom/venus/core.h
>>> b/drivers/media/platform/qcom/venus/core.h
>>> index 4d6c05f156c4..85e66e2dd672 100644
>>> --- a/drivers/media/platform/qcom/venus/core.h
>>> +++ b/drivers/media/platform/qcom/venus/core.h
>>> @@ -259,6 +259,7 @@ struct venus_inst {
>>>      struct list_head list;
>>>      struct mutex lock;
>>>      struct venus_core *core;
>>> +    struct list_head dpbbufs;
>>>      struct list_head internalbufs;
>>>      struct list_head registeredbufs;
>>>      struct list_head delayed_process;
> 
> <snip>
> 
>> 
>> The dpb buffers queued to hardware will be returned back to host 
>> either during flush
>> or when the session is stopped. Host should not send these buffers to 
>> client.
> 
> That's correct.
> 
>> vdec_buf_done should be handling in a way to drop dpb buffers from 
>> sending to client.
> 
> That is also correct, vdec_buf_done is trying to find the buffer by
> index from a list of queued buffers from v4l2 clients. See
> venus_helper_vb2_buf_queue where it is calling v4l2_m2m_buf_queue.
> 
> So for the dpb buffers venus_helper_find_buf() will return NULL.

My bad, i could see that the DPB buffers are not sent to client in the 
existing patch.

Instead of bailing out on NULL, i was thinking it is better to keep 
explicit check for dpb
buffers and exit with a debug log.

> 
> regards,
> Stan

[PATCH 27/28] venus: add sdm845 compatible and resource data

[Stanimir Varbanov]

This adds sdm845 DT compatible string with it's resource
data table.

Cc: devicetree@vger.kernel.org
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 .../devicetree/bindings/media/qcom,venus.txt       |  1 +
 drivers/media/platform/qcom/venus/core.c           | 22 ++++++++++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/qcom,venus.txt b/Documentation/devicetree/bindings/media/qcom,venus.txt
index 2693449daf73..00d0d1bf7647 100644
--- a/Documentation/devicetree/bindings/media/qcom,venus.txt
+++ b/Documentation/devicetree/bindings/media/qcom,venus.txt
@@ -6,6 +6,7 @@
 	Definition: Value should contain one of:
 		- "qcom,msm8916-venus"
 		- "qcom,msm8996-venus"
+		- "qcom,sdm845-venus"
 - reg:
 	Usage: required
 	Value type: <prop-encoded-array>
diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/venus/core.c
index 1b72bfbb6297..13084880cc42 100644
--- a/drivers/media/platform/qcom/venus/core.c
+++ b/drivers/media/platform/qcom/venus/core.c
@@ -445,9 +445,31 @@ static const struct venus_resources msm8996_res = {
 	.fwname = "qcom/venus-4.2/venus.mdt",
 };
 
+static const struct freq_tbl sdm845_freq_table[] = {
+	{ 1944000, 380000000 },	/* 4k UHD @ 60 */
+	{  972000, 320000000 },	/* 4k UHD @ 30 */
+	{  489600, 200000000 },	/* 1080p @ 60 */
+	{  244800, 100000000 },	/* 1080p @ 30 */
+};
+
+static const struct venus_resources sdm845_res = {
+	.freq_tbl = sdm845_freq_table,
+	.freq_tbl_size = ARRAY_SIZE(sdm845_freq_table),
+	.clks = {"core", "iface", "bus" },
+	.clks_num = 3,
+	.max_load = 2563200,
+	.hfi_version = HFI_VERSION_4XX,
+	.vmem_id = VIDC_RESOURCE_NONE,
+	.vmem_size = 0,
+	.vmem_addr = 0,
+	.dma_mask = 0xe0000000 - 1,
+	.fwname = "qcom/venus-5.2/venus.mdt",
+};
+
 static const struct of_device_id venus_dt_match[] = {
 	{ .compatible = "qcom,msm8916-venus", .data = &msm8916_res, },
 	{ .compatible = "qcom,msm8996-venus", .data = &msm8996_res, },
+	{ .compatible = "qcom,sdm845-venus", .data = &sdm845_res, },
 	{ }
 };
 MODULE_DEVICE_TABLE(of, venus_dt_match);
-- 
2.14.1

Re: [PATCH 27/28] venus: add sdm845 compatible and resource data

[Rob Herring]

On Tue, Apr 24, 2018 at 03:44:35PM +0300, Stanimir Varbanov wrote:
> This adds sdm845 DT compatible string with it's resource
> data table.
> 
> Cc: devicetree@vger.kernel.org
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  .../devicetree/bindings/media/qcom,venus.txt       |  1 +
>  drivers/media/platform/qcom/venus/core.c           | 22 ++++++++++++++++++++++
>  2 files changed, 23 insertions(+)

Reviewed-by: Rob Herring <robh@kernel.org>

[PATCH 28/28] venus: add HEVC codec support

[Stanimir Varbanov]

This add HEVC codec support for venus versions 3xx and 4xx.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
 drivers/media/platform/qcom/venus/helpers.c | 3 +++
 drivers/media/platform/qcom/venus/hfi.c     | 2 ++
 drivers/media/platform/qcom/venus/vdec.c    | 4 ++++
 drivers/media/platform/qcom/venus/venc.c    | 4 ++++
 4 files changed, 13 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 87dcf9973e6f..fecadba039cf 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -71,6 +71,9 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
 	case V4L2_PIX_FMT_XVID:
 		codec = HFI_VIDEO_CODEC_DIVX;
 		break;
+	case V4L2_PIX_FMT_HEVC:
+		codec = HFI_VIDEO_CODEC_HEVC;
+		break;
 	default:
 		return false;
 	}
diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
index 94ca27b0bb99..24207829982f 100644
--- a/drivers/media/platform/qcom/venus/hfi.c
+++ b/drivers/media/platform/qcom/venus/hfi.c
@@ -49,6 +49,8 @@ static u32 to_codec_type(u32 pixfmt)
 		return HFI_VIDEO_CODEC_VP9;
 	case V4L2_PIX_FMT_XVID:
 		return HFI_VIDEO_CODEC_DIVX;
+	case V4L2_PIX_FMT_HEVC:
+		return HFI_VIDEO_CODEC_HEVC;
 	default:
 		return 0;
 	}
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 7deee104ac56..a114f421edad 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -77,6 +77,10 @@ static const struct venus_format vdec_formats[] = {
 		.pixfmt = V4L2_PIX_FMT_XVID,
 		.num_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_HEVC,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
 	},
 };
 
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index a703bce78abc..fdb76b69786f 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -59,6 +59,10 @@ static const struct venus_format venc_formats[] = {
 		.pixfmt = V4L2_PIX_FMT_VP8,
 		.num_planes = 1,
 		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
+	}, {
+		.pixfmt = V4L2_PIX_FMT_HEVC,
+		.num_planes = 1,
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
 	},
 };
 
-- 
2.14.1

Re: [PATCH 28/28] venus: add HEVC codec support

[Hans Verkuil]

On 24/04/18 14:44, Stanimir Varbanov wrote:
> This add HEVC codec support for venus versions 3xx and 4xx.
> 
> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
> ---
>  drivers/media/platform/qcom/venus/helpers.c | 3 +++
>  drivers/media/platform/qcom/venus/hfi.c     | 2 ++
>  drivers/media/platform/qcom/venus/vdec.c    | 4 ++++
>  drivers/media/platform/qcom/venus/venc.c    | 4 ++++
>  4 files changed, 13 insertions(+)
> 
> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
> index 87dcf9973e6f..fecadba039cf 100644
> --- a/drivers/media/platform/qcom/venus/helpers.c
> +++ b/drivers/media/platform/qcom/venus/helpers.c
> @@ -71,6 +71,9 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
>  	case V4L2_PIX_FMT_XVID:
>  		codec = HFI_VIDEO_CODEC_DIVX;
>  		break;
> +	case V4L2_PIX_FMT_HEVC:
> +		codec = HFI_VIDEO_CODEC_HEVC;
> +		break;
>  	default:
>  		return false;
>  	}
> diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
> index 94ca27b0bb99..24207829982f 100644
> --- a/drivers/media/platform/qcom/venus/hfi.c
> +++ b/drivers/media/platform/qcom/venus/hfi.c
> @@ -49,6 +49,8 @@ static u32 to_codec_type(u32 pixfmt)
>  		return HFI_VIDEO_CODEC_VP9;
>  	case V4L2_PIX_FMT_XVID:
>  		return HFI_VIDEO_CODEC_DIVX;
> +	case V4L2_PIX_FMT_HEVC:
> +		return HFI_VIDEO_CODEC_HEVC;
>  	default:
>  		return 0;
>  	}
> diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
> index 7deee104ac56..a114f421edad 100644
> --- a/drivers/media/platform/qcom/venus/vdec.c
> +++ b/drivers/media/platform/qcom/venus/vdec.c
> @@ -77,6 +77,10 @@ static const struct venus_format vdec_formats[] = {
>  		.pixfmt = V4L2_PIX_FMT_XVID,
>  		.num_planes = 1,
>  		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
> +	}, {
> +		.pixfmt = V4L2_PIX_FMT_HEVC,
> +		.num_planes = 1,
> +		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
>  	},
>  };
>  
> diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
> index a703bce78abc..fdb76b69786f 100644
> --- a/drivers/media/platform/qcom/venus/venc.c
> +++ b/drivers/media/platform/qcom/venus/venc.c
> @@ -59,6 +59,10 @@ static const struct venus_format venc_formats[] = {
>  		.pixfmt = V4L2_PIX_FMT_VP8,
>  		.num_planes = 1,
>  		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
> +	}, {
> +		.pixfmt = V4L2_PIX_FMT_HEVC,
> +		.num_planes = 1,
> +		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>  	},
>  };
>  
> 

No changes are necessary to venc_set_properties() for HEVC support?

Just checking, I kind of expected that.

Regards,

	Hans

Re: [PATCH 28/28] venus: add HEVC codec support

[Stanimir Varbanov]

Hi Hans,

On  7.05.2018 13:39, Hans Verkuil wrote:
> On 24/04/18 14:44, Stanimir Varbanov wrote:
>> This add HEVC codec support for venus versions 3xx and 4xx.
>>
>> Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
>> ---
>>   drivers/media/platform/qcom/venus/helpers.c | 3 +++
>>   drivers/media/platform/qcom/venus/hfi.c     | 2 ++
>>   drivers/media/platform/qcom/venus/vdec.c    | 4 ++++
>>   drivers/media/platform/qcom/venus/venc.c    | 4 ++++
>>   4 files changed, 13 insertions(+)
>>
>> diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
>> index 87dcf9973e6f..fecadba039cf 100644
>> --- a/drivers/media/platform/qcom/venus/helpers.c
>> +++ b/drivers/media/platform/qcom/venus/helpers.c
>> @@ -71,6 +71,9 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
>>   	case V4L2_PIX_FMT_XVID:
>>   		codec = HFI_VIDEO_CODEC_DIVX;
>>   		break;
>> +	case V4L2_PIX_FMT_HEVC:
>> +		codec = HFI_VIDEO_CODEC_HEVC;
>> +		break;
>>   	default:
>>   		return false;
>>   	}
>> diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
>> index 94ca27b0bb99..24207829982f 100644
>> --- a/drivers/media/platform/qcom/venus/hfi.c
>> +++ b/drivers/media/platform/qcom/venus/hfi.c
>> @@ -49,6 +49,8 @@ static u32 to_codec_type(u32 pixfmt)
>>   		return HFI_VIDEO_CODEC_VP9;
>>   	case V4L2_PIX_FMT_XVID:
>>   		return HFI_VIDEO_CODEC_DIVX;
>> +	case V4L2_PIX_FMT_HEVC:
>> +		return HFI_VIDEO_CODEC_HEVC;
>>   	default:
>>   		return 0;
>>   	}
>> diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
>> index 7deee104ac56..a114f421edad 100644
>> --- a/drivers/media/platform/qcom/venus/vdec.c
>> +++ b/drivers/media/platform/qcom/venus/vdec.c
>> @@ -77,6 +77,10 @@ static const struct venus_format vdec_formats[] = {
>>   		.pixfmt = V4L2_PIX_FMT_XVID,
>>   		.num_planes = 1,
>>   		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
>> +	}, {
>> +		.pixfmt = V4L2_PIX_FMT_HEVC,
>> +		.num_planes = 1,
>> +		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
>>   	},
>>   };
>>   
>> diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
>> index a703bce78abc..fdb76b69786f 100644
>> --- a/drivers/media/platform/qcom/venus/venc.c
>> +++ b/drivers/media/platform/qcom/venus/venc.c
>> @@ -59,6 +59,10 @@ static const struct venus_format venc_formats[] = {
>>   		.pixfmt = V4L2_PIX_FMT_VP8,
>>   		.num_planes = 1,
>>   		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>> +	}, {
>> +		.pixfmt = V4L2_PIX_FMT_HEVC,
>> +		.num_planes = 1,
>> +		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE,
>>   	},
>>   };
>>   
>>
> 
> No changes are necessary to venc_set_properties() for HEVC support?
> 
> Just checking, I kind of expected that.

yes, I think so, but will leave that for future updates.

regards,
Stan

Re: [PATCH 00/28] Venus updates

[Hans Verkuil]

On 24/04/18 14:44, Stanimir Varbanov wrote:
> Hello,
> 
> This patch set aims to:
> 
> * add initial support for Venus version 4xx (found on sdm845).
> 
> * introduce a common capability parser to enumerate better
>   supported uncompressed formats, capabilities by codec,
>   supported codecs and so on.
> 
> * also contains various cleanups, readability improvements
>   and fixes.
> 
> * adds HEVC codec support for the Venus versions which has
>   support for it.
> 
> * add multi-stream support (secondary decoder output), which
>   will give as an opportunity to use UBWC compressed formats
>   to optimize internal interconnect bandwidth on higher
>   resolutions.

I'm a bit confused about this: is this a purely driver-internal thing,
or is this exposed somehow to userspace as well? It seems to be purely
internal.

Regards,

	Hans

> 
> Comments are welcome!
> 
> regards,
> Stan
> 
> Stanimir Varbanov (28):
>   venus: hfi_msgs: correct pointer increment
>   venus: hfi: preparation to support venus 4xx
>   venus: hfi: update sequence event to handle more properties
>   venus: hfi_cmds: add set_properties for 4xx version
>   venus: hfi: support session continue for 4xx version
>   venus: hfi: handle buffer output2 type as well
>   venus: hfi_venus: add halt AXI support for Venus 4xx
>   venus: hfi_venus: add suspend function for 4xx version
>   venus: venc,vdec: adds clocks needed for venus 4xx
>   venus: vdec: call session_continue in insufficient event
>   venus: add common capability parser
>   venus: helpers: make a commmon function for power_enable
>   venus: core: delete not used flag for buffer mode
>   venus: helpers: rename a helper function and use buffer mode from caps
>   venus: add a helper function to set dynamic buffer mode
>   venus: add helper function to set actual buffer size
>   venus: delete no longer used bufmode flag from instance
>   venus: helpers: add buffer type argument to a helper
>   venus: helpers: add a new helper to set raw format
>   venus: helpers,vdec,venc: add helpers to set work mode and core usage
>   venus: helpers: extend set_num_bufs helper with one more argument
>   venus: helpers: add a helper to return opb buffer sizes
>   venus: vdec: get required input buffers as well
>   venus: vdec: new function for output configuration
>   venus: move frame size calculations in common place
>   venus: implementing multi-stream support
>   venus: add sdm845 compatible and resource data
>   venus: add HEVC codec support
> 
>  .../devicetree/bindings/media/qcom,venus.txt       |   1 +
>  drivers/media/platform/qcom/venus/Makefile         |   3 +-
>  drivers/media/platform/qcom/venus/core.c           | 102 ++++
>  drivers/media/platform/qcom/venus/core.h           |  91 ++--
>  drivers/media/platform/qcom/venus/helpers.c        | 558 +++++++++++++++++++--
>  drivers/media/platform/qcom/venus/helpers.h        |  23 +-
>  drivers/media/platform/qcom/venus/hfi.c            |  12 +-
>  drivers/media/platform/qcom/venus/hfi.h            |   9 +
>  drivers/media/platform/qcom/venus/hfi_cmds.c       |  64 ++-
>  drivers/media/platform/qcom/venus/hfi_helper.h     | 112 ++++-
>  drivers/media/platform/qcom/venus/hfi_msgs.c       | 401 +++------------
>  drivers/media/platform/qcom/venus/hfi_parser.c     | 290 +++++++++++
>  drivers/media/platform/qcom/venus/hfi_parser.h     |  45 ++
>  drivers/media/platform/qcom/venus/hfi_venus.c      |  69 +++
>  drivers/media/platform/qcom/venus/hfi_venus_io.h   |  24 +
>  drivers/media/platform/qcom/venus/vdec.c           | 324 +++++++-----
>  drivers/media/platform/qcom/venus/venc.c           | 166 +++---
>  17 files changed, 1641 insertions(+), 653 deletions(-)
>  create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.c
>  create mode 100644 drivers/media/platform/qcom/venus/hfi_parser.h
> 

Re: [PATCH 00/28] Venus updates

[Stanimir Varbanov]

Hi Hans,

On  7.05.2018 13:41, Hans Verkuil wrote:
> On 24/04/18 14:44, Stanimir Varbanov wrote:
>> Hello,
>>
>> This patch set aims to:
>>
>> * add initial support for Venus version 4xx (found on sdm845).
>>
>> * introduce a common capability parser to enumerate better
>>    supported uncompressed formats, capabilities by codec,
>>    supported codecs and so on.
>>
>> * also contains various cleanups, readability improvements
>>    and fixes.
>>
>> * adds HEVC codec support for the Venus versions which has
>>    support for it.
>>
>> * add multi-stream support (secondary decoder output), which
>>    will give as an opportunity to use UBWC compressed formats
>>    to optimize internal interconnect bandwidth on higher
>>    resolutions.
> 
> I'm a bit confused about this: is this a purely driver-internal thing,
> or is this exposed somehow to userspace as well? It seems to be purely
> internal.

For now it'll be internal for the driver. In downstream driver it is 
exposed to userspace via custom v4l_control but that is not generic and 
I decided to not expose it.

regards,
Stan