* [PATCH v2 01/11] venus: venc: amend buffer size for bitstream plane
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
@ 2019-06-28 12:59 ` Stanimir Varbanov
2019-06-28 12:59 ` [PATCH v2 02/11] venus: helpers: export few helper functions Stanimir Varbanov
` (10 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 12:59 UTC (permalink / raw)
To: linux-media
Cc: Mauro Carvalho Chehab, Hans Verkuil, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot, Malathi Gottam,
Stanimir Varbanov
From: Malathi Gottam <mgottam@codeaurora.org>
Accept the buffer size requested by client and compare it
against driver calculated size and set the maximum to
bitstream plane.
Signed-off-by: Malathi Gottam <mgottam@codeaurora.org>
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/venc.c | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index a5f3d2c46bea..1b7fb2d5887c 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -294,6 +294,7 @@ venc_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
const struct venus_format *fmt;
+ u32 sizeimage;
memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
@@ -325,9 +326,10 @@ venc_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
pixmp->num_planes = fmt->num_planes;
pixmp->flags = 0;
- pfmt[0].sizeimage = venus_helper_get_framesz(pixmp->pixelformat,
- pixmp->width,
- pixmp->height);
+ sizeimage = venus_helper_get_framesz(pixmp->pixelformat,
+ pixmp->width,
+ pixmp->height);
+ pfmt[0].sizeimage = max(ALIGN(pfmt[0].sizeimage, SZ_4K), sizeimage);
if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
pfmt[0].bytesperline = ALIGN(pixmp->width, 128);
@@ -399,8 +401,10 @@ static int venc_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
inst->fmt_out = fmt;
- else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
inst->fmt_cap = fmt;
+ inst->output_buf_size = pixmp->plane_fmt[0].sizeimage;
+ }
return 0;
}
@@ -918,6 +922,7 @@ static int venc_queue_setup(struct vb2_queue *q,
sizes[0] = venus_helper_get_framesz(inst->fmt_cap->pixfmt,
inst->width,
inst->height);
+ sizes[0] = max(sizes[0], inst->output_buf_size);
inst->output_buf_size = sizes[0];
break;
default:
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 02/11] venus: helpers: export few helper functions
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
2019-06-28 12:59 ` [PATCH v2 01/11] venus: venc: amend buffer size for bitstream plane Stanimir Varbanov
@ 2019-06-28 12:59 ` Stanimir Varbanov
2019-06-28 12:59 ` [PATCH v2 03/11] venus: hfi: add type argument to hfi flush function Stanimir Varbanov
` (9 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 12:59 UTC (permalink / raw)
To: linux-media
Cc: Mauro Carvalho Chehab, Hans Verkuil, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot,
Stanimir Varbanov
Here we export few helper function to use them from decoder to
implement more granular control needed for stateful Codec API
compliance.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/helpers.c | 29 ++++++++++++---------
drivers/media/platform/qcom/venus/helpers.h | 6 +++++
2 files changed, 23 insertions(+), 12 deletions(-)
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 71b06dfc6dc4..63af69acc068 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -79,7 +79,7 @@ bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt)
}
EXPORT_SYMBOL_GPL(venus_helper_check_codec);
-static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
+int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
{
struct intbuf *buf;
int ret = 0;
@@ -100,6 +100,7 @@ static int venus_helper_queue_dpb_bufs(struct venus_inst *inst)
fail:
return ret;
}
+EXPORT_SYMBOL_GPL(venus_helper_queue_dpb_bufs);
int venus_helper_free_dpb_bufs(struct venus_inst *inst)
{
@@ -278,7 +279,7 @@ static const unsigned int intbuf_types_4xx[] = {
HFI_BUFFER_INTERNAL_PERSIST_1,
};
-static int intbufs_alloc(struct venus_inst *inst)
+int venus_helper_intbufs_alloc(struct venus_inst *inst)
{
const unsigned int *intbuf;
size_t arr_sz, i;
@@ -304,11 +305,13 @@ static int intbufs_alloc(struct venus_inst *inst)
intbufs_unset_buffers(inst);
return ret;
}
+EXPORT_SYMBOL_GPL(venus_helper_intbufs_alloc);
-static int intbufs_free(struct venus_inst *inst)
+int venus_helper_intbufs_free(struct venus_inst *inst)
{
return intbufs_unset_buffers(inst);
}
+EXPORT_SYMBOL_GPL(venus_helper_intbufs_free);
static u32 load_per_instance(struct venus_inst *inst)
{
@@ -339,7 +342,7 @@ static u32 load_per_type(struct venus_core *core, u32 session_type)
return mbs_per_sec;
}
-static int load_scale_clocks(struct venus_core *core)
+int venus_helper_load_scale_clocks(struct venus_core *core)
{
const struct freq_tbl *table = core->res->freq_tbl;
unsigned int num_rows = core->res->freq_tbl_size;
@@ -388,6 +391,7 @@ static int load_scale_clocks(struct venus_core *core)
dev_err(dev, "failed to set clock rate %lu (%d)\n", freq, ret);
return ret;
}
+EXPORT_SYMBOL_GPL(venus_helper_load_scale_clocks);
static void fill_buffer_desc(const struct venus_buffer *buf,
struct hfi_buffer_desc *bd, bool response)
@@ -472,7 +476,7 @@ static bool is_dynamic_bufmode(struct venus_inst *inst)
return caps->cap_bufs_mode_dynamic;
}
-static int session_unregister_bufs(struct venus_inst *inst)
+int venus_helper_unregister_bufs(struct venus_inst *inst)
{
struct venus_buffer *buf, *n;
struct hfi_buffer_desc bd;
@@ -489,6 +493,7 @@ static int session_unregister_bufs(struct venus_inst *inst)
return ret;
}
+EXPORT_SYMBOL_GPL(venus_helper_unregister_bufs);
static int session_register_bufs(struct venus_inst *inst)
{
@@ -1009,8 +1014,8 @@ void venus_helper_vb2_stop_streaming(struct vb2_queue *q)
if (inst->streamon_out & inst->streamon_cap) {
ret = hfi_session_stop(inst);
ret |= hfi_session_unload_res(inst);
- ret |= session_unregister_bufs(inst);
- ret |= intbufs_free(inst);
+ ret |= venus_helper_unregister_bufs(inst);
+ ret |= venus_helper_intbufs_free(inst);
ret |= hfi_session_deinit(inst);
if (inst->session_error || core->sys_error)
@@ -1021,7 +1026,7 @@ void venus_helper_vb2_stop_streaming(struct vb2_queue *q)
venus_helper_free_dpb_bufs(inst);
- load_scale_clocks(core);
+ venus_helper_load_scale_clocks(core);
INIT_LIST_HEAD(&inst->registeredbufs);
}
@@ -1041,7 +1046,7 @@ int venus_helper_vb2_start_streaming(struct venus_inst *inst)
struct venus_core *core = inst->core;
int ret;
- ret = intbufs_alloc(inst);
+ ret = venus_helper_intbufs_alloc(inst);
if (ret)
return ret;
@@ -1049,7 +1054,7 @@ int venus_helper_vb2_start_streaming(struct venus_inst *inst)
if (ret)
goto err_bufs_free;
- load_scale_clocks(core);
+ venus_helper_load_scale_clocks(core);
ret = hfi_session_load_res(inst);
if (ret)
@@ -1070,9 +1075,9 @@ int venus_helper_vb2_start_streaming(struct venus_inst *inst)
err_unload_res:
hfi_session_unload_res(inst);
err_unreg_bufs:
- session_unregister_bufs(inst);
+ venus_helper_unregister_bufs(inst);
err_bufs_free:
- intbufs_free(inst);
+ venus_helper_intbufs_free(inst);
return ret;
}
EXPORT_SYMBOL_GPL(venus_helper_vb2_start_streaming);
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 153783687a0c..70288dc860ff 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -9,6 +9,7 @@
#include <media/videobuf2-v4l2.h>
struct venus_inst;
+struct venus_core;
bool venus_helper_check_codec(struct venus_inst *inst, u32 v4l2_pixfmt);
struct vb2_v4l2_buffer *venus_helper_find_buf(struct venus_inst *inst,
@@ -53,4 +54,9 @@ int venus_helper_alloc_dpb_bufs(struct venus_inst *inst);
int venus_helper_free_dpb_bufs(struct venus_inst *inst);
int venus_helper_power_enable(struct venus_core *core, u32 session_type,
bool enable);
+int venus_helper_intbufs_alloc(struct venus_inst *inst);
+int venus_helper_intbufs_free(struct venus_inst *inst);
+int venus_helper_queue_dpb_bufs(struct venus_inst *inst);
+int venus_helper_unregister_bufs(struct venus_inst *inst);
+int venus_helper_load_scale_clocks(struct venus_core *core);
#endif
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 03/11] venus: hfi: add type argument to hfi flush function
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
2019-06-28 12:59 ` [PATCH v2 01/11] venus: venc: amend buffer size for bitstream plane Stanimir Varbanov
2019-06-28 12:59 ` [PATCH v2 02/11] venus: helpers: export few helper functions Stanimir Varbanov
@ 2019-06-28 12:59 ` Stanimir Varbanov
2019-06-28 12:59 ` [PATCH v2 04/11] venus: hfi: export few HFI functions Stanimir Varbanov
` (8 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 12:59 UTC (permalink / raw)
To: linux-media
Cc: Mauro Carvalho Chehab, Hans Verkuil, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot,
Stanimir Varbanov
Make hfi_flush function to receive an argument for the type
of flush.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/hfi.c | 4 ++--
drivers/media/platform/qcom/venus/hfi.h | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
index 6ad0c1772ea7..7c5bf5cb32de 100644
--- a/drivers/media/platform/qcom/venus/hfi.c
+++ b/drivers/media/platform/qcom/venus/hfi.c
@@ -375,14 +375,14 @@ int hfi_session_unload_res(struct venus_inst *inst)
return 0;
}
-int hfi_session_flush(struct venus_inst *inst)
+int hfi_session_flush(struct venus_inst *inst, u32 type)
{
const struct hfi_ops *ops = inst->core->ops;
int ret;
reinit_completion(&inst->done);
- ret = ops->session_flush(inst, HFI_FLUSH_ALL);
+ ret = ops->session_flush(inst, type);
if (ret)
return ret;
diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/venus/hfi.h
index b121cb1427ac..855822c9f39b 100644
--- a/drivers/media/platform/qcom/venus/hfi.h
+++ b/drivers/media/platform/qcom/venus/hfi.h
@@ -161,7 +161,7 @@ int hfi_session_continue(struct venus_inst *inst);
int hfi_session_abort(struct venus_inst *inst);
int hfi_session_load_res(struct venus_inst *inst);
int hfi_session_unload_res(struct venus_inst *inst);
-int hfi_session_flush(struct venus_inst *inst);
+int hfi_session_flush(struct venus_inst *inst, u32 type);
int hfi_session_set_buffers(struct venus_inst *inst,
struct hfi_buffer_desc *bd);
int hfi_session_unset_buffers(struct venus_inst *inst,
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 04/11] venus: hfi: export few HFI functions
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
` (2 preceding siblings ...)
2019-06-28 12:59 ` [PATCH v2 03/11] venus: hfi: add type argument to hfi flush function Stanimir Varbanov
@ 2019-06-28 12:59 ` Stanimir Varbanov
2019-06-28 12:59 ` [PATCH v2 05/11] venus: hfi: return an error if session_init is already called Stanimir Varbanov
` (7 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 12:59 UTC (permalink / raw)
To: linux-media
Cc: Mauro Carvalho Chehab, Hans Verkuil, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot,
Stanimir Varbanov
Export few HFI functions to use them from decoder to implement
more granular control needed for stateful Codec API compliance.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/hfi.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
index 7c5bf5cb32de..82eb889ab541 100644
--- a/drivers/media/platform/qcom/venus/hfi.c
+++ b/drivers/media/platform/qcom/venus/hfi.c
@@ -276,6 +276,7 @@ int hfi_session_start(struct venus_inst *inst)
return 0;
}
+EXPORT_SYMBOL_GPL(hfi_session_start);
int hfi_session_stop(struct venus_inst *inst)
{
@@ -299,6 +300,7 @@ int hfi_session_stop(struct venus_inst *inst)
return 0;
}
+EXPORT_SYMBOL_GPL(hfi_session_stop);
int hfi_session_continue(struct venus_inst *inst)
{
@@ -328,6 +330,7 @@ int hfi_session_abort(struct venus_inst *inst)
return 0;
}
+EXPORT_SYMBOL_GPL(hfi_session_abort);
int hfi_session_load_res(struct venus_inst *inst)
{
@@ -374,6 +377,7 @@ int hfi_session_unload_res(struct venus_inst *inst)
return 0;
}
+EXPORT_SYMBOL_GPL(hfi_session_unload_res);
int hfi_session_flush(struct venus_inst *inst, u32 type)
{
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 05/11] venus: hfi: return an error if session_init is already called
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
` (3 preceding siblings ...)
2019-06-28 12:59 ` [PATCH v2 04/11] venus: hfi: export few HFI functions Stanimir Varbanov
@ 2019-06-28 12:59 ` Stanimir Varbanov
2019-06-28 12:59 ` [PATCH v2 06/11] venus: helpers: add three more helper functions Stanimir Varbanov
` (6 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 12:59 UTC (permalink / raw)
To: linux-media
Cc: Mauro Carvalho Chehab, Hans Verkuil, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot,
Stanimir Varbanov
This makes hfi_session_init to return an error when it is
already called without a call to hfi_session_deinit.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/hfi.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/venus/hfi.c
index 82eb889ab541..3d8b1284d1f3 100644
--- a/drivers/media/platform/qcom/venus/hfi.c
+++ b/drivers/media/platform/qcom/venus/hfi.c
@@ -198,6 +198,9 @@ int hfi_session_init(struct venus_inst *inst, u32 pixfmt)
const struct hfi_ops *ops = core->ops;
int ret;
+ if (inst->state != INST_UNINIT)
+ return -EINVAL;
+
inst->hfi_codec = to_codec_type(pixfmt);
reinit_completion(&inst->done);
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 06/11] venus: helpers: add three more helper functions
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
` (4 preceding siblings ...)
2019-06-28 12:59 ` [PATCH v2 05/11] venus: hfi: return an error if session_init is already called Stanimir Varbanov
@ 2019-06-28 12:59 ` Stanimir Varbanov
2019-06-28 12:59 ` [PATCH v2 07/11] venus: vdec_ctrls: get real minimum buffers for capture Stanimir Varbanov
` (5 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 12:59 UTC (permalink / raw)
To: linux-media
Cc: Mauro Carvalho Chehab, Hans Verkuil, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot,
Stanimir Varbanov
This adds three more helper functions:
* for internal buffers reallocation, applicable when we are doing
dynamic resolution change
* for initial buffer processing of capture and output queue buffer
types
All of them will be needed for stateful Codec API support.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/helpers.c | 82 +++++++++++++++++++++
drivers/media/platform/qcom/venus/helpers.h | 3 +
2 files changed, 85 insertions(+)
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 63af69acc068..ab20fd3b36d9 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -313,6 +313,52 @@ int venus_helper_intbufs_free(struct venus_inst *inst)
}
EXPORT_SYMBOL_GPL(venus_helper_intbufs_free);
+int venus_helper_intbufs_realloc(struct venus_inst *inst)
+{
+ enum hfi_version ver = inst->core->res->hfi_version;
+ struct hfi_buffer_desc bd;
+ struct intbuf *buf, *n;
+ int ret;
+
+ list_for_each_entry_safe(buf, n, &inst->internalbufs, list) {
+ if (buf->type == HFI_BUFFER_INTERNAL_PERSIST ||
+ buf->type == HFI_BUFFER_INTERNAL_PERSIST_1)
+ continue;
+
+ memset(&bd, 0, sizeof(bd));
+ bd.buffer_size = buf->size;
+ bd.buffer_type = buf->type;
+ bd.num_buffers = 1;
+ bd.device_addr = buf->da;
+ bd.response_required = true;
+
+ ret = hfi_session_unset_buffers(inst, &bd);
+
+ dma_free_attrs(inst->core->dev, buf->size, buf->va, buf->da,
+ buf->attrs);
+
+ list_del_init(&buf->list);
+ kfree(buf);
+ }
+
+ ret = intbufs_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH(ver));
+ if (ret)
+ goto err;
+
+ ret = intbufs_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH_1(ver));
+ if (ret)
+ goto err;
+
+ ret = intbufs_set_buffer(inst, HFI_BUFFER_INTERNAL_SCRATCH_2(ver));
+ if (ret)
+ goto err;
+
+ return 0;
+err:
+ return ret;
+}
+EXPORT_SYMBOL_GPL(venus_helper_intbufs_realloc);
+
static u32 load_per_instance(struct venus_inst *inst)
{
u32 mbs;
@@ -1041,6 +1087,42 @@ void venus_helper_vb2_stop_streaming(struct vb2_queue *q)
}
EXPORT_SYMBOL_GPL(venus_helper_vb2_stop_streaming);
+int venus_helper_process_initial_cap_bufs(struct venus_inst *inst)
+{
+ struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx;
+ struct v4l2_m2m_buffer *buf, *n;
+ int ret;
+
+ v4l2_m2m_for_each_dst_buf_safe(m2m_ctx, buf, n) {
+ ret = session_process_buf(inst, &buf->vb);
+ if (ret) {
+ return_buf_error(inst, &buf->vb);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(venus_helper_process_initial_cap_bufs);
+
+int venus_helper_process_initial_out_bufs(struct venus_inst *inst)
+{
+ struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx;
+ struct v4l2_m2m_buffer *buf, *n;
+ int ret;
+
+ v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buf, n) {
+ ret = session_process_buf(inst, &buf->vb);
+ if (ret) {
+ return_buf_error(inst, &buf->vb);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(venus_helper_process_initial_out_bufs);
+
int venus_helper_vb2_start_streaming(struct venus_inst *inst)
{
struct venus_core *core = inst->core;
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 70288dc860ff..486598d52811 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -56,7 +56,10 @@ int venus_helper_power_enable(struct venus_core *core, u32 session_type,
bool enable);
int venus_helper_intbufs_alloc(struct venus_inst *inst);
int venus_helper_intbufs_free(struct venus_inst *inst);
+int venus_helper_intbufs_realloc(struct venus_inst *inst);
int venus_helper_queue_dpb_bufs(struct venus_inst *inst);
int venus_helper_unregister_bufs(struct venus_inst *inst);
int venus_helper_load_scale_clocks(struct venus_core *core);
+int venus_helper_process_initial_cap_bufs(struct venus_inst *inst);
+int venus_helper_process_initial_out_bufs(struct venus_inst *inst);
#endif
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 07/11] venus: vdec_ctrls: get real minimum buffers for capture
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
` (5 preceding siblings ...)
2019-06-28 12:59 ` [PATCH v2 06/11] venus: helpers: add three more helper functions Stanimir Varbanov
@ 2019-06-28 12:59 ` Stanimir Varbanov
2019-06-28 12:59 ` [PATCH v2 08/11] venus: vdec: allow bigger sizeimage set by clients Stanimir Varbanov
` (4 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 12:59 UTC (permalink / raw)
To: linux-media
Cc: Mauro Carvalho Chehab, Hans Verkuil, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot,
Stanimir Varbanov
Until now we returned num_output_bufs set during reqbuf but
that could be wrong when we implement stateful Codec API. So
get the minimum buffers for capture from HFI. This is supposed
to be called after stream header parsing, i.e. after dequeue
v4l2 event for change resolution.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/vdec_ctrls.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/qcom/venus/vdec_ctrls.c b/drivers/media/platform/qcom/venus/vdec_ctrls.c
index 300350bfe8bd..3a963cbd342a 100644
--- a/drivers/media/platform/qcom/venus/vdec_ctrls.c
+++ b/drivers/media/platform/qcom/venus/vdec_ctrls.c
@@ -7,6 +7,7 @@
#include <media/v4l2-ctrls.h>
#include "core.h"
+#include "helpers.h"
#include "vdec.h"
static int vdec_op_s_ctrl(struct v4l2_ctrl *ctrl)
@@ -38,7 +39,9 @@ static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
{
struct venus_inst *inst = ctrl_to_inst(ctrl);
struct vdec_controls *ctr = &inst->controls.dec;
+ struct hfi_buffer_requirements bufreq;
union hfi_get_property hprop;
+ enum hfi_version ver = inst->core->res->hfi_version;
u32 ptype = HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT;
int ret;
@@ -62,7 +65,9 @@ static int vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
ctrl->val = ctr->post_loop_deb_mode;
break;
case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
- ctrl->val = inst->num_output_bufs;
+ ret = venus_helper_get_bufreq(inst, HFI_BUFFER_OUTPUT, &bufreq);
+ if (!ret)
+ ctrl->val = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
break;
default:
return -EINVAL;
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 08/11] venus: vdec: allow bigger sizeimage set by clients
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
` (6 preceding siblings ...)
2019-06-28 12:59 ` [PATCH v2 07/11] venus: vdec_ctrls: get real minimum buffers for capture Stanimir Varbanov
@ 2019-06-28 12:59 ` Stanimir Varbanov
2019-06-28 13:00 ` [PATCH v2 09/11] venus: make decoder compliant with stateful codec API Stanimir Varbanov
` (3 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 12:59 UTC (permalink / raw)
To: linux-media
Cc: Mauro Carvalho Chehab, Hans Verkuil, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot,
Stanimir Varbanov
In most of the cases the client will know better what could be
the maximum size for compressed data buffers. Change the driver
to permit the user to set bigger size for the compressed buffer
but make reasonable sanitation.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/vdec.c | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index e1f998656c07..fb9afee76793 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -133,6 +133,7 @@ vdec_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
const struct venus_format *fmt;
+ u32 szimage;
memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
@@ -161,14 +162,17 @@ vdec_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f)
pixmp->num_planes = fmt->num_planes;
pixmp->flags = 0;
- pfmt[0].sizeimage = venus_helper_get_framesz(pixmp->pixelformat,
- pixmp->width,
- pixmp->height);
+ szimage = venus_helper_get_framesz(pixmp->pixelformat, pixmp->width,
+ pixmp->height);
- if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ pfmt[0].sizeimage = szimage;
pfmt[0].bytesperline = ALIGN(pixmp->width, 128);
- else
+ } else {
+ pfmt[0].sizeimage = clamp_t(u32, pfmt[0].sizeimage, 0, SZ_8M);
+ pfmt[0].sizeimage = max(pfmt[0].sizeimage, szimage);
pfmt[0].bytesperline = 0;
+ }
return fmt;
}
@@ -266,6 +270,7 @@ static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
inst->ycbcr_enc = pixmp->ycbcr_enc;
inst->quantization = pixmp->quantization;
inst->xfer_func = pixmp->xfer_func;
+ inst->input_buf_size = pixmp->plane_fmt[0].sizeimage;
}
memset(&format, 0, sizeof(format));
@@ -728,6 +733,7 @@ static int vdec_queue_setup(struct vb2_queue *q,
sizes[0] = venus_helper_get_framesz(inst->fmt_out->pixfmt,
inst->out_width,
inst->out_height);
+ sizes[0] = max(sizes[0], inst->input_buf_size);
inst->input_buf_size = sizes[0];
*num_buffers = max(*num_buffers, in_num);
inst->num_input_bufs = *num_buffers;
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 09/11] venus: make decoder compliant with stateful codec API
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
` (7 preceding siblings ...)
2019-06-28 12:59 ` [PATCH v2 08/11] venus: vdec: allow bigger sizeimage set by clients Stanimir Varbanov
@ 2019-06-28 13:00 ` Stanimir Varbanov
2019-06-28 13:00 ` [PATCH v2 10/11] venus: helpers: handle correctly vbuf field Stanimir Varbanov
` (2 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 13:00 UTC (permalink / raw)
To: linux-media
Cc: Mauro Carvalho Chehab, Hans Verkuil, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot,
Stanimir Varbanov
This refactors code for start/stop streaming vb2 operations and
adds a state machine handling similar to the one in stateful codec
API documentation. One major change is that now the HFI session is
started on STREAMON(OUTPUT) and stopped on REQBUF(OUTPUT,count=0),
during that time STREAMOFF(CAP,OUT) just flush buffers but doesn't
stop the session. The other major change is that now the capture
and output queues are completely separated.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/core.h | 24 +-
drivers/media/platform/qcom/venus/helpers.c | 23 +-
drivers/media/platform/qcom/venus/helpers.h | 1 +
drivers/media/platform/qcom/venus/vdec.c | 519 +++++++++++++++-----
4 files changed, 435 insertions(+), 132 deletions(-)
diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 9ab95fd57760..139b5d786375 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -209,6 +209,18 @@ struct venus_buffer {
#define to_venus_buffer(ptr) container_of(ptr, struct venus_buffer, vb)
+enum venus_dec_state {
+ VENUS_DEC_STATE_DEINIT = 0,
+ VENUS_DEC_STATE_INIT = 1,
+ VENUS_DEC_STATE_CAPTURE_SETUP = 2,
+ VENUS_DEC_STATE_STOPPED = 3,
+ VENUS_DEC_STATE_SEEK = 4,
+ VENUS_DEC_STATE_DRAIN = 5,
+ VENUS_DEC_STATE_DECODING = 6,
+ VENUS_DEC_STATE_DRC = 7
+};
+
+
/**
* struct venus_inst - holds per instance parameters
*
@@ -232,6 +244,10 @@ struct venus_buffer {
* @colorspace: current color space
* @quantization: current quantization
* @xfer_func: current xfer function
+ * @codec_state: current codec API state (see DEC/ENC_STATE_)
+ * @reconf_wait: wait queue for resolution change event
+ * @subscriptions: used to hold current events subscriptions
+ * @buf_count: used to count number of buffers (reqbuf(0))
* @fps: holds current FPS
* @timeperframe: holds current time per frame structure
* @fmt_out: a reference to output format structure
@@ -246,8 +262,6 @@ struct venus_buffer {
* @opb_buftype: output picture buffer type
* @opb_fmt: output picture buffer raw format
* @reconfig: a flag raised by decoder when the stream resolution changed
- * @reconfig_width: holds the new width
- * @reconfig_height: holds the new height
* @hfi_codec: current codec for this instance in HFI space
* @sequence_cap: a sequence counter for capture queue
* @sequence_out: a sequence counter for output queue
@@ -287,6 +301,10 @@ struct venus_inst {
u8 ycbcr_enc;
u8 quantization;
u8 xfer_func;
+ enum venus_dec_state codec_state;
+ wait_queue_head_t reconf_wait;
+ unsigned int subscriptions;
+ int buf_count;
u64 fps;
struct v4l2_fract timeperframe;
const struct venus_format *fmt_out;
@@ -301,8 +319,6 @@ struct venus_inst {
u32 opb_buftype;
u32 opb_fmt;
bool reconfig;
- u32 reconfig_width;
- u32 reconfig_height;
u32 hfi_codec;
u32 sequence_cap;
u32 sequence_out;
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index ab20fd3b36d9..176facdd415d 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -1021,16 +1021,19 @@ void venus_helper_vb2_buf_queue(struct vb2_buffer *vb)
v4l2_m2m_buf_queue(m2m_ctx, vbuf);
- if (!(inst->streamon_out & inst->streamon_cap))
+ if (inst->session_type == VIDC_SESSION_TYPE_ENC &&
+ !(inst->streamon_out && inst->streamon_cap))
goto unlock;
- ret = is_buf_refed(inst, vbuf);
- if (ret)
- goto unlock;
+ if (vb2_start_streaming_called(vb->vb2_queue)) {
+ ret = is_buf_refed(inst, vbuf);
+ if (ret)
+ goto unlock;
- ret = session_process_buf(inst, vbuf);
- if (ret)
- return_buf_error(inst, vbuf);
+ ret = session_process_buf(inst, vbuf);
+ if (ret)
+ return_buf_error(inst, vbuf);
+ }
unlock:
mutex_unlock(&inst->lock);
@@ -1146,14 +1149,8 @@ int venus_helper_vb2_start_streaming(struct venus_inst *inst)
if (ret)
goto err_unload_res;
- ret = venus_helper_queue_dpb_bufs(inst);
- if (ret)
- goto err_session_stop;
-
return 0;
-err_session_stop:
- hfi_session_stop(inst);
err_unload_res:
hfi_session_unload_res(inst);
err_unreg_bufs:
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index 486598d52811..f2e111555bd9 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -8,6 +8,7 @@
#include <media/videobuf2-v4l2.h>
+
struct venus_inst;
struct venus_core;
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index fb9afee76793..336d49132d19 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -186,33 +186,56 @@ static int vdec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
return 0;
}
+static int vdec_check_src_change(struct venus_inst *inst)
+{
+ int ret;
+
+ if (inst->subscriptions & V4L2_EVENT_SOURCE_CHANGE &&
+ inst->codec_state == VENUS_DEC_STATE_INIT &&
+ !inst->reconfig)
+ return -EINVAL;
+
+ if (inst->subscriptions & V4L2_EVENT_SOURCE_CHANGE)
+ return 0;
+
+ /*
+ * The code snippet below is a workaround for backward compatibility
+ * with applications which doesn't support V4L2 events. It will be
+ * dropped in future once those applications are fixed.
+ */
+
+ if (inst->codec_state != VENUS_DEC_STATE_INIT)
+ goto done;
+
+ ret = wait_event_timeout(inst->reconf_wait, inst->reconfig,
+ msecs_to_jiffies(100));
+ if (!ret)
+ return -EINVAL;
+
+ if (!(inst->codec_state == VENUS_DEC_STATE_CAPTURE_SETUP) ||
+ !inst->reconfig)
+ dev_dbg(inst->core->dev, "%s: wrong state\n", __func__);
+
+done:
+ return 0;
+}
+
static int vdec_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
{
struct venus_inst *inst = to_inst(file);
const struct venus_format *fmt = NULL;
struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
+ int ret;
if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
fmt = inst->fmt_cap;
else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
fmt = inst->fmt_out;
- if (inst->reconfig) {
- struct v4l2_format format = {};
-
- inst->out_width = inst->reconfig_width;
- inst->out_height = inst->reconfig_height;
- inst->reconfig = false;
-
- format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
- format.fmt.pix_mp.pixelformat = inst->fmt_cap->pixfmt;
- format.fmt.pix_mp.width = inst->out_width;
- format.fmt.pix_mp.height = inst->out_height;
-
- vdec_try_fmt_common(inst, &format);
-
- inst->width = format.fmt.pix_mp.width;
- inst->height = format.fmt.pix_mp.height;
+ if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+ ret = vdec_check_src_change(inst);
+ if (ret)
+ return ret;
}
pixmp->pixelformat = fmt->pixfmt;
@@ -427,11 +450,18 @@ static int vdec_enum_framesizes(struct file *file, void *fh,
static int vdec_subscribe_event(struct v4l2_fh *fh,
const struct v4l2_event_subscription *sub)
{
+ struct venus_inst *inst = container_of(fh, struct venus_inst, fh);
+ int ret;
+
switch (sub->type) {
case V4L2_EVENT_EOS:
return v4l2_event_subscribe(fh, sub, 2, NULL);
case V4L2_EVENT_SOURCE_CHANGE:
- return v4l2_src_change_event_subscribe(fh, sub);
+ ret = v4l2_src_change_event_subscribe(fh, sub);
+ if (ret)
+ return ret;
+ inst->subscriptions |= V4L2_EVENT_SOURCE_CHANGE;
+ return 0;
case V4L2_EVENT_CTRL:
return v4l2_ctrl_subscribe_event(fh, sub);
default:
@@ -439,21 +469,6 @@ static int vdec_subscribe_event(struct v4l2_fh *fh,
}
}
-static int
-vdec_try_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *cmd)
-{
- switch (cmd->cmd) {
- case V4L2_DEC_CMD_STOP:
- if (cmd->flags & V4L2_DEC_CMD_STOP_TO_BLACK)
- return -EINVAL;
- break;
- default:
- return -EINVAL;
- }
-
- return 0;
-}
-
static int
vdec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *cmd)
{
@@ -461,24 +476,29 @@ vdec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *cmd)
struct hfi_frame_data fdata = {0};
int ret;
- ret = vdec_try_decoder_cmd(file, fh, cmd);
+ ret = v4l2_m2m_ioctl_try_decoder_cmd(file, fh, cmd);
if (ret)
return ret;
mutex_lock(&inst->lock);
- /*
- * Implement V4L2_DEC_CMD_STOP by enqueue an empty buffer on decoder
- * input to signal EOS.
- */
- if (!(inst->streamon_out & inst->streamon_cap))
- goto unlock;
+ if (cmd->cmd == V4L2_DEC_CMD_STOP) {
+ /*
+ * Implement V4L2_DEC_CMD_STOP by enqueue an empty buffer on
+ * decoder input to signal EOS.
+ */
+ if (!(inst->streamon_out && inst->streamon_cap))
+ goto unlock;
- fdata.buffer_type = HFI_BUFFER_INPUT;
- fdata.flags |= HFI_BUFFERFLAG_EOS;
- fdata.device_addr = 0xdeadbeef;
+ fdata.buffer_type = HFI_BUFFER_INPUT;
+ fdata.flags |= HFI_BUFFERFLAG_EOS;
+ fdata.device_addr = 0xdeadb000;
- ret = hfi_session_process_buf(inst, &fdata);
+ ret = hfi_session_process_buf(inst, &fdata);
+
+ if (!ret && inst->codec_state == VENUS_DEC_STATE_DECODING)
+ inst->codec_state = VENUS_DEC_STATE_DRAIN;
+ }
unlock:
mutex_unlock(&inst->lock);
@@ -509,7 +529,7 @@ static const struct v4l2_ioctl_ops vdec_ioctl_ops = {
.vidioc_enum_framesizes = vdec_enum_framesizes,
.vidioc_subscribe_event = vdec_subscribe_event,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
- .vidioc_try_decoder_cmd = vdec_try_decoder_cmd,
+ .vidioc_try_decoder_cmd = v4l2_m2m_ioctl_try_decoder_cmd,
.vidioc_decoder_cmd = vdec_decoder_cmd,
};
@@ -639,20 +659,18 @@ static int vdec_output_conf(struct venus_inst *inst)
return 0;
}
-static int vdec_init_session(struct venus_inst *inst)
+static int vdec_session_init(struct venus_inst *inst)
{
int ret;
ret = hfi_session_init(inst, inst->fmt_out->pixfmt);
- if (ret)
+ if (ret == -EINVAL)
+ return 0;
+ else if (ret)
return ret;
- ret = venus_helper_set_input_resolution(inst, inst->out_width,
- inst->out_height);
- if (ret)
- goto deinit;
-
- ret = venus_helper_set_color_format(inst, inst->fmt_cap->pixfmt);
+ ret = venus_helper_set_input_resolution(inst, frame_width_min(inst),
+ frame_height_min(inst));
if (ret)
goto deinit;
@@ -671,26 +689,19 @@ static int vdec_num_buffers(struct venus_inst *inst, unsigned int *in_num,
*in_num = *out_num = 0;
- ret = vdec_init_session(inst);
- if (ret)
- return ret;
-
ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq);
if (ret)
- goto deinit;
+ return ret;
*in_num = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
ret = venus_helper_get_bufreq(inst, HFI_BUFFER_OUTPUT, &bufreq);
if (ret)
- goto deinit;
+ return ret;
*out_num = HFI_BUFREQ_COUNT_MIN(&bufreq, ver);
-deinit:
- hfi_session_deinit(inst);
-
- return ret;
+ return 0;
}
static int vdec_queue_setup(struct vb2_queue *q,
@@ -723,6 +734,10 @@ static int vdec_queue_setup(struct vb2_queue *q,
return 0;
}
+ ret = vdec_session_init(inst);
+ if (ret)
+ return ret;
+
ret = vdec_num_buffers(inst, &in_num, &out_num);
if (ret)
return ret;
@@ -747,6 +762,11 @@ static int vdec_queue_setup(struct vb2_queue *q,
inst->output_buf_size = sizes[0];
*num_buffers = max(*num_buffers, out_num);
inst->num_output_bufs = *num_buffers;
+
+ mutex_lock(&inst->lock);
+ if (inst->codec_state == VENUS_DEC_STATE_CAPTURE_SETUP)
+ inst->codec_state = VENUS_DEC_STATE_STOPPED;
+ mutex_unlock(&inst->lock);
break;
default:
ret = -EINVAL;
@@ -783,80 +803,295 @@ static int vdec_verify_conf(struct venus_inst *inst)
return 0;
}
-static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
+static int vdec_start_capture(struct venus_inst *inst)
{
- struct venus_inst *inst = vb2_get_drv_priv(q);
int ret;
- mutex_lock(&inst->lock);
+ if (!inst->streamon_out)
+ return 0;
- if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
- inst->streamon_out = 1;
- else
- inst->streamon_cap = 1;
+ if (inst->codec_state == VENUS_DEC_STATE_DECODING) {
+ if (inst->reconfig)
+ goto reconfigure;
- if (!(inst->streamon_out & inst->streamon_cap)) {
- mutex_unlock(&inst->lock);
+ venus_helper_queue_dpb_bufs(inst);
+ venus_helper_process_initial_cap_bufs(inst);
+ inst->streamon_cap = 1;
return 0;
}
- venus_helper_init_instance(inst);
+ if (inst->codec_state != VENUS_DEC_STATE_STOPPED)
+ return 0;
- inst->reconfig = false;
- inst->sequence_cap = 0;
- inst->sequence_out = 0;
+reconfigure:
+ ret = hfi_session_flush(inst, HFI_FLUSH_OUTPUT);
+ if (ret)
+ return ret;
+
+ ret = vdec_output_conf(inst);
+ if (ret)
+ return ret;
+
+ ret = venus_helper_set_num_bufs(inst, inst->num_input_bufs,
+ VB2_MAX_FRAME, VB2_MAX_FRAME);
+ if (ret)
+ return ret;
+
+ ret = venus_helper_intbufs_realloc(inst);
+ if (ret)
+ goto err;
+
+ ret = venus_helper_alloc_dpb_bufs(inst);
+ if (ret)
+ goto err;
+
+ ret = venus_helper_queue_dpb_bufs(inst);
+ if (ret)
+ goto free_dpb_bufs;
- ret = vdec_init_session(inst);
+ ret = venus_helper_process_initial_cap_bufs(inst);
if (ret)
- goto bufs_done;
+ goto free_dpb_bufs;
+
+ venus_helper_load_scale_clocks(inst->core);
+
+ ret = hfi_session_continue(inst);
+ if (ret)
+ goto free_dpb_bufs;
+
+ inst->codec_state = VENUS_DEC_STATE_DECODING;
+
+ inst->streamon_cap = 1;
+ inst->sequence_cap = 0;
+ inst->reconfig = false;
+
+ return 0;
+
+free_dpb_bufs:
+ venus_helper_free_dpb_bufs(inst);
+err:
+ return ret;
+}
+
+static int vdec_start_output(struct venus_inst *inst)
+{
+ int ret;
+
+ if (inst->codec_state == VENUS_DEC_STATE_SEEK) {
+ ret = venus_helper_process_initial_out_bufs(inst);
+ inst->codec_state = VENUS_DEC_STATE_DECODING;
+ goto done;
+ }
+
+ if (inst->codec_state == VENUS_DEC_STATE_INIT ||
+ inst->codec_state == VENUS_DEC_STATE_CAPTURE_SETUP) {
+ ret = venus_helper_process_initial_out_bufs(inst);
+ goto done;
+ }
+
+ if (inst->codec_state != VENUS_DEC_STATE_DEINIT)
+ return -EINVAL;
+
+ venus_helper_init_instance(inst);
+ inst->sequence_out = 0;
+ inst->reconfig = false;
ret = vdec_set_properties(inst);
if (ret)
- goto deinit_sess;
+ return ret;
ret = vdec_output_conf(inst);
if (ret)
- goto deinit_sess;
+ return ret;
ret = vdec_verify_conf(inst);
if (ret)
- goto deinit_sess;
+ return ret;
ret = venus_helper_set_num_bufs(inst, inst->num_input_bufs,
VB2_MAX_FRAME, VB2_MAX_FRAME);
if (ret)
- goto deinit_sess;
+ return ret;
- ret = venus_helper_alloc_dpb_bufs(inst);
+ ret = venus_helper_vb2_start_streaming(inst);
+ if (ret)
+ return ret;
+
+ ret = venus_helper_process_initial_out_bufs(inst);
if (ret)
- goto deinit_sess;
+ return ret;
+
+ inst->codec_state = VENUS_DEC_STATE_INIT;
+
+done:
+ inst->streamon_out = 1;
+ return ret;
+}
+
+static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct venus_inst *inst = vb2_get_drv_priv(q);
+ int ret;
+
+ mutex_lock(&inst->lock);
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ ret = vdec_start_capture(inst);
+ else
+ ret = vdec_start_output(inst);
- ret = venus_helper_vb2_start_streaming(inst);
if (ret)
- goto deinit_sess;
+ goto error;
mutex_unlock(&inst->lock);
-
return 0;
-deinit_sess:
- hfi_session_deinit(inst);
-bufs_done:
+error:
venus_helper_buffers_done(inst, VB2_BUF_STATE_QUEUED);
+ mutex_unlock(&inst->lock);
+ return ret;
+}
+
+static void vdec_cancel_dst_buffers(struct venus_inst *inst)
+{
+ struct vb2_v4l2_buffer *buf;
+
+ while ((buf = v4l2_m2m_dst_buf_remove(inst->m2m_ctx)))
+ v4l2_m2m_buf_done(buf, VB2_BUF_STATE_ERROR);
+}
+
+static int vdec_stop_capture(struct venus_inst *inst)
+{
+ int ret = 0;
+
+ switch (inst->codec_state) {
+ case VENUS_DEC_STATE_DECODING:
+ ret = hfi_session_flush(inst, HFI_FLUSH_ALL);
+ /* fallthrough */
+ case VENUS_DEC_STATE_DRAIN:
+ vdec_cancel_dst_buffers(inst);
+ inst->codec_state = VENUS_DEC_STATE_STOPPED;
+ break;
+ case VENUS_DEC_STATE_DRC:
+ ret = hfi_session_flush(inst, HFI_FLUSH_OUTPUT);
+ vdec_cancel_dst_buffers(inst);
+ inst->codec_state = VENUS_DEC_STATE_CAPTURE_SETUP;
+ INIT_LIST_HEAD(&inst->registeredbufs);
+ venus_helper_free_dpb_bufs(inst);
+ break;
+ default:
+ return 0;
+ }
+
+ return ret;
+}
+
+static int vdec_stop_output(struct venus_inst *inst)
+{
+ int ret = 0;
+
+ switch (inst->codec_state) {
+ case VENUS_DEC_STATE_DECODING:
+ case VENUS_DEC_STATE_DRAIN:
+ case VENUS_DEC_STATE_STOPPED:
+ ret = hfi_session_flush(inst, HFI_FLUSH_ALL);
+ inst->codec_state = VENUS_DEC_STATE_SEEK;
+ break;
+ case VENUS_DEC_STATE_INIT:
+ case VENUS_DEC_STATE_CAPTURE_SETUP:
+ ret = hfi_session_flush(inst, HFI_FLUSH_INPUT);
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+static void vdec_stop_streaming(struct vb2_queue *q)
+{
+ struct venus_inst *inst = vb2_get_drv_priv(q);
+ int ret = -EINVAL;
+
+ mutex_lock(&inst->lock);
+
+ if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ ret = vdec_stop_capture(inst);
+ else
+ ret = vdec_stop_output(inst);
+
+ venus_helper_buffers_done(inst, VB2_BUF_STATE_ERROR);
+
+ if (ret)
+ goto unlock;
+
if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
inst->streamon_out = 0;
else
inst->streamon_cap = 0;
+
+unlock:
+ mutex_unlock(&inst->lock);
+}
+
+static void vdec_session_release(struct venus_inst *inst)
+{
+ struct venus_core *core = inst->core;
+ int ret, abort = 0;
+
+ mutex_lock(&inst->lock);
+
+ inst->codec_state = VENUS_DEC_STATE_DEINIT;
+
+ ret = hfi_session_stop(inst);
+ abort = (ret && ret != -EINVAL) ? 1 : 0;
+ ret = hfi_session_unload_res(inst);
+ abort = (ret && ret != -EINVAL) ? 1 : 0;
+ ret = venus_helper_unregister_bufs(inst);
+ abort = (ret && ret != -EINVAL) ? 1 : 0;
+ ret = venus_helper_intbufs_free(inst);
+ abort = (ret && ret != -EINVAL) ? 1 : 0;
+ ret = hfi_session_deinit(inst);
+ abort = (ret && ret != -EINVAL) ? 1 : 0;
+
+ if (inst->session_error || core->sys_error)
+ abort = 1;
+
+ if (abort)
+ hfi_session_abort(inst);
+
+ venus_helper_free_dpb_bufs(inst);
+ venus_helper_load_scale_clocks(core);
+ INIT_LIST_HEAD(&inst->registeredbufs);
+
mutex_unlock(&inst->lock);
- return ret;
+}
+
+static int vdec_buf_init(struct vb2_buffer *vb)
+{
+ struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+
+ inst->buf_count++;
+
+ return venus_helper_vb2_buf_init(vb);
+}
+
+static void vdec_buf_cleanup(struct vb2_buffer *vb)
+{
+ struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
+
+ inst->buf_count--;
+ if (!inst->buf_count)
+ vdec_session_release(inst);
}
static const struct vb2_ops vdec_vb2_ops = {
.queue_setup = vdec_queue_setup,
- .buf_init = venus_helper_vb2_buf_init,
+ .buf_init = vdec_buf_init,
+ .buf_cleanup = vdec_buf_cleanup,
.buf_prepare = venus_helper_vb2_buf_prepare,
.start_streaming = vdec_start_streaming,
- .stop_streaming = venus_helper_vb2_stop_streaming,
+ .stop_streaming = vdec_stop_streaming,
.buf_queue = venus_helper_vb2_buf_queue,
};
@@ -880,9 +1115,9 @@ static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type,
vbuf->flags = flags;
vbuf->field = V4L2_FIELD_NONE;
+ vb = &vbuf->vb2_buf;
if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
- vb = &vbuf->vb2_buf;
vb2_set_plane_payload(vb, 0, bytesused);
vb->planes[0].data_offset = data_offset;
vb->timestamp = timestamp_us * NSEC_PER_USEC;
@@ -892,6 +1127,9 @@ static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type,
const struct v4l2_event ev = { .type = V4L2_EVENT_EOS };
v4l2_event_queue_fh(&inst->fh, &ev);
+
+ if (inst->codec_state == VENUS_DEC_STATE_DRAIN)
+ inst->codec_state = VENUS_DEC_STATE_STOPPED;
}
} else {
vbuf->sequence = inst->sequence_out++;
@@ -903,17 +1141,69 @@ static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type,
if (hfi_flags & HFI_BUFFERFLAG_DATACORRUPT)
state = VB2_BUF_STATE_ERROR;
+ if (hfi_flags & HFI_BUFFERFLAG_DROP_FRAME) {
+ state = VB2_BUF_STATE_ERROR;
+ vb2_set_plane_payload(vb, 0, 0);
+ vb->timestamp = 0;
+ }
+
v4l2_m2m_buf_done(vbuf, state);
}
+static void vdec_event_change(struct venus_inst *inst,
+ struct hfi_event_data *ev_data, bool sufficient)
+{
+ static const struct v4l2_event ev = {
+ .type = V4L2_EVENT_SOURCE_CHANGE,
+ .u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION };
+ struct device *dev = inst->core->dev_dec;
+ struct v4l2_format format = {};
+
+ mutex_lock(&inst->lock);
+
+ format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ format.fmt.pix_mp.pixelformat = inst->fmt_cap->pixfmt;
+ format.fmt.pix_mp.width = ev_data->width;
+ format.fmt.pix_mp.height = ev_data->height;
+
+ vdec_try_fmt_common(inst, &format);
+
+ inst->width = format.fmt.pix_mp.width;
+ inst->height = format.fmt.pix_mp.height;
+
+ inst->out_width = ev_data->width;
+ inst->out_height = ev_data->height;
+
+ dev_dbg(dev, "event %s sufficient resources (%ux%u)\n",
+ sufficient ? "" : "not", ev_data->width, ev_data->height);
+
+ if (sufficient) {
+ hfi_session_continue(inst);
+ } else {
+ switch (inst->codec_state) {
+ case VENUS_DEC_STATE_INIT:
+ inst->codec_state = VENUS_DEC_STATE_CAPTURE_SETUP;
+ break;
+ case VENUS_DEC_STATE_DECODING:
+ inst->codec_state = VENUS_DEC_STATE_DRC;
+ break;
+ default:
+ break;
+ }
+ }
+
+ inst->reconfig = true;
+ v4l2_event_queue_fh(&inst->fh, &ev);
+ wake_up(&inst->reconf_wait);
+
+ mutex_unlock(&inst->lock);
+}
+
static void vdec_event_notify(struct venus_inst *inst, u32 event,
struct hfi_event_data *data)
{
struct venus_core *core = inst->core;
struct device *dev = core->dev_dec;
- static const struct v4l2_event ev = {
- .type = V4L2_EVENT_SOURCE_CHANGE,
- .u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION };
switch (event) {
case EVT_SESSION_ERROR:
@@ -923,18 +1213,10 @@ static void vdec_event_notify(struct venus_inst *inst, u32 event,
case EVT_SYS_EVENT_CHANGE:
switch (data->event_type) {
case HFI_EVENT_DATA_SEQUENCE_CHANGED_SUFFICIENT_BUF_RESOURCES:
- hfi_session_continue(inst);
- dev_dbg(dev, "event sufficient resources\n");
+ vdec_event_change(inst, data, true);
break;
case HFI_EVENT_DATA_SEQUENCE_CHANGED_INSUFFICIENT_BUF_RESOURCES:
- inst->reconfig_height = data->height;
- inst->reconfig_width = data->width;
- inst->reconfig = true;
-
- v4l2_event_queue_fh(&inst->fh, &ev);
-
- dev_dbg(dev, "event not sufficient resources (%ux%u)\n",
- data->width, data->height);
+ vdec_event_change(inst, data, false);
break;
case HFI_EVENT_RELEASE_BUFFER_REFERENCE:
venus_helper_release_buf_ref(inst, data->tag);
@@ -955,20 +1237,25 @@ static const struct hfi_inst_ops vdec_hfi_ops = {
static void vdec_inst_init(struct venus_inst *inst)
{
+ inst->hfi_codec = HFI_VIDEO_CODEC_H264;
inst->fmt_out = &vdec_formats[6];
inst->fmt_cap = &vdec_formats[0];
- inst->width = 1280;
- inst->height = ALIGN(720, 32);
- inst->out_width = 1280;
- inst->out_height = 720;
+ inst->width = frame_width_min(inst);
+ inst->height = ALIGN(frame_height_min(inst), 32);
+ inst->out_width = frame_width_min(inst);
+ inst->out_height = frame_height_min(inst);
inst->fps = 30;
inst->timeperframe.numerator = 1;
inst->timeperframe.denominator = 30;
- inst->hfi_codec = HFI_VIDEO_CODEC_H264;
+ inst->opb_buftype = HFI_BUFFER_OUTPUT;
+}
+
+static void vdec_m2m_device_run(void *priv)
+{
}
static const struct v4l2_m2m_ops vdec_m2m_ops = {
- .device_run = venus_helper_m2m_device_run,
+ .device_run = vdec_m2m_device_run,
.job_abort = venus_helper_m2m_job_abort,
};
@@ -986,7 +1273,7 @@ static int m2m_queue_init(void *priv, struct vb2_queue *src_vq,
src_vq->drv_priv = inst;
src_vq->buf_struct_size = sizeof(struct venus_buffer);
src_vq->allow_zero_bytesused = 1;
- src_vq->min_buffers_needed = 1;
+ src_vq->min_buffers_needed = 0;
src_vq->dev = inst->core->dev;
ret = vb2_queue_init(src_vq);
if (ret)
@@ -1000,7 +1287,7 @@ static int m2m_queue_init(void *priv, struct vb2_queue *src_vq,
dst_vq->drv_priv = inst;
dst_vq->buf_struct_size = sizeof(struct venus_buffer);
dst_vq->allow_zero_bytesused = 1;
- dst_vq->min_buffers_needed = 1;
+ dst_vq->min_buffers_needed = 0;
dst_vq->dev = inst->core->dev;
ret = vb2_queue_init(dst_vq);
if (ret) {
@@ -1030,7 +1317,9 @@ static int vdec_open(struct file *file)
inst->core = core;
inst->session_type = VIDC_SESSION_TYPE_DEC;
inst->num_output_bufs = 1;
-
+ inst->codec_state = VENUS_DEC_STATE_DEINIT;
+ inst->buf_count = 0;
+ init_waitqueue_head(&inst->reconf_wait);
venus_helper_init_instance(inst);
ret = pm_runtime_get_sync(core->dev_dec);
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 10/11] venus: helpers: handle correctly vbuf field
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
` (8 preceding siblings ...)
2019-06-28 13:00 ` [PATCH v2 09/11] venus: make decoder compliant with stateful codec API Stanimir Varbanov
@ 2019-06-28 13:00 ` Stanimir Varbanov
2019-06-28 13:00 ` [PATCH v2 11/11] venus: dec: populate properly timestamps and flags for capture buffers Stanimir Varbanov
2019-06-28 13:37 ` [PATCH v2 00/11] Venus stateful Codec API Hans Verkuil
11 siblings, 0 replies; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 13:00 UTC (permalink / raw)
To: linux-media
Cc: Mauro Carvalho Chehab, Hans Verkuil, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot,
Stanimir Varbanov
Correct handling of OUTPUT buffers field and make v4l2-compliance
happy.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/helpers.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index 176facdd415d..c948c4e809b5 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -998,6 +998,17 @@ int venus_helper_vb2_buf_prepare(struct vb2_buffer *vb)
{
struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue);
unsigned int out_buf_size = venus_helper_get_opb_size(inst);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+ if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+ if (vbuf->field == V4L2_FIELD_ANY)
+ vbuf->field = V4L2_FIELD_NONE;
+ if (vbuf->field != V4L2_FIELD_NONE) {
+ dev_err(inst->core->dev, "%s field isn't supported\n",
+ __func__);
+ return -EINVAL;
+ }
+ }
if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
vb2_plane_size(vb, 0) < out_buf_size)
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* [PATCH v2 11/11] venus: dec: populate properly timestamps and flags for capture buffers
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
` (9 preceding siblings ...)
2019-06-28 13:00 ` [PATCH v2 10/11] venus: helpers: handle correctly vbuf field Stanimir Varbanov
@ 2019-06-28 13:00 ` Stanimir Varbanov
2019-06-28 13:37 ` [PATCH v2 00/11] Venus stateful Codec API Hans Verkuil
11 siblings, 0 replies; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 13:00 UTC (permalink / raw)
To: linux-media
Cc: Mauro Carvalho Chehab, Hans Verkuil, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot,
Stanimir Varbanov
Cache flags, timestamps and timecode structure of OUTPUT buffers
in per-instance structure array and fill correctly the same when
the CAPTURE buffers are done.
This will make v4l2-compliance decoder streaming test happy.
Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
---
drivers/media/platform/qcom/venus/core.h | 8 +++
drivers/media/platform/qcom/venus/helpers.c | 54 +++++++++++++++++++++
drivers/media/platform/qcom/venus/helpers.h | 2 +
drivers/media/platform/qcom/venus/vdec.c | 2 +
4 files changed, 66 insertions(+)
diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h
index 139b5d786375..5c6d4145138d 100644
--- a/drivers/media/platform/qcom/venus/core.h
+++ b/drivers/media/platform/qcom/venus/core.h
@@ -220,6 +220,13 @@ enum venus_dec_state {
VENUS_DEC_STATE_DRC = 7
};
+struct venus_ts_metadata {
+ bool used;
+ u64 ts_ns;
+ u64 ts_us;
+ u32 flags;
+ struct v4l2_timecode tc;
+};
/**
* struct venus_inst - holds per instance parameters
@@ -305,6 +312,7 @@ struct venus_inst {
wait_queue_head_t reconf_wait;
unsigned int subscriptions;
int buf_count;
+ struct venus_ts_metadata tss[VIDEO_MAX_FRAME];
u64 fps;
struct v4l2_fract timeperframe;
const struct venus_format *fmt_out;
diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c
index c948c4e809b5..b42a1fce273a 100644
--- a/drivers/media/platform/qcom/venus/helpers.c
+++ b/drivers/media/platform/qcom/venus/helpers.c
@@ -463,6 +463,57 @@ static void return_buf_error(struct venus_inst *inst,
v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
}
+static void
+put_ts_metadata(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf)
+{
+ struct vb2_buffer *vb = &vbuf->vb2_buf;
+ unsigned int i;
+ int slot = -1;
+ u64 ts_us = vb->timestamp;
+
+ for (i = 0; i < ARRAY_SIZE(inst->tss); i++) {
+ if (!inst->tss[i].used) {
+ slot = i;
+ break;
+ }
+ }
+
+ if (slot == -1) {
+ dev_dbg(inst->core->dev, "%s: no free slot\n", __func__);
+ return;
+ }
+
+ do_div(ts_us, NSEC_PER_USEC);
+
+ inst->tss[slot].used = true;
+ inst->tss[slot].flags = vbuf->flags;
+ inst->tss[slot].tc = vbuf->timecode;
+ inst->tss[slot].ts_us = ts_us;
+ inst->tss[slot].ts_ns = vb->timestamp;
+}
+
+void venus_helper_get_ts_metadata(struct venus_inst *inst, u64 timestamp_us,
+ struct vb2_v4l2_buffer *vbuf)
+{
+ struct vb2_buffer *vb = &vbuf->vb2_buf;
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(inst->tss); ++i) {
+ if (!inst->tss[i].used)
+ continue;
+
+ if (inst->tss[i].ts_us != timestamp_us)
+ continue;
+
+ inst->tss[i].used = false;
+ vbuf->flags |= inst->tss[i].flags;
+ vbuf->timecode = inst->tss[i].tc;
+ vb->timestamp = inst->tss[i].ts_ns;
+ break;
+ }
+}
+EXPORT_SYMBOL(venus_helper_get_ts_metadata);
+
static int
session_process_buf(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf)
{
@@ -487,6 +538,9 @@ session_process_buf(struct venus_inst *inst, struct vb2_v4l2_buffer *vbuf)
if (vbuf->flags & V4L2_BUF_FLAG_LAST || !fdata.filled_len)
fdata.flags |= HFI_BUFFERFLAG_EOS;
+
+ if (inst->session_type == VIDC_SESSION_TYPE_DEC)
+ put_ts_metadata(inst, vbuf);
} else if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
if (inst->session_type == VIDC_SESSION_TYPE_ENC)
fdata.buffer_type = HFI_BUFFER_OUTPUT;
diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h
index f2e111555bd9..791c35c64ca3 100644
--- a/drivers/media/platform/qcom/venus/helpers.h
+++ b/drivers/media/platform/qcom/venus/helpers.h
@@ -63,4 +63,6 @@ int venus_helper_unregister_bufs(struct venus_inst *inst);
int venus_helper_load_scale_clocks(struct venus_core *core);
int venus_helper_process_initial_cap_bufs(struct venus_inst *inst);
int venus_helper_process_initial_out_bufs(struct venus_inst *inst);
+void venus_helper_get_ts_metadata(struct venus_inst *inst, u64 timestamp_us,
+ struct vb2_v4l2_buffer *vbuf);
#endif
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 336d49132d19..0b7d65db5cdc 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -1135,6 +1135,8 @@ static void vdec_buf_done(struct venus_inst *inst, unsigned int buf_type,
vbuf->sequence = inst->sequence_out++;
}
+ venus_helper_get_ts_metadata(inst, timestamp_us, vbuf);
+
if (hfi_flags & HFI_BUFFERFLAG_READONLY)
venus_helper_acquire_buf_ref(vbuf);
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* Re: [PATCH v2 00/11] Venus stateful Codec API
2019-06-28 12:59 [PATCH v2 00/11] Venus stateful Codec API Stanimir Varbanov
` (10 preceding siblings ...)
2019-06-28 13:00 ` [PATCH v2 11/11] venus: dec: populate properly timestamps and flags for capture buffers Stanimir Varbanov
@ 2019-06-28 13:37 ` Hans Verkuil
2019-06-28 14:23 ` Stanimir Varbanov
11 siblings, 1 reply; 17+ messages in thread
From: Hans Verkuil @ 2019-06-28 13:37 UTC (permalink / raw)
To: Stanimir Varbanov, linux-media
Cc: Mauro Carvalho Chehab, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot
On 6/28/19 2:59 PM, Stanimir Varbanov wrote:
> Hello,
>
> Here is v2 of the Venus transition to stateful codec API
> compliance. The v2 can be found at [1].
>
> Changes since v1:
> * codec_state is now enum
> * dropped IS_OUT and IS_CAP macros and use vb2_start_streaming_called()
> * corrected g_fmt and reconfig logic
> * s/vdec_dst_buffers_done/vdec_cancel_dst_buffers
> * use v4l2_m2m_ioctl_try_decoder_cmd M2M helper
> * various fixes to make v4l2-compliance pass the streaming test
>
> To test the streaming with --stream-from-hdr v4l2-compliance option I have
> to make the following hack (it is needed because the size of decoder input
> buffers (OUTPUT queue) is not enough for the h264 bitstream, i.e the driver
> default resolution is 64x64 but the h264 stream is 320x240):
>
> diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp
> index c71dcf65b721..dc0fcf20d3e4 100644
> --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp
> +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp
> @@ -1294,6 +1294,11 @@ int testMmap(struct node *node, unsigned frame_count, enum poll_mode pollmode)
> fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p);
> }
> fail_on_test(q.create_bufs(node, 1, &fmt));
> +
> + for (unsigned p = 0; p < fmt.g_num_planes(); p++)
> + fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p);
> + node->s_fmt(fmt);
> +
> fail_on_test(q.reqbufs(node, 2));
> }
> if (v4l_type_is_output(type))
Does the venus driver set sizeimage based on the given output resolution?
E.g. if v4l2-compliance would first set the output resolution to 320x240,
is the returned sizeimage value OK in that case?
And this also means that the venus driver requires each buffer to have
a single compressed frame, right? I.e. it can't be spread over multiple
OUTPUT buffers.
We really need to let userspace know about such restrictions.
Stanimir, can you list the restrictions of the decoder for the various
codecs?
Regards,
Hans
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [PATCH v2 00/11] Venus stateful Codec API
2019-06-28 13:37 ` [PATCH v2 00/11] Venus stateful Codec API Hans Verkuil
@ 2019-06-28 14:23 ` Stanimir Varbanov
2019-06-28 14:25 ` Hans Verkuil
0 siblings, 1 reply; 17+ messages in thread
From: Stanimir Varbanov @ 2019-06-28 14:23 UTC (permalink / raw)
To: Hans Verkuil, linux-media
Cc: Mauro Carvalho Chehab, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot
Hi Hans,
On 6/28/19 4:37 PM, Hans Verkuil wrote:
> On 6/28/19 2:59 PM, Stanimir Varbanov wrote:
>> Hello,
>>
>> Here is v2 of the Venus transition to stateful codec API
>> compliance. The v2 can be found at [1].
>>
>> Changes since v1:
>> * codec_state is now enum
>> * dropped IS_OUT and IS_CAP macros and use vb2_start_streaming_called()
>> * corrected g_fmt and reconfig logic
>> * s/vdec_dst_buffers_done/vdec_cancel_dst_buffers
>> * use v4l2_m2m_ioctl_try_decoder_cmd M2M helper
>> * various fixes to make v4l2-compliance pass the streaming test
>>
>> To test the streaming with --stream-from-hdr v4l2-compliance option I have
>> to make the following hack (it is needed because the size of decoder input
>> buffers (OUTPUT queue) is not enough for the h264 bitstream, i.e the driver
>> default resolution is 64x64 but the h264 stream is 320x240):
>>
>> diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp
>> index c71dcf65b721..dc0fcf20d3e4 100644
>> --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp
>> +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp
>> @@ -1294,6 +1294,11 @@ int testMmap(struct node *node, unsigned frame_count, enum poll_mode pollmode)
>> fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p);
>> }
>> fail_on_test(q.create_bufs(node, 1, &fmt));
>> +
>> + for (unsigned p = 0; p < fmt.g_num_planes(); p++)
>> + fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p);
>> + node->s_fmt(fmt);
>> +
>> fail_on_test(q.reqbufs(node, 2));
>> }
>> if (v4l_type_is_output(type))
>
> Does the venus driver set sizeimage based on the given output resolution?
Yes.
>
> E.g. if v4l2-compliance would first set the output resolution to 320x240,
> is the returned sizeimage value OK in that case?
Yes.
Here are few options to me:
- set the correct resolution
- set 0x0 and sizeimage at some arbitrary value (1 or 2MB). Despite if
the bitstream is 4K it will not be enough if the bitrate is huge.
- invent some mechanism to trigger reconfiguration on the OUTPUT queue
as well (similar to the CAPTURE queue)
>
> And this also means that the venus driver requires each buffer to have
> a single compressed frame, right? I.e. it can't be spread over multiple
> OUTPUT buffers.
I cannot say for sure but that is how all downstream cases uses it i.e.
one compressed frame per input buffer. I wonder if you fill input
decoder buffer with many compressed frames in one input decoder buffer
how you pass the timestamp for every packet?
>
> We really need to let userspace know about such restrictions.
>
> Stanimir, can you list the restrictions of the decoder for the various
> codecs?
What you mean? Restrictions like "one compressed frame per input buffer"?
--
regards,
Stan
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 00/11] Venus stateful Codec API
2019-06-28 14:23 ` Stanimir Varbanov
@ 2019-06-28 14:25 ` Hans Verkuil
2019-06-28 14:31 ` Nicolas Dufresne
0 siblings, 1 reply; 17+ messages in thread
From: Hans Verkuil @ 2019-06-28 14:25 UTC (permalink / raw)
To: Stanimir Varbanov, linux-media
Cc: Mauro Carvalho Chehab, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot
On 6/28/19 4:23 PM, Stanimir Varbanov wrote:
> Hi Hans,
>
> On 6/28/19 4:37 PM, Hans Verkuil wrote:
>> On 6/28/19 2:59 PM, Stanimir Varbanov wrote:
>>> Hello,
>>>
>>> Here is v2 of the Venus transition to stateful codec API
>>> compliance. The v2 can be found at [1].
>>>
>>> Changes since v1:
>>> * codec_state is now enum
>>> * dropped IS_OUT and IS_CAP macros and use vb2_start_streaming_called()
>>> * corrected g_fmt and reconfig logic
>>> * s/vdec_dst_buffers_done/vdec_cancel_dst_buffers
>>> * use v4l2_m2m_ioctl_try_decoder_cmd M2M helper
>>> * various fixes to make v4l2-compliance pass the streaming test
>>>
>>> To test the streaming with --stream-from-hdr v4l2-compliance option I have
>>> to make the following hack (it is needed because the size of decoder input
>>> buffers (OUTPUT queue) is not enough for the h264 bitstream, i.e the driver
>>> default resolution is 64x64 but the h264 stream is 320x240):
>>>
>>> diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp
>>> index c71dcf65b721..dc0fcf20d3e4 100644
>>> --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp
>>> +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp
>>> @@ -1294,6 +1294,11 @@ int testMmap(struct node *node, unsigned frame_count, enum poll_mode pollmode)
>>> fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p);
>>> }
>>> fail_on_test(q.create_bufs(node, 1, &fmt));
>>> +
>>> + for (unsigned p = 0; p < fmt.g_num_planes(); p++)
>>> + fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p);
>>> + node->s_fmt(fmt);
>>> +
>>> fail_on_test(q.reqbufs(node, 2));
>>> }
>>> if (v4l_type_is_output(type))
>>
>> Does the venus driver set sizeimage based on the given output resolution?
>
> Yes.
>
>>
>> E.g. if v4l2-compliance would first set the output resolution to 320x240,
>> is the returned sizeimage value OK in that case?
>
> Yes.
>
> Here are few options to me:
> - set the correct resolution
> - set 0x0 and sizeimage at some arbitrary value (1 or 2MB). Despite if
> the bitstream is 4K it will not be enough if the bitrate is huge.
> - invent some mechanism to trigger reconfiguration on the OUTPUT queue
> as well (similar to the CAPTURE queue)
>
>>
>> And this also means that the venus driver requires each buffer to have
>> a single compressed frame, right? I.e. it can't be spread over multiple
>> OUTPUT buffers.
>
> I cannot say for sure but that is how all downstream cases uses it i.e.
> one compressed frame per input buffer. I wonder if you fill input
> decoder buffer with many compressed frames in one input decoder buffer
> how you pass the timestamp for every packet?
>
>>
>> We really need to let userspace know about such restrictions.
>>
>> Stanimir, can you list the restrictions of the decoder for the various
>> codecs?
>
> What you mean? Restrictions like "one compressed frame per input buffer"?
>
Yes :-)
Regards,
Hans
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 00/11] Venus stateful Codec API
2019-06-28 14:25 ` Hans Verkuil
@ 2019-06-28 14:31 ` Nicolas Dufresne
2019-06-28 14:32 ` Nicolas Dufresne
0 siblings, 1 reply; 17+ messages in thread
From: Nicolas Dufresne @ 2019-06-28 14:31 UTC (permalink / raw)
To: Hans Verkuil, Stanimir Varbanov, linux-media
Cc: Mauro Carvalho Chehab, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot
[-- Attachment #1: Type: text/plain, Size: 3837 bytes --]
Le vendredi 28 juin 2019 à 16:25 +0200, Hans Verkuil a écrit :
> On 6/28/19 4:23 PM, Stanimir Varbanov wrote:
> > Hi Hans,
> >
> > On 6/28/19 4:37 PM, Hans Verkuil wrote:
> > > On 6/28/19 2:59 PM, Stanimir Varbanov wrote:
> > > > Hello,
> > > >
> > > > Here is v2 of the Venus transition to stateful codec API
> > > > compliance. The v2 can be found at [1].
> > > >
> > > > Changes since v1:
> > > > * codec_state is now enum
> > > > * dropped IS_OUT and IS_CAP macros and use vb2_start_streaming_called()
> > > > * corrected g_fmt and reconfig logic
> > > > * s/vdec_dst_buffers_done/vdec_cancel_dst_buffers
> > > > * use v4l2_m2m_ioctl_try_decoder_cmd M2M helper
> > > > * various fixes to make v4l2-compliance pass the streaming test
> > > >
> > > > To test the streaming with --stream-from-hdr v4l2-compliance option I have
> > > > to make the following hack (it is needed because the size of decoder input
> > > > buffers (OUTPUT queue) is not enough for the h264 bitstream, i.e the driver
> > > > default resolution is 64x64 but the h264 stream is 320x240):
> > > >
> > > > diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp
> > > > index c71dcf65b721..dc0fcf20d3e4 100644
> > > > --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp
> > > > +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp
> > > > @@ -1294,6 +1294,11 @@ int testMmap(struct node *node, unsigned frame_count, enum poll_mode pollmode)
> > > > fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p);
> > > > }
> > > > fail_on_test(q.create_bufs(node, 1, &fmt));
> > > > +
> > > > + for (unsigned p = 0; p < fmt.g_num_planes(); p++)
> > > > + fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p);
> > > > + node->s_fmt(fmt);
> > > > +
> > > > fail_on_test(q.reqbufs(node, 2));
> > > > }
> > > > if (v4l_type_is_output(type))
> > >
> > > Does the venus driver set sizeimage based on the given output resolution?
> >
> > Yes.
> >
> > > E.g. if v4l2-compliance would first set the output resolution to 320x240,
> > > is the returned sizeimage value OK in that case?
> >
> > Yes.
> >
> > Here are few options to me:
> > - set the correct resolution
> > - set 0x0 and sizeimage at some arbitrary value (1 or 2MB). Despite if
> > the bitstream is 4K it will not be enough if the bitrate is huge.
> > - invent some mechanism to trigger reconfiguration on the OUTPUT queue
> > as well (similar to the CAPTURE queue)
> >
> > > And this also means that the venus driver requires each buffer to have
> > > a single compressed frame, right? I.e. it can't be spread over multiple
> > > OUTPUT buffers.
> >
> > I cannot say for sure but that is how all downstream cases uses it i.e.
> > one compressed frame per input buffer. I wonder if you fill input
> > decoder buffer with many compressed frames in one input decoder buffer
> > how you pass the timestamp for every packet?
> >
> > > We really need to let userspace know about such restrictions.
> > >
> > > Stanimir, can you list the restrictions of the decoder for the various
> > > codecs?
> >
> > What you mean? Restrictions like "one compressed frame per input buffer"?
> >
>
> Yes :-)
I think I just had the same discussions through some RPi patches
reviews. All the stateless codec drivers we have so far assumes full
frames and some wording iirc specified this in the spec at some point.
That removes ambiguity for timestamps application. Even though,
timestamps in V4L2 are useless for B-Frame enabled streams (ffmpeg and
GStreamer will ignore them).
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v2 00/11] Venus stateful Codec API
2019-06-28 14:31 ` Nicolas Dufresne
@ 2019-06-28 14:32 ` Nicolas Dufresne
0 siblings, 0 replies; 17+ messages in thread
From: Nicolas Dufresne @ 2019-06-28 14:32 UTC (permalink / raw)
To: Hans Verkuil, Stanimir Varbanov, linux-media
Cc: Mauro Carvalho Chehab, linux-kernel, linux-arm-msm,
Vikash Garodia, Tomasz Figa, Alexandre Courbot
[-- Attachment #1: Type: text/plain, Size: 4151 bytes --]
Le vendredi 28 juin 2019 à 10:31 -0400, Nicolas Dufresne a écrit :
> Le vendredi 28 juin 2019 à 16:25 +0200, Hans Verkuil a écrit :
> > On 6/28/19 4:23 PM, Stanimir Varbanov wrote:
> > > Hi Hans,
> > >
> > > On 6/28/19 4:37 PM, Hans Verkuil wrote:
> > > > On 6/28/19 2:59 PM, Stanimir Varbanov wrote:
> > > > > Hello,
> > > > >
> > > > > Here is v2 of the Venus transition to stateful codec API
> > > > > compliance. The v2 can be found at [1].
> > > > >
> > > > > Changes since v1:
> > > > > * codec_state is now enum
> > > > > * dropped IS_OUT and IS_CAP macros and use vb2_start_streaming_called()
> > > > > * corrected g_fmt and reconfig logic
> > > > > * s/vdec_dst_buffers_done/vdec_cancel_dst_buffers
> > > > > * use v4l2_m2m_ioctl_try_decoder_cmd M2M helper
> > > > > * various fixes to make v4l2-compliance pass the streaming test
> > > > >
> > > > > To test the streaming with --stream-from-hdr v4l2-compliance option I have
> > > > > to make the following hack (it is needed because the size of decoder input
> > > > > buffers (OUTPUT queue) is not enough for the h264 bitstream, i.e the driver
> > > > > default resolution is 64x64 but the h264 stream is 320x240):
> > > > >
> > > > > diff --git a/utils/v4l2-compliance/v4l2-test-buffers.cpp b/utils/v4l2-compliance/v4l2-test-buffers.cpp
> > > > > index c71dcf65b721..dc0fcf20d3e4 100644
> > > > > --- a/utils/v4l2-compliance/v4l2-test-buffers.cpp
> > > > > +++ b/utils/v4l2-compliance/v4l2-test-buffers.cpp
> > > > > @@ -1294,6 +1294,11 @@ int testMmap(struct node *node, unsigned frame_count, enum poll_mode pollmode)
> > > > > fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p);
> > > > > }
> > > > > fail_on_test(q.create_bufs(node, 1, &fmt));
> > > > > +
> > > > > + for (unsigned p = 0; p < fmt.g_num_planes(); p++)
> > > > > + fmt.s_sizeimage(fmt.g_sizeimage(p) * 2, p);
> > > > > + node->s_fmt(fmt);
> > > > > +
> > > > > fail_on_test(q.reqbufs(node, 2));
> > > > > }
> > > > > if (v4l_type_is_output(type))
> > > >
> > > > Does the venus driver set sizeimage based on the given output resolution?
> > >
> > > Yes.
> > >
> > > > E.g. if v4l2-compliance would first set the output resolution to 320x240,
> > > > is the returned sizeimage value OK in that case?
> > >
> > > Yes.
> > >
> > > Here are few options to me:
> > > - set the correct resolution
> > > - set 0x0 and sizeimage at some arbitrary value (1 or 2MB). Despite if
> > > the bitstream is 4K it will not be enough if the bitrate is huge.
> > > - invent some mechanism to trigger reconfiguration on the OUTPUT queue
> > > as well (similar to the CAPTURE queue)
> > >
> > > > And this also means that the venus driver requires each buffer to have
> > > > a single compressed frame, right? I.e. it can't be spread over multiple
> > > > OUTPUT buffers.
> > >
> > > I cannot say for sure but that is how all downstream cases uses it i.e.
> > > one compressed frame per input buffer. I wonder if you fill input
> > > decoder buffer with many compressed frames in one input decoder buffer
> > > how you pass the timestamp for every packet?
> > >
> > > > We really need to let userspace know about such restrictions.
> > > >
> > > > Stanimir, can you list the restrictions of the decoder for the various
> > > > codecs?
> > >
> > > What you mean? Restrictions like "one compressed frame per input buffer"?
> > >
> >
> > Yes :-)
>
> I think I just had the same discussions through some RPi patches
> reviews. All the stateless codec drivers we have so far assumes full
> frames and some wording iirc specified this in the spec at some point.
> That removes ambiguity for timestamps application. Even though,
> timestamps in V4L2 are useless for B-Frame enabled streams (ffmpeg and
> GStreamer will ignore them).
Sorry, I mean they are not used as timestamp, only for frame matching.
>
> Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread