* [PATCH v3 00/11] Add DELETE_BUF ioctl
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Unlike when resolution change on keyframes, dynamic resolution change
on inter frames doesn't allow to do a stream off/on sequence because
it is need to keep all previous references alive to decode inter frames.
This constraint have two main problems:
- more memory consumption.
- more buffers in use.
To solve these issue this series introduce DELETE_BUF ioctl and remove
the 32 buffers limit per queue.
VP9 conformance tests using fluster give a score of 210/305.
The 25 resize inter tests (vp90-2-21-resize_inter_* files) are ok
but require to use postprocessor.
Kernel branch is available here:
https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/remove_vb2_queue_limit_v3
GStreamer branch to use DELETE_BUF ioctl and testing dynamic resolution
change is here:
https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer/-/commits/VP9_drc
changes in version 3:
- Use Xarray API to store allocated video buffers.
- No module parameter to limit the number of buffer per queue.
- Use Xarray inside Verisilicon driver to store postprocessor buffers
and remove VB2_MAX_FRAME limit.
- Allow Versilicon driver to change of resolution while streaming
- Various fixes the Verisilicon VP9 code to improve fluster score.
changes in version 2:
- Use a dynamic array and not a list to keep trace of allocated buffers.
Not use IDR interface because it is marked as deprecated in kernel
documentation.
- Add a module parameter to limit the number of buffer per queue.
- Add DELETE_BUF ioctl and m2m helpers.
Benjamin Gaignard (11):
media: videobuf2: Access vb2_queue bufs array through helper functions
media: videobuf2: Use Xarray instead of static buffers array
media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
media: videobuf2: Stop define VB2_MAX_FRAME as global
media: verisilicon: Refactor postprocessor to store more buffers
media: verisilicon: Store chroma and motion vectors offset
media: verisilicon: vp9: Use destination buffer height to compute
chroma offset
media: verisilicon: postproc: Fix down scale test
media: verisilicon: vp9: Allow to change resolution while streaming
media: v4l2: Add DELETE_BUF ioctl
media: v4l2: Add mem2mem helpers for DELETE_BUF ioctl
.../userspace-api/media/v4l/user-func.rst | 1 +
.../media/v4l/vidioc-delete-buf.rst | 51 ++++
.../media/common/videobuf2/videobuf2-core.c | 275 ++++++++++++++----
.../media/common/videobuf2/videobuf2-v4l2.c | 34 ++-
drivers/media/platform/amphion/vdec.c | 1 +
drivers/media/platform/amphion/vpu_dbg.c | 22 +-
.../platform/mediatek/jpeg/mtk_jpeg_core.c | 6 +-
.../vcodec/vdec/vdec_vp9_req_lat_if.c | 4 +-
drivers/media/platform/qcom/venus/hfi.h | 2 +
drivers/media/platform/st/sti/hva/hva-v4l2.c | 4 +
drivers/media/platform/verisilicon/hantro.h | 8 +-
.../platform/verisilicon/hantro_g2_vp9_dec.c | 10 +-
.../media/platform/verisilicon/hantro_hw.h | 4 +-
.../platform/verisilicon/hantro_postproc.c | 114 +++++---
.../media/platform/verisilicon/hantro_v4l2.c | 37 +--
drivers/media/test-drivers/vim2m.c | 1 +
drivers/media/test-drivers/visl/visl-dec.c | 28 +-
drivers/media/v4l2-core/v4l2-dev.c | 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 10 +
drivers/media/v4l2-core/v4l2-mem2mem.c | 20 ++
.../staging/media/atomisp/pci/atomisp_ioctl.c | 2 +-
drivers/staging/media/ipu3/ipu3-v4l2.c | 2 +
include/media/v4l2-ioctl.h | 4 +
include/media/v4l2-mem2mem.h | 12 +
include/media/videobuf2-core.h | 16 +-
include/media/videobuf2-v4l2.h | 15 +-
include/uapi/linux/videodev2.h | 2 +
27 files changed, 523 insertions(+), 163 deletions(-)
create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* [PATCH v3 00/11] Add DELETE_BUF ioctl
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Unlike when resolution change on keyframes, dynamic resolution change
on inter frames doesn't allow to do a stream off/on sequence because
it is need to keep all previous references alive to decode inter frames.
This constraint have two main problems:
- more memory consumption.
- more buffers in use.
To solve these issue this series introduce DELETE_BUF ioctl and remove
the 32 buffers limit per queue.
VP9 conformance tests using fluster give a score of 210/305.
The 25 resize inter tests (vp90-2-21-resize_inter_* files) are ok
but require to use postprocessor.
Kernel branch is available here:
https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/remove_vb2_queue_limit_v3
GStreamer branch to use DELETE_BUF ioctl and testing dynamic resolution
change is here:
https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer/-/commits/VP9_drc
changes in version 3:
- Use Xarray API to store allocated video buffers.
- No module parameter to limit the number of buffer per queue.
- Use Xarray inside Verisilicon driver to store postprocessor buffers
and remove VB2_MAX_FRAME limit.
- Allow Versilicon driver to change of resolution while streaming
- Various fixes the Verisilicon VP9 code to improve fluster score.
changes in version 2:
- Use a dynamic array and not a list to keep trace of allocated buffers.
Not use IDR interface because it is marked as deprecated in kernel
documentation.
- Add a module parameter to limit the number of buffer per queue.
- Add DELETE_BUF ioctl and m2m helpers.
Benjamin Gaignard (11):
media: videobuf2: Access vb2_queue bufs array through helper functions
media: videobuf2: Use Xarray instead of static buffers array
media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
media: videobuf2: Stop define VB2_MAX_FRAME as global
media: verisilicon: Refactor postprocessor to store more buffers
media: verisilicon: Store chroma and motion vectors offset
media: verisilicon: vp9: Use destination buffer height to compute
chroma offset
media: verisilicon: postproc: Fix down scale test
media: verisilicon: vp9: Allow to change resolution while streaming
media: v4l2: Add DELETE_BUF ioctl
media: v4l2: Add mem2mem helpers for DELETE_BUF ioctl
.../userspace-api/media/v4l/user-func.rst | 1 +
.../media/v4l/vidioc-delete-buf.rst | 51 ++++
.../media/common/videobuf2/videobuf2-core.c | 275 ++++++++++++++----
.../media/common/videobuf2/videobuf2-v4l2.c | 34 ++-
drivers/media/platform/amphion/vdec.c | 1 +
drivers/media/platform/amphion/vpu_dbg.c | 22 +-
.../platform/mediatek/jpeg/mtk_jpeg_core.c | 6 +-
.../vcodec/vdec/vdec_vp9_req_lat_if.c | 4 +-
drivers/media/platform/qcom/venus/hfi.h | 2 +
drivers/media/platform/st/sti/hva/hva-v4l2.c | 4 +
drivers/media/platform/verisilicon/hantro.h | 8 +-
.../platform/verisilicon/hantro_g2_vp9_dec.c | 10 +-
.../media/platform/verisilicon/hantro_hw.h | 4 +-
.../platform/verisilicon/hantro_postproc.c | 114 +++++---
.../media/platform/verisilicon/hantro_v4l2.c | 37 +--
drivers/media/test-drivers/vim2m.c | 1 +
drivers/media/test-drivers/visl/visl-dec.c | 28 +-
drivers/media/v4l2-core/v4l2-dev.c | 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 10 +
drivers/media/v4l2-core/v4l2-mem2mem.c | 20 ++
.../staging/media/atomisp/pci/atomisp_ioctl.c | 2 +-
drivers/staging/media/ipu3/ipu3-v4l2.c | 2 +
include/media/v4l2-ioctl.h | 4 +
include/media/v4l2-mem2mem.h | 12 +
include/media/videobuf2-core.h | 16 +-
include/media/videobuf2-v4l2.h | 15 +-
include/uapi/linux/videodev2.h | 2 +
27 files changed, 523 insertions(+), 163 deletions(-)
create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
The first step before changing how vb2 buffers are stored into queue
is to avoid direct access to bufs arrays.
This patch adds 2 helpers functions to add and remove vb2 buffers
from a queue. With these 2 and vb2_get_buffer(), bufs field of
struct vb2_queue becomes like a private member of the structure.
After each call to vb2_get_buffer() we need to be sure that we get
a valid pointer so check the return value of all of them.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/common/videobuf2/videobuf2-core.c | 215 ++++++++++++++----
.../media/common/videobuf2/videobuf2-v4l2.c | 28 ++-
drivers/media/platform/amphion/vpu_dbg.c | 22 +-
.../platform/mediatek/jpeg/mtk_jpeg_core.c | 6 +-
.../vcodec/vdec/vdec_vp9_req_lat_if.c | 2 +-
drivers/media/platform/st/sti/hva/hva-v4l2.c | 4 +
drivers/media/test-drivers/visl/visl-dec.c | 28 ++-
.../staging/media/atomisp/pci/atomisp_ioctl.c | 2 +-
8 files changed, 240 insertions(+), 67 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index cf6727d9c81f..42fd3984c2bc 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -359,8 +359,16 @@ static void __setup_offsets(struct vb2_buffer *vb)
unsigned long off = 0;
if (vb->index) {
- struct vb2_buffer *prev = q->bufs[vb->index - 1];
- struct vb2_plane *p = &prev->planes[prev->num_planes - 1];
+ struct vb2_buffer *prev;
+ struct vb2_plane *p;
+
+ prev = vb2_get_buffer(q, vb->index - 1);
+ if (!prev) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return;
+ }
+
+ p = &prev->planes[prev->num_planes - 1];
off = PAGE_ALIGN(p->m.offset + p->length);
}
@@ -397,6 +405,37 @@ static void init_buffer_cache_hints(struct vb2_queue *q, struct vb2_buffer *vb)
vb->skip_cache_sync_on_finish = 1;
}
+/**
+ * vb2_queue_add_buffer() - add a buffer to a queue
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @vb: pointer to &struct vb2_buffer to be added to the queue.
+ * @index: index where add vb2_buffer in the queue
+ */
+static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb, int index)
+{
+ if (index < VB2_MAX_FRAME && !q->bufs[index]) {
+ q->bufs[index] = vb;
+ vb->index = index;
+ vb->vb2_queue = q;
+ return true;
+ }
+
+ return false;
+}
+
+/**
+ * vb2_queue_remove_buffer() - remove a buffer from a queue
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @vb: pointer to &struct vb2_buffer to be removed from the queue.
+ */
+static void vb2_queue_remove_buffer(struct vb2_queue *q, struct vb2_buffer *vb)
+{
+ if (vb->index < VB2_MAX_FRAME) {
+ q->bufs[vb->index] = NULL;
+ vb->vb2_queue = NULL;
+ }
+}
+
/*
* __vb2_queue_alloc() - allocate vb2 buffer structures and (for MMAP type)
* video buffer memory for all buffers/planes on the queue and initializes the
@@ -425,9 +464,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
}
vb->state = VB2_BUF_STATE_DEQUEUED;
- vb->vb2_queue = q;
vb->num_planes = num_planes;
- vb->index = q->num_buffers + buffer;
vb->type = q->type;
vb->memory = memory;
init_buffer_cache_hints(q, vb);
@@ -437,7 +474,11 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
}
call_void_bufop(q, init_buffer, vb);
- q->bufs[vb->index] = vb;
+ if (!vb2_queue_add_buffer(q, vb, q->num_buffers + buffer)) {
+ dprintk(q, 1, "failed adding buffer %d to queue\n", buffer);
+ kfree(vb);
+ break;
+ }
/* Allocate video buffer memory for the MMAP type */
if (memory == VB2_MEMORY_MMAP) {
@@ -445,7 +486,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
if (ret) {
dprintk(q, 1, "failed allocating memory for buffer %d\n",
buffer);
- q->bufs[vb->index] = NULL;
+ vb2_queue_remove_buffer(q, vb);
kfree(vb);
break;
}
@@ -460,7 +501,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
dprintk(q, 1, "buffer %d %p initialization failed\n",
buffer, vb);
__vb2_buf_mem_free(vb);
- q->bufs[vb->index] = NULL;
+ vb2_queue_remove_buffer(q, vb);
kfree(vb);
break;
}
@@ -483,7 +524,7 @@ static void __vb2_free_mem(struct vb2_queue *q, unsigned int buffers)
for (buffer = q->num_buffers - buffers; buffer < q->num_buffers;
++buffer) {
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
if (!vb)
continue;
@@ -511,7 +552,7 @@ static void __vb2_queue_free(struct vb2_queue *q, unsigned int buffers)
/* Call driver-provided cleanup function for each buffer, if provided */
for (buffer = q->num_buffers - buffers; buffer < q->num_buffers;
++buffer) {
- struct vb2_buffer *vb = q->bufs[buffer];
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
if (vb && vb->planes[0].mem_priv)
call_void_vb_qop(vb, buf_cleanup, vb);
@@ -551,15 +592,20 @@ static void __vb2_queue_free(struct vb2_queue *q, unsigned int buffers)
q->cnt_unprepare_streaming = 0;
}
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
- struct vb2_buffer *vb = q->bufs[buffer];
- bool unbalanced = vb->cnt_mem_alloc != vb->cnt_mem_put ||
- vb->cnt_mem_prepare != vb->cnt_mem_finish ||
- vb->cnt_mem_get_userptr != vb->cnt_mem_put_userptr ||
- vb->cnt_mem_attach_dmabuf != vb->cnt_mem_detach_dmabuf ||
- vb->cnt_mem_map_dmabuf != vb->cnt_mem_unmap_dmabuf ||
- vb->cnt_buf_queue != vb->cnt_buf_done ||
- vb->cnt_buf_prepare != vb->cnt_buf_finish ||
- vb->cnt_buf_init != vb->cnt_buf_cleanup;
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
+ bool unbalanced;
+
+ if (!vb)
+ continue;
+
+ unbalanced = vb->cnt_mem_alloc != vb->cnt_mem_put ||
+ vb->cnt_mem_prepare != vb->cnt_mem_finish ||
+ vb->cnt_mem_get_userptr != vb->cnt_mem_put_userptr ||
+ vb->cnt_mem_attach_dmabuf != vb->cnt_mem_detach_dmabuf ||
+ vb->cnt_mem_map_dmabuf != vb->cnt_mem_unmap_dmabuf ||
+ vb->cnt_buf_queue != vb->cnt_buf_done ||
+ vb->cnt_buf_prepare != vb->cnt_buf_finish ||
+ vb->cnt_buf_init != vb->cnt_buf_cleanup;
if (unbalanced || debug) {
pr_info(" counters for queue %p, buffer %d:%s\n",
@@ -591,8 +637,13 @@ static void __vb2_queue_free(struct vb2_queue *q, unsigned int buffers)
/* Free vb2 buffers */
for (buffer = q->num_buffers - buffers; buffer < q->num_buffers;
++buffer) {
- kfree(q->bufs[buffer]);
- q->bufs[buffer] = NULL;
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
+
+ if (!vb)
+ continue;
+
+ vb2_queue_remove_buffer(q, vb);
+ kfree(vb);
}
q->num_buffers -= buffers;
@@ -628,7 +679,12 @@ static bool __buffers_in_use(struct vb2_queue *q)
{
unsigned int buffer;
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
- if (vb2_buffer_in_use(q, q->bufs[buffer]))
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
+
+ if (!vb)
+ continue;
+
+ if (vb2_buffer_in_use(q, vb))
return true;
}
return false;
@@ -636,7 +692,10 @@ static bool __buffers_in_use(struct vb2_queue *q)
void vb2_core_querybuf(struct vb2_queue *q, unsigned int index, void *pb)
{
- call_void_bufop(q, fill_user_buffer, q->bufs[index], pb);
+ struct vb2_buffer *vb = vb2_get_buffer(q, index);
+
+ if (vb)
+ call_void_bufop(q, fill_user_buffer, vb, pb);
}
EXPORT_SYMBOL_GPL(vb2_core_querybuf);
@@ -1547,7 +1606,13 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
struct vb2_buffer *vb;
int ret;
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
+
if (vb->state != VB2_BUF_STATE_DEQUEUED) {
dprintk(q, 1, "invalid buffer state %s\n",
vb2_state_name(vb->state));
@@ -1618,7 +1683,11 @@ static int vb2_start_streaming(struct vb2_queue *q)
* correctly return them to vb2.
*/
for (i = 0; i < q->num_buffers; ++i) {
- vb = q->bufs[i];
+ vb = vb2_get_buffer(q, i);
+
+ if (!vb)
+ continue;
+
if (vb->state == VB2_BUF_STATE_ACTIVE)
vb2_buffer_done(vb, VB2_BUF_STATE_QUEUED);
}
@@ -1646,7 +1715,12 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
return -EIO;
}
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
if (!req && vb->state != VB2_BUF_STATE_IN_REQUEST &&
q->requires_requests) {
@@ -2022,12 +2096,18 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
* to vb2 in stop_streaming().
*/
if (WARN_ON(atomic_read(&q->owned_by_drv_count))) {
- for (i = 0; i < q->num_buffers; ++i)
- if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
+ for (i = 0; i < q->num_buffers; ++i) {
+ struct vb2_buffer *vb = vb2_get_buffer(q, i);
+
+ if (!vb)
+ continue;
+
+ if (vb->state == VB2_BUF_STATE_ACTIVE) {
pr_warn("driver bug: stop_streaming operation is leaving buf %p in active state\n",
- q->bufs[i]);
- vb2_buffer_done(q->bufs[i], VB2_BUF_STATE_ERROR);
+ vb);
+ vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
}
+ }
/* Must be zero now */
WARN_ON(atomic_read(&q->owned_by_drv_count));
}
@@ -2061,9 +2141,14 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
* be changed, so we can't move the buf_finish() to __vb2_dqbuf().
*/
for (i = 0; i < q->num_buffers; ++i) {
- struct vb2_buffer *vb = q->bufs[i];
- struct media_request *req = vb->req_obj.req;
+ struct vb2_buffer *vb;
+ struct media_request *req;
+ vb = vb2_get_buffer(q, i);
+ if (!vb)
+ continue;
+
+ req = vb->req_obj.req;
/*
* If a request is associated with this buffer, then
* call buf_request_cancel() to give the driver to complete()
@@ -2215,7 +2300,10 @@ static int __find_plane_by_offset(struct vb2_queue *q, unsigned long off,
* return its buffer and plane numbers.
*/
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
+
+ if (!vb)
+ continue;
for (plane = 0; plane < vb->num_planes; ++plane) {
if (vb->planes[plane].m.offset == off) {
@@ -2262,7 +2350,12 @@ int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
return -EINVAL;
}
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
if (plane >= vb->num_planes) {
dprintk(q, 1, "buffer plane out of range\n");
@@ -2339,7 +2432,13 @@ int vb2_mmap(struct vb2_queue *q, struct vm_area_struct *vma)
if (ret)
goto unlock;
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ ret = -EINVAL;
+ goto unlock;
+ }
/*
* MMAP requires page_aligned buffers.
@@ -2396,7 +2495,12 @@ unsigned long vb2_get_unmapped_area(struct vb2_queue *q,
if (ret)
goto unlock;
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ ret = -EINVAL;
+ goto unlock;
+ }
vaddr = vb2_plane_vaddr(vb, plane);
mutex_unlock(&q->mmap_lock);
@@ -2625,6 +2729,7 @@ struct vb2_fileio_data {
static int __vb2_init_fileio(struct vb2_queue *q, int read)
{
struct vb2_fileio_data *fileio;
+ struct vb2_buffer *vb;
int i, ret;
unsigned int count = 0;
@@ -2679,7 +2784,13 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
* Check if plane_count is correct
* (multiplane buffers are not supported).
*/
- if (q->bufs[0]->num_planes != 1) {
+ vb = vb2_get_buffer(q, 0);
+ if (!vb) {
+ ret = -EBUSY;
+ goto err_reqbufs;
+ }
+
+ if (vb->num_planes != 1) {
ret = -EBUSY;
goto err_reqbufs;
}
@@ -2688,12 +2799,17 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
* Get kernel address of each buffer.
*/
for (i = 0; i < q->num_buffers; i++) {
- fileio->bufs[i].vaddr = vb2_plane_vaddr(q->bufs[i], 0);
+ vb = vb2_get_buffer(q, i);
+
+ if (!vb)
+ continue;
+
+ fileio->bufs[i].vaddr = vb2_plane_vaddr(vb, 0);
if (fileio->bufs[i].vaddr == NULL) {
ret = -EINVAL;
goto err_reqbufs;
}
- fileio->bufs[i].size = vb2_plane_size(q->bufs[i], 0);
+ fileio->bufs[i].size = vb2_plane_size(vb, 0);
}
/*
@@ -2821,15 +2937,18 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
fileio->cur_index = index;
buf = &fileio->bufs[index];
- b = q->bufs[index];
+ b = vb2_get_buffer(q, index);
+
+ if (!b)
+ return -EINVAL;
/*
* Get number of bytes filled by the driver
*/
buf->pos = 0;
buf->queued = 0;
- buf->size = read ? vb2_get_plane_payload(q->bufs[index], 0)
- : vb2_plane_size(q->bufs[index], 0);
+ buf->size = read ? vb2_get_plane_payload(b, 0)
+ : vb2_plane_size(b, 0);
/* Compensate for data_offset on read in the multiplanar case. */
if (is_multiplanar && read &&
b->planes[0].data_offset < buf->size) {
@@ -2872,8 +2991,12 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
* Queue next buffer if required.
*/
if (buf->pos == buf->size || (!read && fileio->write_immediately)) {
- struct vb2_buffer *b = q->bufs[index];
+ struct vb2_buffer *b = vb2_get_buffer(q, index);
+ if (!b) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
/*
* Check if this is the last buffer to read.
*/
@@ -2899,7 +3022,7 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
*/
buf->pos = 0;
buf->queued = 1;
- buf->size = vb2_plane_size(q->bufs[index], 0);
+ buf->size = vb2_plane_size(b, 0);
fileio->q_count += 1;
/*
* If we are queuing up buffers for the first time, then
@@ -2970,7 +3093,9 @@ static int vb2_thread(void *data)
* Call vb2_dqbuf to get buffer back.
*/
if (prequeue) {
- vb = q->bufs[index++];
+ vb = vb2_get_buffer(q, index++);
+ if (!vb)
+ continue;
prequeue--;
} else {
call_void_qop(q, wait_finish, q);
@@ -2979,7 +3104,7 @@ static int vb2_thread(void *data)
call_void_qop(q, wait_prepare, q);
dprintk(q, 5, "file io: vb2_dqbuf result: %d\n", ret);
if (!ret)
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
}
if (ret || threadio->stop)
break;
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index c7a54d82a55e..724135d41f7f 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -383,8 +383,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
return -EINVAL;
}
- if (q->bufs[b->index] == NULL) {
- /* Should never happen */
+ if (!vb2_get_buffer(q, b->index)) {
dprintk(q, 1, "%s: buffer is NULL\n", opname);
return -EINVAL;
}
@@ -394,7 +393,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
return -EINVAL;
}
- vb = q->bufs[b->index];
+ vb = vb2_get_buffer(q, b->index);
vbuf = to_vb2_v4l2_buffer(vb);
ret = __verify_planes_array(vb, b);
if (ret)
@@ -628,11 +627,18 @@ static const struct vb2_buf_ops v4l2_buf_ops = {
struct vb2_buffer *vb2_find_buffer(struct vb2_queue *q, u64 timestamp)
{
unsigned int i;
+ struct vb2_buffer *vb2;
- for (i = 0; i < q->num_buffers; i++)
- if (q->bufs[i]->copied_timestamp &&
- q->bufs[i]->timestamp == timestamp)
- return vb2_get_buffer(q, i);
+ for (i = 0; i < q->num_buffers; i++) {
+ vb2 = vb2_get_buffer(q, i);
+
+ if (!vb2)
+ continue;
+
+ if (vb2->copied_timestamp &&
+ vb2->timestamp == timestamp)
+ return vb2;
+ }
return NULL;
}
EXPORT_SYMBOL_GPL(vb2_find_buffer);
@@ -664,7 +670,13 @@ int vb2_querybuf(struct vb2_queue *q, struct v4l2_buffer *b)
dprintk(q, 1, "buffer index out of range\n");
return -EINVAL;
}
- vb = q->bufs[b->index];
+ vb = vb2_get_buffer(q, b->index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
+
ret = __verify_planes_array(vb, b);
if (!ret)
vb2_core_querybuf(q, b->index, b);
diff --git a/drivers/media/platform/amphion/vpu_dbg.c b/drivers/media/platform/amphion/vpu_dbg.c
index 44b830ae01d8..e2c371c2fde5 100644
--- a/drivers/media/platform/amphion/vpu_dbg.c
+++ b/drivers/media/platform/amphion/vpu_dbg.c
@@ -133,11 +133,18 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
vq = v4l2_m2m_get_src_vq(inst->fh.m2m_ctx);
for (i = 0; i < vq->num_buffers; i++) {
- struct vb2_buffer *vb = vq->bufs[i];
- struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vb2_buffer *vb;
+ struct vb2_v4l2_buffer *vbuf;
+
+ vb = vb2_get_buffer(vq, i);
+ if (!vb)
+ continue;
if (vb->state == VB2_BUF_STATE_DEQUEUED)
continue;
+
+ vbuf = to_vb2_v4l2_buffer(vb);
+
num = scnprintf(str, sizeof(str),
"output [%2d] state = %10s, %8s\n",
i, vb2_stat_name[vb->state],
@@ -148,11 +155,18 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
vq = v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx);
for (i = 0; i < vq->num_buffers; i++) {
- struct vb2_buffer *vb = vq->bufs[i];
- struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vb2_buffer *vb;
+ struct vb2_v4l2_buffer *vbuf;
+
+ vb = vb2_get_buffer(vq, i);
+ if (!vb)
+ continue;
if (vb->state == VB2_BUF_STATE_DEQUEUED)
continue;
+
+ vbuf = to_vb2_v4l2_buffer(vb);
+
num = scnprintf(str, sizeof(str),
"capture[%2d] state = %10s, %8s\n",
i, vb2_stat_name[vb->state],
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 0051f372a66c..ea37069ba355 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -603,7 +603,11 @@ static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
return -EINVAL;
}
- vb = vq->bufs[buf->index];
+ vb = vb2_get_buffer(vq, buf->index);
+ if (!vb) {
+ dev_err(ctx->jpeg->dev, "buffer not found\n");
+ return -EINVAL;
+ }
jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(vb);
jpeg_src_buf->bs_size = buf->m.planes[0].bytesused;
diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
index cf16cf2807f0..6532a69f1fa8 100644
--- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
+++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
@@ -1701,7 +1701,7 @@ static int vdec_vp9_slice_setup_core_buffer(struct vdec_vp9_slice_instance *inst
/* update internal buffer's width/height */
for (i = 0; i < vq->num_buffers; i++) {
- if (vb == vq->bufs[i]) {
+ if (vb == vb2_get_buffer(vq, i)) {
instance->dpb[i].width = w;
instance->dpb[i].height = h;
break;
diff --git a/drivers/media/platform/st/sti/hva/hva-v4l2.c b/drivers/media/platform/st/sti/hva/hva-v4l2.c
index 3a848ca32a0e..326be09bdb55 100644
--- a/drivers/media/platform/st/sti/hva/hva-v4l2.c
+++ b/drivers/media/platform/st/sti/hva/hva-v4l2.c
@@ -577,6 +577,10 @@ static int hva_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
}
vb2_buf = vb2_get_buffer(vq, buf->index);
+ if (!vb2_buf) {
+ dev_dbg(dev, "%s buffer index %d not found\n", ctx->name, buf->index);
+ return -EINVAL;
+ }
stream = to_hva_stream(to_vb2_v4l2_buffer(vb2_buf));
stream->bytesused = buf->bytesused;
}
diff --git a/drivers/media/test-drivers/visl/visl-dec.c b/drivers/media/test-drivers/visl/visl-dec.c
index 318d675e5668..ba20ea998d19 100644
--- a/drivers/media/test-drivers/visl/visl-dec.c
+++ b/drivers/media/test-drivers/visl/visl-dec.c
@@ -290,13 +290,20 @@ static void visl_tpg_fill(struct visl_ctx *ctx, struct visl_run *run)
for (i = 0; i < out_q->num_buffers; i++) {
char entry[] = "index: %u, state: %s, request_fd: %d, ";
u32 old_len = len;
- char *q_status = visl_get_vb2_state(out_q->bufs[i]->state);
+ struct vb2_buffer *vb2;
+ char *q_status;
+
+ vb2 = vb2_get_buffer(out_q, i);
+ if (!vb2)
+ continue;
+
+ q_status = visl_get_vb2_state(vb2->state);
len += scnprintf(&buf[len], TPG_STR_BUF_SZ - len,
entry, i, q_status,
- to_vb2_v4l2_buffer(out_q->bufs[i])->request_fd);
+ to_vb2_v4l2_buffer(vb2)->request_fd);
- len += visl_fill_bytesused(to_vb2_v4l2_buffer(out_q->bufs[i]),
+ len += visl_fill_bytesused(to_vb2_v4l2_buffer(vb2),
&buf[len],
TPG_STR_BUF_SZ - len);
@@ -342,13 +349,20 @@ static void visl_tpg_fill(struct visl_ctx *ctx, struct visl_run *run)
len = 0;
for (i = 0; i < cap_q->num_buffers; i++) {
u32 old_len = len;
- char *q_status = visl_get_vb2_state(cap_q->bufs[i]->state);
+ struct vb2_buffer *vb2;
+ char *q_status;
+
+ vb2 = vb2_get_buffer(cap_q, i);
+ if (!vb2)
+ continue;
+
+ q_status = visl_get_vb2_state(vb2->state);
len += scnprintf(&buf[len], TPG_STR_BUF_SZ - len,
"index: %u, status: %s, timestamp: %llu, is_held: %d",
- cap_q->bufs[i]->index, q_status,
- cap_q->bufs[i]->timestamp,
- to_vb2_v4l2_buffer(cap_q->bufs[i])->is_held);
+ vb2->index, q_status,
+ vb2->timestamp,
+ to_vb2_v4l2_buffer(vb2)->is_held);
tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, &buf[old_len]);
frame_dprintk(ctx->dev, run->dst->sequence, "%s", &buf[old_len]);
diff --git a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
index 384f31fc66c5..ec7e05263c8e 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
@@ -1077,7 +1077,7 @@ static int atomisp_dqbuf_wrapper(struct file *file, void *fh, struct v4l2_buffer
if (ret)
return ret;
- vb = pipe->vb_queue.bufs[buf->index];
+ vb = vb2_get_buffer(pipe->vb_queue, buf->index);
frame = vb_to_frame(vb);
buf->reserved = asd->frame_status[buf->index];
--
2.39.2
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
The first step before changing how vb2 buffers are stored into queue
is to avoid direct access to bufs arrays.
This patch adds 2 helpers functions to add and remove vb2 buffers
from a queue. With these 2 and vb2_get_buffer(), bufs field of
struct vb2_queue becomes like a private member of the structure.
After each call to vb2_get_buffer() we need to be sure that we get
a valid pointer so check the return value of all of them.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/common/videobuf2/videobuf2-core.c | 215 ++++++++++++++----
.../media/common/videobuf2/videobuf2-v4l2.c | 28 ++-
drivers/media/platform/amphion/vpu_dbg.c | 22 +-
.../platform/mediatek/jpeg/mtk_jpeg_core.c | 6 +-
.../vcodec/vdec/vdec_vp9_req_lat_if.c | 2 +-
drivers/media/platform/st/sti/hva/hva-v4l2.c | 4 +
drivers/media/test-drivers/visl/visl-dec.c | 28 ++-
.../staging/media/atomisp/pci/atomisp_ioctl.c | 2 +-
8 files changed, 240 insertions(+), 67 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index cf6727d9c81f..42fd3984c2bc 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -359,8 +359,16 @@ static void __setup_offsets(struct vb2_buffer *vb)
unsigned long off = 0;
if (vb->index) {
- struct vb2_buffer *prev = q->bufs[vb->index - 1];
- struct vb2_plane *p = &prev->planes[prev->num_planes - 1];
+ struct vb2_buffer *prev;
+ struct vb2_plane *p;
+
+ prev = vb2_get_buffer(q, vb->index - 1);
+ if (!prev) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return;
+ }
+
+ p = &prev->planes[prev->num_planes - 1];
off = PAGE_ALIGN(p->m.offset + p->length);
}
@@ -397,6 +405,37 @@ static void init_buffer_cache_hints(struct vb2_queue *q, struct vb2_buffer *vb)
vb->skip_cache_sync_on_finish = 1;
}
+/**
+ * vb2_queue_add_buffer() - add a buffer to a queue
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @vb: pointer to &struct vb2_buffer to be added to the queue.
+ * @index: index where add vb2_buffer in the queue
+ */
+static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb, int index)
+{
+ if (index < VB2_MAX_FRAME && !q->bufs[index]) {
+ q->bufs[index] = vb;
+ vb->index = index;
+ vb->vb2_queue = q;
+ return true;
+ }
+
+ return false;
+}
+
+/**
+ * vb2_queue_remove_buffer() - remove a buffer from a queue
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @vb: pointer to &struct vb2_buffer to be removed from the queue.
+ */
+static void vb2_queue_remove_buffer(struct vb2_queue *q, struct vb2_buffer *vb)
+{
+ if (vb->index < VB2_MAX_FRAME) {
+ q->bufs[vb->index] = NULL;
+ vb->vb2_queue = NULL;
+ }
+}
+
/*
* __vb2_queue_alloc() - allocate vb2 buffer structures and (for MMAP type)
* video buffer memory for all buffers/planes on the queue and initializes the
@@ -425,9 +464,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
}
vb->state = VB2_BUF_STATE_DEQUEUED;
- vb->vb2_queue = q;
vb->num_planes = num_planes;
- vb->index = q->num_buffers + buffer;
vb->type = q->type;
vb->memory = memory;
init_buffer_cache_hints(q, vb);
@@ -437,7 +474,11 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
}
call_void_bufop(q, init_buffer, vb);
- q->bufs[vb->index] = vb;
+ if (!vb2_queue_add_buffer(q, vb, q->num_buffers + buffer)) {
+ dprintk(q, 1, "failed adding buffer %d to queue\n", buffer);
+ kfree(vb);
+ break;
+ }
/* Allocate video buffer memory for the MMAP type */
if (memory == VB2_MEMORY_MMAP) {
@@ -445,7 +486,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
if (ret) {
dprintk(q, 1, "failed allocating memory for buffer %d\n",
buffer);
- q->bufs[vb->index] = NULL;
+ vb2_queue_remove_buffer(q, vb);
kfree(vb);
break;
}
@@ -460,7 +501,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
dprintk(q, 1, "buffer %d %p initialization failed\n",
buffer, vb);
__vb2_buf_mem_free(vb);
- q->bufs[vb->index] = NULL;
+ vb2_queue_remove_buffer(q, vb);
kfree(vb);
break;
}
@@ -483,7 +524,7 @@ static void __vb2_free_mem(struct vb2_queue *q, unsigned int buffers)
for (buffer = q->num_buffers - buffers; buffer < q->num_buffers;
++buffer) {
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
if (!vb)
continue;
@@ -511,7 +552,7 @@ static void __vb2_queue_free(struct vb2_queue *q, unsigned int buffers)
/* Call driver-provided cleanup function for each buffer, if provided */
for (buffer = q->num_buffers - buffers; buffer < q->num_buffers;
++buffer) {
- struct vb2_buffer *vb = q->bufs[buffer];
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
if (vb && vb->planes[0].mem_priv)
call_void_vb_qop(vb, buf_cleanup, vb);
@@ -551,15 +592,20 @@ static void __vb2_queue_free(struct vb2_queue *q, unsigned int buffers)
q->cnt_unprepare_streaming = 0;
}
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
- struct vb2_buffer *vb = q->bufs[buffer];
- bool unbalanced = vb->cnt_mem_alloc != vb->cnt_mem_put ||
- vb->cnt_mem_prepare != vb->cnt_mem_finish ||
- vb->cnt_mem_get_userptr != vb->cnt_mem_put_userptr ||
- vb->cnt_mem_attach_dmabuf != vb->cnt_mem_detach_dmabuf ||
- vb->cnt_mem_map_dmabuf != vb->cnt_mem_unmap_dmabuf ||
- vb->cnt_buf_queue != vb->cnt_buf_done ||
- vb->cnt_buf_prepare != vb->cnt_buf_finish ||
- vb->cnt_buf_init != vb->cnt_buf_cleanup;
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
+ bool unbalanced;
+
+ if (!vb)
+ continue;
+
+ unbalanced = vb->cnt_mem_alloc != vb->cnt_mem_put ||
+ vb->cnt_mem_prepare != vb->cnt_mem_finish ||
+ vb->cnt_mem_get_userptr != vb->cnt_mem_put_userptr ||
+ vb->cnt_mem_attach_dmabuf != vb->cnt_mem_detach_dmabuf ||
+ vb->cnt_mem_map_dmabuf != vb->cnt_mem_unmap_dmabuf ||
+ vb->cnt_buf_queue != vb->cnt_buf_done ||
+ vb->cnt_buf_prepare != vb->cnt_buf_finish ||
+ vb->cnt_buf_init != vb->cnt_buf_cleanup;
if (unbalanced || debug) {
pr_info(" counters for queue %p, buffer %d:%s\n",
@@ -591,8 +637,13 @@ static void __vb2_queue_free(struct vb2_queue *q, unsigned int buffers)
/* Free vb2 buffers */
for (buffer = q->num_buffers - buffers; buffer < q->num_buffers;
++buffer) {
- kfree(q->bufs[buffer]);
- q->bufs[buffer] = NULL;
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
+
+ if (!vb)
+ continue;
+
+ vb2_queue_remove_buffer(q, vb);
+ kfree(vb);
}
q->num_buffers -= buffers;
@@ -628,7 +679,12 @@ static bool __buffers_in_use(struct vb2_queue *q)
{
unsigned int buffer;
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
- if (vb2_buffer_in_use(q, q->bufs[buffer]))
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
+
+ if (!vb)
+ continue;
+
+ if (vb2_buffer_in_use(q, vb))
return true;
}
return false;
@@ -636,7 +692,10 @@ static bool __buffers_in_use(struct vb2_queue *q)
void vb2_core_querybuf(struct vb2_queue *q, unsigned int index, void *pb)
{
- call_void_bufop(q, fill_user_buffer, q->bufs[index], pb);
+ struct vb2_buffer *vb = vb2_get_buffer(q, index);
+
+ if (vb)
+ call_void_bufop(q, fill_user_buffer, vb, pb);
}
EXPORT_SYMBOL_GPL(vb2_core_querybuf);
@@ -1547,7 +1606,13 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
struct vb2_buffer *vb;
int ret;
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
+
if (vb->state != VB2_BUF_STATE_DEQUEUED) {
dprintk(q, 1, "invalid buffer state %s\n",
vb2_state_name(vb->state));
@@ -1618,7 +1683,11 @@ static int vb2_start_streaming(struct vb2_queue *q)
* correctly return them to vb2.
*/
for (i = 0; i < q->num_buffers; ++i) {
- vb = q->bufs[i];
+ vb = vb2_get_buffer(q, i);
+
+ if (!vb)
+ continue;
+
if (vb->state == VB2_BUF_STATE_ACTIVE)
vb2_buffer_done(vb, VB2_BUF_STATE_QUEUED);
}
@@ -1646,7 +1715,12 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
return -EIO;
}
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
if (!req && vb->state != VB2_BUF_STATE_IN_REQUEST &&
q->requires_requests) {
@@ -2022,12 +2096,18 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
* to vb2 in stop_streaming().
*/
if (WARN_ON(atomic_read(&q->owned_by_drv_count))) {
- for (i = 0; i < q->num_buffers; ++i)
- if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
+ for (i = 0; i < q->num_buffers; ++i) {
+ struct vb2_buffer *vb = vb2_get_buffer(q, i);
+
+ if (!vb)
+ continue;
+
+ if (vb->state == VB2_BUF_STATE_ACTIVE) {
pr_warn("driver bug: stop_streaming operation is leaving buf %p in active state\n",
- q->bufs[i]);
- vb2_buffer_done(q->bufs[i], VB2_BUF_STATE_ERROR);
+ vb);
+ vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
}
+ }
/* Must be zero now */
WARN_ON(atomic_read(&q->owned_by_drv_count));
}
@@ -2061,9 +2141,14 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
* be changed, so we can't move the buf_finish() to __vb2_dqbuf().
*/
for (i = 0; i < q->num_buffers; ++i) {
- struct vb2_buffer *vb = q->bufs[i];
- struct media_request *req = vb->req_obj.req;
+ struct vb2_buffer *vb;
+ struct media_request *req;
+ vb = vb2_get_buffer(q, i);
+ if (!vb)
+ continue;
+
+ req = vb->req_obj.req;
/*
* If a request is associated with this buffer, then
* call buf_request_cancel() to give the driver to complete()
@@ -2215,7 +2300,10 @@ static int __find_plane_by_offset(struct vb2_queue *q, unsigned long off,
* return its buffer and plane numbers.
*/
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
+
+ if (!vb)
+ continue;
for (plane = 0; plane < vb->num_planes; ++plane) {
if (vb->planes[plane].m.offset == off) {
@@ -2262,7 +2350,12 @@ int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
return -EINVAL;
}
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
if (plane >= vb->num_planes) {
dprintk(q, 1, "buffer plane out of range\n");
@@ -2339,7 +2432,13 @@ int vb2_mmap(struct vb2_queue *q, struct vm_area_struct *vma)
if (ret)
goto unlock;
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ ret = -EINVAL;
+ goto unlock;
+ }
/*
* MMAP requires page_aligned buffers.
@@ -2396,7 +2495,12 @@ unsigned long vb2_get_unmapped_area(struct vb2_queue *q,
if (ret)
goto unlock;
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ ret = -EINVAL;
+ goto unlock;
+ }
vaddr = vb2_plane_vaddr(vb, plane);
mutex_unlock(&q->mmap_lock);
@@ -2625,6 +2729,7 @@ struct vb2_fileio_data {
static int __vb2_init_fileio(struct vb2_queue *q, int read)
{
struct vb2_fileio_data *fileio;
+ struct vb2_buffer *vb;
int i, ret;
unsigned int count = 0;
@@ -2679,7 +2784,13 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
* Check if plane_count is correct
* (multiplane buffers are not supported).
*/
- if (q->bufs[0]->num_planes != 1) {
+ vb = vb2_get_buffer(q, 0);
+ if (!vb) {
+ ret = -EBUSY;
+ goto err_reqbufs;
+ }
+
+ if (vb->num_planes != 1) {
ret = -EBUSY;
goto err_reqbufs;
}
@@ -2688,12 +2799,17 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
* Get kernel address of each buffer.
*/
for (i = 0; i < q->num_buffers; i++) {
- fileio->bufs[i].vaddr = vb2_plane_vaddr(q->bufs[i], 0);
+ vb = vb2_get_buffer(q, i);
+
+ if (!vb)
+ continue;
+
+ fileio->bufs[i].vaddr = vb2_plane_vaddr(vb, 0);
if (fileio->bufs[i].vaddr == NULL) {
ret = -EINVAL;
goto err_reqbufs;
}
- fileio->bufs[i].size = vb2_plane_size(q->bufs[i], 0);
+ fileio->bufs[i].size = vb2_plane_size(vb, 0);
}
/*
@@ -2821,15 +2937,18 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
fileio->cur_index = index;
buf = &fileio->bufs[index];
- b = q->bufs[index];
+ b = vb2_get_buffer(q, index);
+
+ if (!b)
+ return -EINVAL;
/*
* Get number of bytes filled by the driver
*/
buf->pos = 0;
buf->queued = 0;
- buf->size = read ? vb2_get_plane_payload(q->bufs[index], 0)
- : vb2_plane_size(q->bufs[index], 0);
+ buf->size = read ? vb2_get_plane_payload(b, 0)
+ : vb2_plane_size(b, 0);
/* Compensate for data_offset on read in the multiplanar case. */
if (is_multiplanar && read &&
b->planes[0].data_offset < buf->size) {
@@ -2872,8 +2991,12 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
* Queue next buffer if required.
*/
if (buf->pos == buf->size || (!read && fileio->write_immediately)) {
- struct vb2_buffer *b = q->bufs[index];
+ struct vb2_buffer *b = vb2_get_buffer(q, index);
+ if (!b) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
/*
* Check if this is the last buffer to read.
*/
@@ -2899,7 +3022,7 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
*/
buf->pos = 0;
buf->queued = 1;
- buf->size = vb2_plane_size(q->bufs[index], 0);
+ buf->size = vb2_plane_size(b, 0);
fileio->q_count += 1;
/*
* If we are queuing up buffers for the first time, then
@@ -2970,7 +3093,9 @@ static int vb2_thread(void *data)
* Call vb2_dqbuf to get buffer back.
*/
if (prequeue) {
- vb = q->bufs[index++];
+ vb = vb2_get_buffer(q, index++);
+ if (!vb)
+ continue;
prequeue--;
} else {
call_void_qop(q, wait_finish, q);
@@ -2979,7 +3104,7 @@ static int vb2_thread(void *data)
call_void_qop(q, wait_prepare, q);
dprintk(q, 5, "file io: vb2_dqbuf result: %d\n", ret);
if (!ret)
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
}
if (ret || threadio->stop)
break;
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index c7a54d82a55e..724135d41f7f 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -383,8 +383,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
return -EINVAL;
}
- if (q->bufs[b->index] == NULL) {
- /* Should never happen */
+ if (!vb2_get_buffer(q, b->index)) {
dprintk(q, 1, "%s: buffer is NULL\n", opname);
return -EINVAL;
}
@@ -394,7 +393,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
return -EINVAL;
}
- vb = q->bufs[b->index];
+ vb = vb2_get_buffer(q, b->index);
vbuf = to_vb2_v4l2_buffer(vb);
ret = __verify_planes_array(vb, b);
if (ret)
@@ -628,11 +627,18 @@ static const struct vb2_buf_ops v4l2_buf_ops = {
struct vb2_buffer *vb2_find_buffer(struct vb2_queue *q, u64 timestamp)
{
unsigned int i;
+ struct vb2_buffer *vb2;
- for (i = 0; i < q->num_buffers; i++)
- if (q->bufs[i]->copied_timestamp &&
- q->bufs[i]->timestamp == timestamp)
- return vb2_get_buffer(q, i);
+ for (i = 0; i < q->num_buffers; i++) {
+ vb2 = vb2_get_buffer(q, i);
+
+ if (!vb2)
+ continue;
+
+ if (vb2->copied_timestamp &&
+ vb2->timestamp == timestamp)
+ return vb2;
+ }
return NULL;
}
EXPORT_SYMBOL_GPL(vb2_find_buffer);
@@ -664,7 +670,13 @@ int vb2_querybuf(struct vb2_queue *q, struct v4l2_buffer *b)
dprintk(q, 1, "buffer index out of range\n");
return -EINVAL;
}
- vb = q->bufs[b->index];
+ vb = vb2_get_buffer(q, b->index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
+
ret = __verify_planes_array(vb, b);
if (!ret)
vb2_core_querybuf(q, b->index, b);
diff --git a/drivers/media/platform/amphion/vpu_dbg.c b/drivers/media/platform/amphion/vpu_dbg.c
index 44b830ae01d8..e2c371c2fde5 100644
--- a/drivers/media/platform/amphion/vpu_dbg.c
+++ b/drivers/media/platform/amphion/vpu_dbg.c
@@ -133,11 +133,18 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
vq = v4l2_m2m_get_src_vq(inst->fh.m2m_ctx);
for (i = 0; i < vq->num_buffers; i++) {
- struct vb2_buffer *vb = vq->bufs[i];
- struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vb2_buffer *vb;
+ struct vb2_v4l2_buffer *vbuf;
+
+ vb = vb2_get_buffer(vq, i);
+ if (!vb)
+ continue;
if (vb->state == VB2_BUF_STATE_DEQUEUED)
continue;
+
+ vbuf = to_vb2_v4l2_buffer(vb);
+
num = scnprintf(str, sizeof(str),
"output [%2d] state = %10s, %8s\n",
i, vb2_stat_name[vb->state],
@@ -148,11 +155,18 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
vq = v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx);
for (i = 0; i < vq->num_buffers; i++) {
- struct vb2_buffer *vb = vq->bufs[i];
- struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vb2_buffer *vb;
+ struct vb2_v4l2_buffer *vbuf;
+
+ vb = vb2_get_buffer(vq, i);
+ if (!vb)
+ continue;
if (vb->state == VB2_BUF_STATE_DEQUEUED)
continue;
+
+ vbuf = to_vb2_v4l2_buffer(vb);
+
num = scnprintf(str, sizeof(str),
"capture[%2d] state = %10s, %8s\n",
i, vb2_stat_name[vb->state],
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 0051f372a66c..ea37069ba355 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -603,7 +603,11 @@ static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
return -EINVAL;
}
- vb = vq->bufs[buf->index];
+ vb = vb2_get_buffer(vq, buf->index);
+ if (!vb) {
+ dev_err(ctx->jpeg->dev, "buffer not found\n");
+ return -EINVAL;
+ }
jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(vb);
jpeg_src_buf->bs_size = buf->m.planes[0].bytesused;
diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
index cf16cf2807f0..6532a69f1fa8 100644
--- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
+++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
@@ -1701,7 +1701,7 @@ static int vdec_vp9_slice_setup_core_buffer(struct vdec_vp9_slice_instance *inst
/* update internal buffer's width/height */
for (i = 0; i < vq->num_buffers; i++) {
- if (vb == vq->bufs[i]) {
+ if (vb == vb2_get_buffer(vq, i)) {
instance->dpb[i].width = w;
instance->dpb[i].height = h;
break;
diff --git a/drivers/media/platform/st/sti/hva/hva-v4l2.c b/drivers/media/platform/st/sti/hva/hva-v4l2.c
index 3a848ca32a0e..326be09bdb55 100644
--- a/drivers/media/platform/st/sti/hva/hva-v4l2.c
+++ b/drivers/media/platform/st/sti/hva/hva-v4l2.c
@@ -577,6 +577,10 @@ static int hva_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
}
vb2_buf = vb2_get_buffer(vq, buf->index);
+ if (!vb2_buf) {
+ dev_dbg(dev, "%s buffer index %d not found\n", ctx->name, buf->index);
+ return -EINVAL;
+ }
stream = to_hva_stream(to_vb2_v4l2_buffer(vb2_buf));
stream->bytesused = buf->bytesused;
}
diff --git a/drivers/media/test-drivers/visl/visl-dec.c b/drivers/media/test-drivers/visl/visl-dec.c
index 318d675e5668..ba20ea998d19 100644
--- a/drivers/media/test-drivers/visl/visl-dec.c
+++ b/drivers/media/test-drivers/visl/visl-dec.c
@@ -290,13 +290,20 @@ static void visl_tpg_fill(struct visl_ctx *ctx, struct visl_run *run)
for (i = 0; i < out_q->num_buffers; i++) {
char entry[] = "index: %u, state: %s, request_fd: %d, ";
u32 old_len = len;
- char *q_status = visl_get_vb2_state(out_q->bufs[i]->state);
+ struct vb2_buffer *vb2;
+ char *q_status;
+
+ vb2 = vb2_get_buffer(out_q, i);
+ if (!vb2)
+ continue;
+
+ q_status = visl_get_vb2_state(vb2->state);
len += scnprintf(&buf[len], TPG_STR_BUF_SZ - len,
entry, i, q_status,
- to_vb2_v4l2_buffer(out_q->bufs[i])->request_fd);
+ to_vb2_v4l2_buffer(vb2)->request_fd);
- len += visl_fill_bytesused(to_vb2_v4l2_buffer(out_q->bufs[i]),
+ len += visl_fill_bytesused(to_vb2_v4l2_buffer(vb2),
&buf[len],
TPG_STR_BUF_SZ - len);
@@ -342,13 +349,20 @@ static void visl_tpg_fill(struct visl_ctx *ctx, struct visl_run *run)
len = 0;
for (i = 0; i < cap_q->num_buffers; i++) {
u32 old_len = len;
- char *q_status = visl_get_vb2_state(cap_q->bufs[i]->state);
+ struct vb2_buffer *vb2;
+ char *q_status;
+
+ vb2 = vb2_get_buffer(cap_q, i);
+ if (!vb2)
+ continue;
+
+ q_status = visl_get_vb2_state(vb2->state);
len += scnprintf(&buf[len], TPG_STR_BUF_SZ - len,
"index: %u, status: %s, timestamp: %llu, is_held: %d",
- cap_q->bufs[i]->index, q_status,
- cap_q->bufs[i]->timestamp,
- to_vb2_v4l2_buffer(cap_q->bufs[i])->is_held);
+ vb2->index, q_status,
+ vb2->timestamp,
+ to_vb2_v4l2_buffer(vb2)->is_held);
tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, &buf[old_len]);
frame_dprintk(ctx->dev, run->dst->sequence, "%s", &buf[old_len]);
diff --git a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
index 384f31fc66c5..ec7e05263c8e 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
@@ -1077,7 +1077,7 @@ static int atomisp_dqbuf_wrapper(struct file *file, void *fh, struct v4l2_buffer
if (ret)
return ret;
- vb = pipe->vb_queue.bufs[buf->index];
+ vb = vb2_get_buffer(pipe->vb_queue, buf->index);
frame = vb_to_frame(vb);
buf->reserved = asd->frame_status[buf->index];
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
The first step before changing how vb2 buffers are stored into queue
is to avoid direct access to bufs arrays.
This patch adds 2 helpers functions to add and remove vb2 buffers
from a queue. With these 2 and vb2_get_buffer(), bufs field of
struct vb2_queue becomes like a private member of the structure.
After each call to vb2_get_buffer() we need to be sure that we get
a valid pointer so check the return value of all of them.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/common/videobuf2/videobuf2-core.c | 215 ++++++++++++++----
.../media/common/videobuf2/videobuf2-v4l2.c | 28 ++-
drivers/media/platform/amphion/vpu_dbg.c | 22 +-
.../platform/mediatek/jpeg/mtk_jpeg_core.c | 6 +-
.../vcodec/vdec/vdec_vp9_req_lat_if.c | 2 +-
drivers/media/platform/st/sti/hva/hva-v4l2.c | 4 +
drivers/media/test-drivers/visl/visl-dec.c | 28 ++-
.../staging/media/atomisp/pci/atomisp_ioctl.c | 2 +-
8 files changed, 240 insertions(+), 67 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index cf6727d9c81f..42fd3984c2bc 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -359,8 +359,16 @@ static void __setup_offsets(struct vb2_buffer *vb)
unsigned long off = 0;
if (vb->index) {
- struct vb2_buffer *prev = q->bufs[vb->index - 1];
- struct vb2_plane *p = &prev->planes[prev->num_planes - 1];
+ struct vb2_buffer *prev;
+ struct vb2_plane *p;
+
+ prev = vb2_get_buffer(q, vb->index - 1);
+ if (!prev) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return;
+ }
+
+ p = &prev->planes[prev->num_planes - 1];
off = PAGE_ALIGN(p->m.offset + p->length);
}
@@ -397,6 +405,37 @@ static void init_buffer_cache_hints(struct vb2_queue *q, struct vb2_buffer *vb)
vb->skip_cache_sync_on_finish = 1;
}
+/**
+ * vb2_queue_add_buffer() - add a buffer to a queue
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @vb: pointer to &struct vb2_buffer to be added to the queue.
+ * @index: index where add vb2_buffer in the queue
+ */
+static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb, int index)
+{
+ if (index < VB2_MAX_FRAME && !q->bufs[index]) {
+ q->bufs[index] = vb;
+ vb->index = index;
+ vb->vb2_queue = q;
+ return true;
+ }
+
+ return false;
+}
+
+/**
+ * vb2_queue_remove_buffer() - remove a buffer from a queue
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @vb: pointer to &struct vb2_buffer to be removed from the queue.
+ */
+static void vb2_queue_remove_buffer(struct vb2_queue *q, struct vb2_buffer *vb)
+{
+ if (vb->index < VB2_MAX_FRAME) {
+ q->bufs[vb->index] = NULL;
+ vb->vb2_queue = NULL;
+ }
+}
+
/*
* __vb2_queue_alloc() - allocate vb2 buffer structures and (for MMAP type)
* video buffer memory for all buffers/planes on the queue and initializes the
@@ -425,9 +464,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
}
vb->state = VB2_BUF_STATE_DEQUEUED;
- vb->vb2_queue = q;
vb->num_planes = num_planes;
- vb->index = q->num_buffers + buffer;
vb->type = q->type;
vb->memory = memory;
init_buffer_cache_hints(q, vb);
@@ -437,7 +474,11 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
}
call_void_bufop(q, init_buffer, vb);
- q->bufs[vb->index] = vb;
+ if (!vb2_queue_add_buffer(q, vb, q->num_buffers + buffer)) {
+ dprintk(q, 1, "failed adding buffer %d to queue\n", buffer);
+ kfree(vb);
+ break;
+ }
/* Allocate video buffer memory for the MMAP type */
if (memory == VB2_MEMORY_MMAP) {
@@ -445,7 +486,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
if (ret) {
dprintk(q, 1, "failed allocating memory for buffer %d\n",
buffer);
- q->bufs[vb->index] = NULL;
+ vb2_queue_remove_buffer(q, vb);
kfree(vb);
break;
}
@@ -460,7 +501,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
dprintk(q, 1, "buffer %d %p initialization failed\n",
buffer, vb);
__vb2_buf_mem_free(vb);
- q->bufs[vb->index] = NULL;
+ vb2_queue_remove_buffer(q, vb);
kfree(vb);
break;
}
@@ -483,7 +524,7 @@ static void __vb2_free_mem(struct vb2_queue *q, unsigned int buffers)
for (buffer = q->num_buffers - buffers; buffer < q->num_buffers;
++buffer) {
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
if (!vb)
continue;
@@ -511,7 +552,7 @@ static void __vb2_queue_free(struct vb2_queue *q, unsigned int buffers)
/* Call driver-provided cleanup function for each buffer, if provided */
for (buffer = q->num_buffers - buffers; buffer < q->num_buffers;
++buffer) {
- struct vb2_buffer *vb = q->bufs[buffer];
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
if (vb && vb->planes[0].mem_priv)
call_void_vb_qop(vb, buf_cleanup, vb);
@@ -551,15 +592,20 @@ static void __vb2_queue_free(struct vb2_queue *q, unsigned int buffers)
q->cnt_unprepare_streaming = 0;
}
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
- struct vb2_buffer *vb = q->bufs[buffer];
- bool unbalanced = vb->cnt_mem_alloc != vb->cnt_mem_put ||
- vb->cnt_mem_prepare != vb->cnt_mem_finish ||
- vb->cnt_mem_get_userptr != vb->cnt_mem_put_userptr ||
- vb->cnt_mem_attach_dmabuf != vb->cnt_mem_detach_dmabuf ||
- vb->cnt_mem_map_dmabuf != vb->cnt_mem_unmap_dmabuf ||
- vb->cnt_buf_queue != vb->cnt_buf_done ||
- vb->cnt_buf_prepare != vb->cnt_buf_finish ||
- vb->cnt_buf_init != vb->cnt_buf_cleanup;
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
+ bool unbalanced;
+
+ if (!vb)
+ continue;
+
+ unbalanced = vb->cnt_mem_alloc != vb->cnt_mem_put ||
+ vb->cnt_mem_prepare != vb->cnt_mem_finish ||
+ vb->cnt_mem_get_userptr != vb->cnt_mem_put_userptr ||
+ vb->cnt_mem_attach_dmabuf != vb->cnt_mem_detach_dmabuf ||
+ vb->cnt_mem_map_dmabuf != vb->cnt_mem_unmap_dmabuf ||
+ vb->cnt_buf_queue != vb->cnt_buf_done ||
+ vb->cnt_buf_prepare != vb->cnt_buf_finish ||
+ vb->cnt_buf_init != vb->cnt_buf_cleanup;
if (unbalanced || debug) {
pr_info(" counters for queue %p, buffer %d:%s\n",
@@ -591,8 +637,13 @@ static void __vb2_queue_free(struct vb2_queue *q, unsigned int buffers)
/* Free vb2 buffers */
for (buffer = q->num_buffers - buffers; buffer < q->num_buffers;
++buffer) {
- kfree(q->bufs[buffer]);
- q->bufs[buffer] = NULL;
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
+
+ if (!vb)
+ continue;
+
+ vb2_queue_remove_buffer(q, vb);
+ kfree(vb);
}
q->num_buffers -= buffers;
@@ -628,7 +679,12 @@ static bool __buffers_in_use(struct vb2_queue *q)
{
unsigned int buffer;
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
- if (vb2_buffer_in_use(q, q->bufs[buffer]))
+ struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
+
+ if (!vb)
+ continue;
+
+ if (vb2_buffer_in_use(q, vb))
return true;
}
return false;
@@ -636,7 +692,10 @@ static bool __buffers_in_use(struct vb2_queue *q)
void vb2_core_querybuf(struct vb2_queue *q, unsigned int index, void *pb)
{
- call_void_bufop(q, fill_user_buffer, q->bufs[index], pb);
+ struct vb2_buffer *vb = vb2_get_buffer(q, index);
+
+ if (vb)
+ call_void_bufop(q, fill_user_buffer, vb, pb);
}
EXPORT_SYMBOL_GPL(vb2_core_querybuf);
@@ -1547,7 +1606,13 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
struct vb2_buffer *vb;
int ret;
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
+
if (vb->state != VB2_BUF_STATE_DEQUEUED) {
dprintk(q, 1, "invalid buffer state %s\n",
vb2_state_name(vb->state));
@@ -1618,7 +1683,11 @@ static int vb2_start_streaming(struct vb2_queue *q)
* correctly return them to vb2.
*/
for (i = 0; i < q->num_buffers; ++i) {
- vb = q->bufs[i];
+ vb = vb2_get_buffer(q, i);
+
+ if (!vb)
+ continue;
+
if (vb->state == VB2_BUF_STATE_ACTIVE)
vb2_buffer_done(vb, VB2_BUF_STATE_QUEUED);
}
@@ -1646,7 +1715,12 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
return -EIO;
}
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
if (!req && vb->state != VB2_BUF_STATE_IN_REQUEST &&
q->requires_requests) {
@@ -2022,12 +2096,18 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
* to vb2 in stop_streaming().
*/
if (WARN_ON(atomic_read(&q->owned_by_drv_count))) {
- for (i = 0; i < q->num_buffers; ++i)
- if (q->bufs[i]->state == VB2_BUF_STATE_ACTIVE) {
+ for (i = 0; i < q->num_buffers; ++i) {
+ struct vb2_buffer *vb = vb2_get_buffer(q, i);
+
+ if (!vb)
+ continue;
+
+ if (vb->state == VB2_BUF_STATE_ACTIVE) {
pr_warn("driver bug: stop_streaming operation is leaving buf %p in active state\n",
- q->bufs[i]);
- vb2_buffer_done(q->bufs[i], VB2_BUF_STATE_ERROR);
+ vb);
+ vb2_buffer_done(vb, VB2_BUF_STATE_ERROR);
}
+ }
/* Must be zero now */
WARN_ON(atomic_read(&q->owned_by_drv_count));
}
@@ -2061,9 +2141,14 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
* be changed, so we can't move the buf_finish() to __vb2_dqbuf().
*/
for (i = 0; i < q->num_buffers; ++i) {
- struct vb2_buffer *vb = q->bufs[i];
- struct media_request *req = vb->req_obj.req;
+ struct vb2_buffer *vb;
+ struct media_request *req;
+ vb = vb2_get_buffer(q, i);
+ if (!vb)
+ continue;
+
+ req = vb->req_obj.req;
/*
* If a request is associated with this buffer, then
* call buf_request_cancel() to give the driver to complete()
@@ -2215,7 +2300,10 @@ static int __find_plane_by_offset(struct vb2_queue *q, unsigned long off,
* return its buffer and plane numbers.
*/
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
+
+ if (!vb)
+ continue;
for (plane = 0; plane < vb->num_planes; ++plane) {
if (vb->planes[plane].m.offset == off) {
@@ -2262,7 +2350,12 @@ int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
return -EINVAL;
}
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
if (plane >= vb->num_planes) {
dprintk(q, 1, "buffer plane out of range\n");
@@ -2339,7 +2432,13 @@ int vb2_mmap(struct vb2_queue *q, struct vm_area_struct *vma)
if (ret)
goto unlock;
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ ret = -EINVAL;
+ goto unlock;
+ }
/*
* MMAP requires page_aligned buffers.
@@ -2396,7 +2495,12 @@ unsigned long vb2_get_unmapped_area(struct vb2_queue *q,
if (ret)
goto unlock;
- vb = q->bufs[buffer];
+ vb = vb2_get_buffer(q, buffer);
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ ret = -EINVAL;
+ goto unlock;
+ }
vaddr = vb2_plane_vaddr(vb, plane);
mutex_unlock(&q->mmap_lock);
@@ -2625,6 +2729,7 @@ struct vb2_fileio_data {
static int __vb2_init_fileio(struct vb2_queue *q, int read)
{
struct vb2_fileio_data *fileio;
+ struct vb2_buffer *vb;
int i, ret;
unsigned int count = 0;
@@ -2679,7 +2784,13 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
* Check if plane_count is correct
* (multiplane buffers are not supported).
*/
- if (q->bufs[0]->num_planes != 1) {
+ vb = vb2_get_buffer(q, 0);
+ if (!vb) {
+ ret = -EBUSY;
+ goto err_reqbufs;
+ }
+
+ if (vb->num_planes != 1) {
ret = -EBUSY;
goto err_reqbufs;
}
@@ -2688,12 +2799,17 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
* Get kernel address of each buffer.
*/
for (i = 0; i < q->num_buffers; i++) {
- fileio->bufs[i].vaddr = vb2_plane_vaddr(q->bufs[i], 0);
+ vb = vb2_get_buffer(q, i);
+
+ if (!vb)
+ continue;
+
+ fileio->bufs[i].vaddr = vb2_plane_vaddr(vb, 0);
if (fileio->bufs[i].vaddr == NULL) {
ret = -EINVAL;
goto err_reqbufs;
}
- fileio->bufs[i].size = vb2_plane_size(q->bufs[i], 0);
+ fileio->bufs[i].size = vb2_plane_size(vb, 0);
}
/*
@@ -2821,15 +2937,18 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
fileio->cur_index = index;
buf = &fileio->bufs[index];
- b = q->bufs[index];
+ b = vb2_get_buffer(q, index);
+
+ if (!b)
+ return -EINVAL;
/*
* Get number of bytes filled by the driver
*/
buf->pos = 0;
buf->queued = 0;
- buf->size = read ? vb2_get_plane_payload(q->bufs[index], 0)
- : vb2_plane_size(q->bufs[index], 0);
+ buf->size = read ? vb2_get_plane_payload(b, 0)
+ : vb2_plane_size(b, 0);
/* Compensate for data_offset on read in the multiplanar case. */
if (is_multiplanar && read &&
b->planes[0].data_offset < buf->size) {
@@ -2872,8 +2991,12 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
* Queue next buffer if required.
*/
if (buf->pos == buf->size || (!read && fileio->write_immediately)) {
- struct vb2_buffer *b = q->bufs[index];
+ struct vb2_buffer *b = vb2_get_buffer(q, index);
+ if (!b) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
/*
* Check if this is the last buffer to read.
*/
@@ -2899,7 +3022,7 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
*/
buf->pos = 0;
buf->queued = 1;
- buf->size = vb2_plane_size(q->bufs[index], 0);
+ buf->size = vb2_plane_size(b, 0);
fileio->q_count += 1;
/*
* If we are queuing up buffers for the first time, then
@@ -2970,7 +3093,9 @@ static int vb2_thread(void *data)
* Call vb2_dqbuf to get buffer back.
*/
if (prequeue) {
- vb = q->bufs[index++];
+ vb = vb2_get_buffer(q, index++);
+ if (!vb)
+ continue;
prequeue--;
} else {
call_void_qop(q, wait_finish, q);
@@ -2979,7 +3104,7 @@ static int vb2_thread(void *data)
call_void_qop(q, wait_prepare, q);
dprintk(q, 5, "file io: vb2_dqbuf result: %d\n", ret);
if (!ret)
- vb = q->bufs[index];
+ vb = vb2_get_buffer(q, index);
}
if (ret || threadio->stop)
break;
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index c7a54d82a55e..724135d41f7f 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -383,8 +383,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
return -EINVAL;
}
- if (q->bufs[b->index] == NULL) {
- /* Should never happen */
+ if (!vb2_get_buffer(q, b->index)) {
dprintk(q, 1, "%s: buffer is NULL\n", opname);
return -EINVAL;
}
@@ -394,7 +393,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
return -EINVAL;
}
- vb = q->bufs[b->index];
+ vb = vb2_get_buffer(q, b->index);
vbuf = to_vb2_v4l2_buffer(vb);
ret = __verify_planes_array(vb, b);
if (ret)
@@ -628,11 +627,18 @@ static const struct vb2_buf_ops v4l2_buf_ops = {
struct vb2_buffer *vb2_find_buffer(struct vb2_queue *q, u64 timestamp)
{
unsigned int i;
+ struct vb2_buffer *vb2;
- for (i = 0; i < q->num_buffers; i++)
- if (q->bufs[i]->copied_timestamp &&
- q->bufs[i]->timestamp == timestamp)
- return vb2_get_buffer(q, i);
+ for (i = 0; i < q->num_buffers; i++) {
+ vb2 = vb2_get_buffer(q, i);
+
+ if (!vb2)
+ continue;
+
+ if (vb2->copied_timestamp &&
+ vb2->timestamp == timestamp)
+ return vb2;
+ }
return NULL;
}
EXPORT_SYMBOL_GPL(vb2_find_buffer);
@@ -664,7 +670,13 @@ int vb2_querybuf(struct vb2_queue *q, struct v4l2_buffer *b)
dprintk(q, 1, "buffer index out of range\n");
return -EINVAL;
}
- vb = q->bufs[b->index];
+ vb = vb2_get_buffer(q, b->index);
+
+ if (!vb) {
+ dprintk(q, 1, "can't find the requested buffer\n");
+ return -EINVAL;
+ }
+
ret = __verify_planes_array(vb, b);
if (!ret)
vb2_core_querybuf(q, b->index, b);
diff --git a/drivers/media/platform/amphion/vpu_dbg.c b/drivers/media/platform/amphion/vpu_dbg.c
index 44b830ae01d8..e2c371c2fde5 100644
--- a/drivers/media/platform/amphion/vpu_dbg.c
+++ b/drivers/media/platform/amphion/vpu_dbg.c
@@ -133,11 +133,18 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
vq = v4l2_m2m_get_src_vq(inst->fh.m2m_ctx);
for (i = 0; i < vq->num_buffers; i++) {
- struct vb2_buffer *vb = vq->bufs[i];
- struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vb2_buffer *vb;
+ struct vb2_v4l2_buffer *vbuf;
+
+ vb = vb2_get_buffer(vq, i);
+ if (!vb)
+ continue;
if (vb->state == VB2_BUF_STATE_DEQUEUED)
continue;
+
+ vbuf = to_vb2_v4l2_buffer(vb);
+
num = scnprintf(str, sizeof(str),
"output [%2d] state = %10s, %8s\n",
i, vb2_stat_name[vb->state],
@@ -148,11 +155,18 @@ static int vpu_dbg_instance(struct seq_file *s, void *data)
vq = v4l2_m2m_get_dst_vq(inst->fh.m2m_ctx);
for (i = 0; i < vq->num_buffers; i++) {
- struct vb2_buffer *vb = vq->bufs[i];
- struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vb2_buffer *vb;
+ struct vb2_v4l2_buffer *vbuf;
+
+ vb = vb2_get_buffer(vq, i);
+ if (!vb)
+ continue;
if (vb->state == VB2_BUF_STATE_DEQUEUED)
continue;
+
+ vbuf = to_vb2_v4l2_buffer(vb);
+
num = scnprintf(str, sizeof(str),
"capture[%2d] state = %10s, %8s\n",
i, vb2_stat_name[vb->state],
diff --git a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
index 0051f372a66c..ea37069ba355 100644
--- a/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
+++ b/drivers/media/platform/mediatek/jpeg/mtk_jpeg_core.c
@@ -603,7 +603,11 @@ static int mtk_jpeg_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
return -EINVAL;
}
- vb = vq->bufs[buf->index];
+ vb = vb2_get_buffer(vq, buf->index);
+ if (!vb) {
+ dev_err(ctx->jpeg->dev, "buffer not found\n");
+ return -EINVAL;
+ }
jpeg_src_buf = mtk_jpeg_vb2_to_srcbuf(vb);
jpeg_src_buf->bs_size = buf->m.planes[0].bytesused;
diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
index cf16cf2807f0..6532a69f1fa8 100644
--- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
+++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
@@ -1701,7 +1701,7 @@ static int vdec_vp9_slice_setup_core_buffer(struct vdec_vp9_slice_instance *inst
/* update internal buffer's width/height */
for (i = 0; i < vq->num_buffers; i++) {
- if (vb == vq->bufs[i]) {
+ if (vb == vb2_get_buffer(vq, i)) {
instance->dpb[i].width = w;
instance->dpb[i].height = h;
break;
diff --git a/drivers/media/platform/st/sti/hva/hva-v4l2.c b/drivers/media/platform/st/sti/hva/hva-v4l2.c
index 3a848ca32a0e..326be09bdb55 100644
--- a/drivers/media/platform/st/sti/hva/hva-v4l2.c
+++ b/drivers/media/platform/st/sti/hva/hva-v4l2.c
@@ -577,6 +577,10 @@ static int hva_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
}
vb2_buf = vb2_get_buffer(vq, buf->index);
+ if (!vb2_buf) {
+ dev_dbg(dev, "%s buffer index %d not found\n", ctx->name, buf->index);
+ return -EINVAL;
+ }
stream = to_hva_stream(to_vb2_v4l2_buffer(vb2_buf));
stream->bytesused = buf->bytesused;
}
diff --git a/drivers/media/test-drivers/visl/visl-dec.c b/drivers/media/test-drivers/visl/visl-dec.c
index 318d675e5668..ba20ea998d19 100644
--- a/drivers/media/test-drivers/visl/visl-dec.c
+++ b/drivers/media/test-drivers/visl/visl-dec.c
@@ -290,13 +290,20 @@ static void visl_tpg_fill(struct visl_ctx *ctx, struct visl_run *run)
for (i = 0; i < out_q->num_buffers; i++) {
char entry[] = "index: %u, state: %s, request_fd: %d, ";
u32 old_len = len;
- char *q_status = visl_get_vb2_state(out_q->bufs[i]->state);
+ struct vb2_buffer *vb2;
+ char *q_status;
+
+ vb2 = vb2_get_buffer(out_q, i);
+ if (!vb2)
+ continue;
+
+ q_status = visl_get_vb2_state(vb2->state);
len += scnprintf(&buf[len], TPG_STR_BUF_SZ - len,
entry, i, q_status,
- to_vb2_v4l2_buffer(out_q->bufs[i])->request_fd);
+ to_vb2_v4l2_buffer(vb2)->request_fd);
- len += visl_fill_bytesused(to_vb2_v4l2_buffer(out_q->bufs[i]),
+ len += visl_fill_bytesused(to_vb2_v4l2_buffer(vb2),
&buf[len],
TPG_STR_BUF_SZ - len);
@@ -342,13 +349,20 @@ static void visl_tpg_fill(struct visl_ctx *ctx, struct visl_run *run)
len = 0;
for (i = 0; i < cap_q->num_buffers; i++) {
u32 old_len = len;
- char *q_status = visl_get_vb2_state(cap_q->bufs[i]->state);
+ struct vb2_buffer *vb2;
+ char *q_status;
+
+ vb2 = vb2_get_buffer(cap_q, i);
+ if (!vb2)
+ continue;
+
+ q_status = visl_get_vb2_state(vb2->state);
len += scnprintf(&buf[len], TPG_STR_BUF_SZ - len,
"index: %u, status: %s, timestamp: %llu, is_held: %d",
- cap_q->bufs[i]->index, q_status,
- cap_q->bufs[i]->timestamp,
- to_vb2_v4l2_buffer(cap_q->bufs[i])->is_held);
+ vb2->index, q_status,
+ vb2->timestamp,
+ to_vb2_v4l2_buffer(vb2)->is_held);
tpg_gen_text(&ctx->tpg, basep, line++ * line_height, 16, &buf[old_len]);
frame_dprintk(ctx->dev, run->dst->sequence, "%s", &buf[old_len]);
diff --git a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
index 384f31fc66c5..ec7e05263c8e 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_ioctl.c
@@ -1077,7 +1077,7 @@ static int atomisp_dqbuf_wrapper(struct file *file, void *fh, struct v4l2_buffer
if (ret)
return ret;
- vb = pipe->vb_queue.bufs[buf->index];
+ vb = vb2_get_buffer(pipe->vb_queue, buf->index);
frame = vb_to_frame(vb);
buf->reserved = asd->frame_status[buf->index];
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 183+ messages in thread* Re: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-23 15:10 ` kernel test robot
-1 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-23 15:10 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7 next-20230623]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-2-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
config: i386-allyesconfig (https://download.01.org/0day-ci/archive/20230623/202306232218.dOcWboPB-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230623/202306232218.dOcWboPB-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306232218.dOcWboPB-lkp@intel.com/
All errors (new ones prefixed by >>):
drivers/staging/media/atomisp/pci/atomisp_ioctl.c: In function 'atomisp_dqbuf_wrapper':
>> drivers/staging/media/atomisp/pci/atomisp_ioctl.c:1080:33: error: incompatible type for argument 1 of 'vb2_get_buffer'
1080 | vb = vb2_get_buffer(pipe->vb_queue, buf->index);
| ~~~~^~~~~~~~~~
| |
| struct vb2_queue
In file included from include/media/videobuf2-v4l2.h:16,
from drivers/staging/media/atomisp//pci/ia_css_frame_public.h:23,
from drivers/staging/media/atomisp/pci/sh_css_legacy.h:22,
from drivers/staging/media/atomisp/pci/atomisp_internal.h:34,
from drivers/staging/media/atomisp/pci/atomisp_cmd.h:30,
from drivers/staging/media/atomisp/pci/atomisp_ioctl.c:27:
include/media/videobuf2-core.h:1239:67: note: expected 'struct vb2_queue *' but argument is of type 'struct vb2_queue'
1239 | static inline struct vb2_buffer *vb2_get_buffer(struct vb2_queue *q,
| ~~~~~~~~~~~~~~~~~~^
vim +/vb2_get_buffer +1080 drivers/staging/media/atomisp/pci/atomisp_ioctl.c
1065
1066 static int atomisp_dqbuf_wrapper(struct file *file, void *fh, struct v4l2_buffer *buf)
1067 {
1068 struct video_device *vdev = video_devdata(file);
1069 struct atomisp_video_pipe *pipe = atomisp_to_video_pipe(vdev);
1070 struct atomisp_sub_device *asd = pipe->asd;
1071 struct atomisp_device *isp = video_get_drvdata(vdev);
1072 struct ia_css_frame *frame;
1073 struct vb2_buffer *vb;
1074 int ret;
1075
1076 ret = vb2_ioctl_dqbuf(file, fh, buf);
1077 if (ret)
1078 return ret;
1079
> 1080 vb = vb2_get_buffer(pipe->vb_queue, buf->index);
1081 frame = vb_to_frame(vb);
1082
1083 buf->reserved = asd->frame_status[buf->index];
1084
1085 /*
1086 * Hack:
1087 * Currently frame_status in the enum type which takes no more lower
1088 * 8 bit.
1089 * use bit[31:16] for exp_id as it is only in the range of 1~255
1090 */
1091 buf->reserved &= 0x0000ffff;
1092 if (!(buf->flags & V4L2_BUF_FLAG_ERROR))
1093 buf->reserved |= frame->exp_id;
1094 buf->reserved2 = pipe->frame_config_id[buf->index];
1095
1096 dev_dbg(isp->dev,
1097 "dqbuf buffer %d (%s) with exp_id %d, isp_config_id %d\n",
1098 buf->index, vdev->name, buf->reserved >> 16, buf->reserved2);
1099 return 0;
1100 }
1101
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
@ 2023-06-23 15:10 ` kernel test robot
0 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-23 15:10 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7 next-20230623]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-2-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
config: i386-allyesconfig (https://download.01.org/0day-ci/archive/20230623/202306232218.dOcWboPB-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230623/202306232218.dOcWboPB-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306232218.dOcWboPB-lkp@intel.com/
All errors (new ones prefixed by >>):
drivers/staging/media/atomisp/pci/atomisp_ioctl.c: In function 'atomisp_dqbuf_wrapper':
>> drivers/staging/media/atomisp/pci/atomisp_ioctl.c:1080:33: error: incompatible type for argument 1 of 'vb2_get_buffer'
1080 | vb = vb2_get_buffer(pipe->vb_queue, buf->index);
| ~~~~^~~~~~~~~~
| |
| struct vb2_queue
In file included from include/media/videobuf2-v4l2.h:16,
from drivers/staging/media/atomisp//pci/ia_css_frame_public.h:23,
from drivers/staging/media/atomisp/pci/sh_css_legacy.h:22,
from drivers/staging/media/atomisp/pci/atomisp_internal.h:34,
from drivers/staging/media/atomisp/pci/atomisp_cmd.h:30,
from drivers/staging/media/atomisp/pci/atomisp_ioctl.c:27:
include/media/videobuf2-core.h:1239:67: note: expected 'struct vb2_queue *' but argument is of type 'struct vb2_queue'
1239 | static inline struct vb2_buffer *vb2_get_buffer(struct vb2_queue *q,
| ~~~~~~~~~~~~~~~~~~^
vim +/vb2_get_buffer +1080 drivers/staging/media/atomisp/pci/atomisp_ioctl.c
1065
1066 static int atomisp_dqbuf_wrapper(struct file *file, void *fh, struct v4l2_buffer *buf)
1067 {
1068 struct video_device *vdev = video_devdata(file);
1069 struct atomisp_video_pipe *pipe = atomisp_to_video_pipe(vdev);
1070 struct atomisp_sub_device *asd = pipe->asd;
1071 struct atomisp_device *isp = video_get_drvdata(vdev);
1072 struct ia_css_frame *frame;
1073 struct vb2_buffer *vb;
1074 int ret;
1075
1076 ret = vb2_ioctl_dqbuf(file, fh, buf);
1077 if (ret)
1078 return ret;
1079
> 1080 vb = vb2_get_buffer(pipe->vb_queue, buf->index);
1081 frame = vb_to_frame(vb);
1082
1083 buf->reserved = asd->frame_status[buf->index];
1084
1085 /*
1086 * Hack:
1087 * Currently frame_status in the enum type which takes no more lower
1088 * 8 bit.
1089 * use bit[31:16] for exp_id as it is only in the range of 1~255
1090 */
1091 buf->reserved &= 0x0000ffff;
1092 if (!(buf->flags & V4L2_BUF_FLAG_ERROR))
1093 buf->reserved |= frame->exp_id;
1094 buf->reserved2 = pipe->frame_config_id[buf->index];
1095
1096 dev_dbg(isp->dev,
1097 "dqbuf buffer %d (%s) with exp_id %d, isp_config_id %d\n",
1098 buf->index, vdev->name, buf->reserved >> 16, buf->reserved2);
1099 return 0;
1100 }
1101
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
@ 2023-06-23 15:10 ` kernel test robot
0 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-23 15:10 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7 next-20230623]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-2-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
config: i386-allyesconfig (https://download.01.org/0day-ci/archive/20230623/202306232218.dOcWboPB-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230623/202306232218.dOcWboPB-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306232218.dOcWboPB-lkp@intel.com/
All errors (new ones prefixed by >>):
drivers/staging/media/atomisp/pci/atomisp_ioctl.c: In function 'atomisp_dqbuf_wrapper':
>> drivers/staging/media/atomisp/pci/atomisp_ioctl.c:1080:33: error: incompatible type for argument 1 of 'vb2_get_buffer'
1080 | vb = vb2_get_buffer(pipe->vb_queue, buf->index);
| ~~~~^~~~~~~~~~
| |
| struct vb2_queue
In file included from include/media/videobuf2-v4l2.h:16,
from drivers/staging/media/atomisp//pci/ia_css_frame_public.h:23,
from drivers/staging/media/atomisp/pci/sh_css_legacy.h:22,
from drivers/staging/media/atomisp/pci/atomisp_internal.h:34,
from drivers/staging/media/atomisp/pci/atomisp_cmd.h:30,
from drivers/staging/media/atomisp/pci/atomisp_ioctl.c:27:
include/media/videobuf2-core.h:1239:67: note: expected 'struct vb2_queue *' but argument is of type 'struct vb2_queue'
1239 | static inline struct vb2_buffer *vb2_get_buffer(struct vb2_queue *q,
| ~~~~~~~~~~~~~~~~~~^
vim +/vb2_get_buffer +1080 drivers/staging/media/atomisp/pci/atomisp_ioctl.c
1065
1066 static int atomisp_dqbuf_wrapper(struct file *file, void *fh, struct v4l2_buffer *buf)
1067 {
1068 struct video_device *vdev = video_devdata(file);
1069 struct atomisp_video_pipe *pipe = atomisp_to_video_pipe(vdev);
1070 struct atomisp_sub_device *asd = pipe->asd;
1071 struct atomisp_device *isp = video_get_drvdata(vdev);
1072 struct ia_css_frame *frame;
1073 struct vb2_buffer *vb;
1074 int ret;
1075
1076 ret = vb2_ioctl_dqbuf(file, fh, buf);
1077 if (ret)
1078 return ret;
1079
> 1080 vb = vb2_get_buffer(pipe->vb_queue, buf->index);
1081 frame = vb_to_frame(vb);
1082
1083 buf->reserved = asd->frame_status[buf->index];
1084
1085 /*
1086 * Hack:
1087 * Currently frame_status in the enum type which takes no more lower
1088 * 8 bit.
1089 * use bit[31:16] for exp_id as it is only in the range of 1~255
1090 */
1091 buf->reserved &= 0x0000ffff;
1092 if (!(buf->flags & V4L2_BUF_FLAG_ERROR))
1093 buf->reserved |= frame->exp_id;
1094 buf->reserved2 = pipe->frame_config_id[buf->index];
1095
1096 dev_dbg(isp->dev,
1097 "dqbuf buffer %d (%s) with exp_id %d, isp_config_id %d\n",
1098 buf->index, vdev->name, buf->reserved >> 16, buf->reserved2);
1099 return 0;
1100 }
1101
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-24 23:34 ` kernel test robot
-1 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-24 23:34 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: llvm, oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7 next-20230623]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-2-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
config: i386-allyesconfig (https://download.01.org/0day-ci/archive/20230625/202306250705.Ts4zHhMG-lkp@intel.com/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project.git 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce: (https://download.01.org/0day-ci/archive/20230625/202306250705.Ts4zHhMG-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306250705.Ts4zHhMG-lkp@intel.com/
All errors (new ones prefixed by >>):
>> drivers/staging/media/atomisp/pci/atomisp_ioctl.c:1080:22: error: passing 'struct vb2_queue' to parameter of incompatible type 'struct vb2_queue *'; take the address with &
vb = vb2_get_buffer(pipe->vb_queue, buf->index);
^~~~~~~~~~~~~~
&
include/media/videobuf2-core.h:1239:67: note: passing argument to parameter 'q' here
static inline struct vb2_buffer *vb2_get_buffer(struct vb2_queue *q,
^
1 error generated.
vim +1080 drivers/staging/media/atomisp/pci/atomisp_ioctl.c
1065
1066 static int atomisp_dqbuf_wrapper(struct file *file, void *fh, struct v4l2_buffer *buf)
1067 {
1068 struct video_device *vdev = video_devdata(file);
1069 struct atomisp_video_pipe *pipe = atomisp_to_video_pipe(vdev);
1070 struct atomisp_sub_device *asd = pipe->asd;
1071 struct atomisp_device *isp = video_get_drvdata(vdev);
1072 struct ia_css_frame *frame;
1073 struct vb2_buffer *vb;
1074 int ret;
1075
1076 ret = vb2_ioctl_dqbuf(file, fh, buf);
1077 if (ret)
1078 return ret;
1079
> 1080 vb = vb2_get_buffer(pipe->vb_queue, buf->index);
1081 frame = vb_to_frame(vb);
1082
1083 buf->reserved = asd->frame_status[buf->index];
1084
1085 /*
1086 * Hack:
1087 * Currently frame_status in the enum type which takes no more lower
1088 * 8 bit.
1089 * use bit[31:16] for exp_id as it is only in the range of 1~255
1090 */
1091 buf->reserved &= 0x0000ffff;
1092 if (!(buf->flags & V4L2_BUF_FLAG_ERROR))
1093 buf->reserved |= frame->exp_id;
1094 buf->reserved2 = pipe->frame_config_id[buf->index];
1095
1096 dev_dbg(isp->dev,
1097 "dqbuf buffer %d (%s) with exp_id %d, isp_config_id %d\n",
1098 buf->index, vdev->name, buf->reserved >> 16, buf->reserved2);
1099 return 0;
1100 }
1101
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
@ 2023-06-24 23:34 ` kernel test robot
0 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-24 23:34 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: llvm, oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7 next-20230623]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-2-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
config: i386-allyesconfig (https://download.01.org/0day-ci/archive/20230625/202306250705.Ts4zHhMG-lkp@intel.com/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project.git 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce: (https://download.01.org/0day-ci/archive/20230625/202306250705.Ts4zHhMG-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306250705.Ts4zHhMG-lkp@intel.com/
All errors (new ones prefixed by >>):
>> drivers/staging/media/atomisp/pci/atomisp_ioctl.c:1080:22: error: passing 'struct vb2_queue' to parameter of incompatible type 'struct vb2_queue *'; take the address with &
vb = vb2_get_buffer(pipe->vb_queue, buf->index);
^~~~~~~~~~~~~~
&
include/media/videobuf2-core.h:1239:67: note: passing argument to parameter 'q' here
static inline struct vb2_buffer *vb2_get_buffer(struct vb2_queue *q,
^
1 error generated.
vim +1080 drivers/staging/media/atomisp/pci/atomisp_ioctl.c
1065
1066 static int atomisp_dqbuf_wrapper(struct file *file, void *fh, struct v4l2_buffer *buf)
1067 {
1068 struct video_device *vdev = video_devdata(file);
1069 struct atomisp_video_pipe *pipe = atomisp_to_video_pipe(vdev);
1070 struct atomisp_sub_device *asd = pipe->asd;
1071 struct atomisp_device *isp = video_get_drvdata(vdev);
1072 struct ia_css_frame *frame;
1073 struct vb2_buffer *vb;
1074 int ret;
1075
1076 ret = vb2_ioctl_dqbuf(file, fh, buf);
1077 if (ret)
1078 return ret;
1079
> 1080 vb = vb2_get_buffer(pipe->vb_queue, buf->index);
1081 frame = vb_to_frame(vb);
1082
1083 buf->reserved = asd->frame_status[buf->index];
1084
1085 /*
1086 * Hack:
1087 * Currently frame_status in the enum type which takes no more lower
1088 * 8 bit.
1089 * use bit[31:16] for exp_id as it is only in the range of 1~255
1090 */
1091 buf->reserved &= 0x0000ffff;
1092 if (!(buf->flags & V4L2_BUF_FLAG_ERROR))
1093 buf->reserved |= frame->exp_id;
1094 buf->reserved2 = pipe->frame_config_id[buf->index];
1095
1096 dev_dbg(isp->dev,
1097 "dqbuf buffer %d (%s) with exp_id %d, isp_config_id %d\n",
1098 buf->index, vdev->name, buf->reserved >> 16, buf->reserved2);
1099 return 0;
1100 }
1101
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
@ 2023-06-24 23:34 ` kernel test robot
0 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-24 23:34 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: llvm, oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7 next-20230623]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-2-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 01/11] media: videobuf2: Access vb2_queue bufs array through helper functions
config: i386-allyesconfig (https://download.01.org/0day-ci/archive/20230625/202306250705.Ts4zHhMG-lkp@intel.com/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project.git 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce: (https://download.01.org/0day-ci/archive/20230625/202306250705.Ts4zHhMG-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306250705.Ts4zHhMG-lkp@intel.com/
All errors (new ones prefixed by >>):
>> drivers/staging/media/atomisp/pci/atomisp_ioctl.c:1080:22: error: passing 'struct vb2_queue' to parameter of incompatible type 'struct vb2_queue *'; take the address with &
vb = vb2_get_buffer(pipe->vb_queue, buf->index);
^~~~~~~~~~~~~~
&
include/media/videobuf2-core.h:1239:67: note: passing argument to parameter 'q' here
static inline struct vb2_buffer *vb2_get_buffer(struct vb2_queue *q,
^
1 error generated.
vim +1080 drivers/staging/media/atomisp/pci/atomisp_ioctl.c
1065
1066 static int atomisp_dqbuf_wrapper(struct file *file, void *fh, struct v4l2_buffer *buf)
1067 {
1068 struct video_device *vdev = video_devdata(file);
1069 struct atomisp_video_pipe *pipe = atomisp_to_video_pipe(vdev);
1070 struct atomisp_sub_device *asd = pipe->asd;
1071 struct atomisp_device *isp = video_get_drvdata(vdev);
1072 struct ia_css_frame *frame;
1073 struct vb2_buffer *vb;
1074 int ret;
1075
1076 ret = vb2_ioctl_dqbuf(file, fh, buf);
1077 if (ret)
1078 return ret;
1079
> 1080 vb = vb2_get_buffer(pipe->vb_queue, buf->index);
1081 frame = vb_to_frame(vb);
1082
1083 buf->reserved = asd->frame_status[buf->index];
1084
1085 /*
1086 * Hack:
1087 * Currently frame_status in the enum type which takes no more lower
1088 * 8 bit.
1089 * use bit[31:16] for exp_id as it is only in the range of 1~255
1090 */
1091 buf->reserved &= 0x0000ffff;
1092 if (!(buf->flags & V4L2_BUF_FLAG_ERROR))
1093 buf->reserved |= frame->exp_id;
1094 buf->reserved2 = pipe->frame_config_id[buf->index];
1095
1096 dev_dbg(isp->dev,
1097 "dqbuf buffer %d (%s) with exp_id %d, isp_config_id %d\n",
1098 buf->index, vdev->name, buf->reserved >> 16, buf->reserved2);
1099 return 0;
1100 }
1101
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* [PATCH v3 02/11] media: videobuf2: Use Xarray instead of static buffers array
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Instead of a static array change bufs to a dynamically allocated array.
This will allow to store more video buffers if needed.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/common/videobuf2/videobuf2-core.c | 38 +++++++++++--------
include/media/videobuf2-core.h | 6 +--
2 files changed, 25 insertions(+), 19 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 42fd3984c2bc..f1ff7af34a9f 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -409,18 +409,24 @@ static void init_buffer_cache_hints(struct vb2_queue *q, struct vb2_buffer *vb)
* vb2_queue_add_buffer() - add a buffer to a queue
* @q: pointer to &struct vb2_queue with videobuf2 queue.
* @vb: pointer to &struct vb2_buffer to be added to the queue.
- * @index: index where add vb2_buffer in the queue
*/
-static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb, int index)
+static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb)
{
- if (index < VB2_MAX_FRAME && !q->bufs[index]) {
- q->bufs[index] = vb;
- vb->index = index;
- vb->vb2_queue = q;
- return true;
- }
+ struct xa_limit range = {
+ .max = UINT_MAX,
+ .min = q->num_buffers,
+ };
+ u32 index;
+ int ret;
- return false;
+ ret = xa_alloc(&q->bufs, &index, vb, range, GFP_KERNEL);
+ if (ret)
+ return false;
+
+ vb->index = index;
+ vb->vb2_queue = q;
+
+ return true;
}
/**
@@ -430,10 +436,8 @@ static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb, int
*/
static void vb2_queue_remove_buffer(struct vb2_queue *q, struct vb2_buffer *vb)
{
- if (vb->index < VB2_MAX_FRAME) {
- q->bufs[vb->index] = NULL;
- vb->vb2_queue = NULL;
- }
+ xa_erase(&q->bufs, vb->index);
+ vb->vb2_queue = NULL;
}
/*
@@ -474,7 +478,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
}
call_void_bufop(q, init_buffer, vb);
- if (!vb2_queue_add_buffer(q, vb, q->num_buffers + buffer)) {
+ if (!vb2_queue_add_buffer(q, vb)) {
dprintk(q, 1, "failed adding buffer %d to queue\n", buffer);
kfree(vb);
break;
@@ -930,7 +934,7 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
}
mutex_lock(&q->mmap_lock);
- q->num_buffers = allocated_buffers;
+ q->num_buffers += allocated_buffers;
if (ret < 0) {
/*
@@ -2547,6 +2551,9 @@ int vb2_core_queue_init(struct vb2_queue *q)
mutex_init(&q->mmap_lock);
init_waitqueue_head(&q->done_wq);
+ xa_init_flags(&q->bufs, XA_FLAGS_ALLOC);
+ q->num_buffers = 0;
+
q->memory = VB2_MEMORY_UNKNOWN;
if (q->buf_struct_size == 0)
@@ -2574,6 +2581,7 @@ void vb2_core_queue_release(struct vb2_queue *q)
mutex_lock(&q->mmap_lock);
__vb2_queue_free(q, q->num_buffers);
mutex_unlock(&q->mmap_lock);
+ xa_destroy(&q->bufs);
}
EXPORT_SYMBOL_GPL(vb2_core_queue_release);
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 4b6a9d2ea372..77921cf894ef 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -619,7 +619,7 @@ struct vb2_queue {
struct mutex mmap_lock;
unsigned int memory;
enum dma_data_direction dma_dir;
- struct vb2_buffer *bufs[VB2_MAX_FRAME];
+ struct xarray bufs;
unsigned int num_buffers;
struct list_head queued_list;
@@ -1239,9 +1239,7 @@ static inline void vb2_clear_last_buffer_dequeued(struct vb2_queue *q)
static inline struct vb2_buffer *vb2_get_buffer(struct vb2_queue *q,
unsigned int index)
{
- if (index < q->num_buffers)
- return q->bufs[index];
- return NULL;
+ return xa_load(&q->bufs, index);
}
/*
--
2.39.2
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 02/11] media: videobuf2: Use Xarray instead of static buffers array
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Instead of a static array change bufs to a dynamically allocated array.
This will allow to store more video buffers if needed.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/common/videobuf2/videobuf2-core.c | 38 +++++++++++--------
include/media/videobuf2-core.h | 6 +--
2 files changed, 25 insertions(+), 19 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 42fd3984c2bc..f1ff7af34a9f 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -409,18 +409,24 @@ static void init_buffer_cache_hints(struct vb2_queue *q, struct vb2_buffer *vb)
* vb2_queue_add_buffer() - add a buffer to a queue
* @q: pointer to &struct vb2_queue with videobuf2 queue.
* @vb: pointer to &struct vb2_buffer to be added to the queue.
- * @index: index where add vb2_buffer in the queue
*/
-static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb, int index)
+static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb)
{
- if (index < VB2_MAX_FRAME && !q->bufs[index]) {
- q->bufs[index] = vb;
- vb->index = index;
- vb->vb2_queue = q;
- return true;
- }
+ struct xa_limit range = {
+ .max = UINT_MAX,
+ .min = q->num_buffers,
+ };
+ u32 index;
+ int ret;
- return false;
+ ret = xa_alloc(&q->bufs, &index, vb, range, GFP_KERNEL);
+ if (ret)
+ return false;
+
+ vb->index = index;
+ vb->vb2_queue = q;
+
+ return true;
}
/**
@@ -430,10 +436,8 @@ static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb, int
*/
static void vb2_queue_remove_buffer(struct vb2_queue *q, struct vb2_buffer *vb)
{
- if (vb->index < VB2_MAX_FRAME) {
- q->bufs[vb->index] = NULL;
- vb->vb2_queue = NULL;
- }
+ xa_erase(&q->bufs, vb->index);
+ vb->vb2_queue = NULL;
}
/*
@@ -474,7 +478,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
}
call_void_bufop(q, init_buffer, vb);
- if (!vb2_queue_add_buffer(q, vb, q->num_buffers + buffer)) {
+ if (!vb2_queue_add_buffer(q, vb)) {
dprintk(q, 1, "failed adding buffer %d to queue\n", buffer);
kfree(vb);
break;
@@ -930,7 +934,7 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
}
mutex_lock(&q->mmap_lock);
- q->num_buffers = allocated_buffers;
+ q->num_buffers += allocated_buffers;
if (ret < 0) {
/*
@@ -2547,6 +2551,9 @@ int vb2_core_queue_init(struct vb2_queue *q)
mutex_init(&q->mmap_lock);
init_waitqueue_head(&q->done_wq);
+ xa_init_flags(&q->bufs, XA_FLAGS_ALLOC);
+ q->num_buffers = 0;
+
q->memory = VB2_MEMORY_UNKNOWN;
if (q->buf_struct_size == 0)
@@ -2574,6 +2581,7 @@ void vb2_core_queue_release(struct vb2_queue *q)
mutex_lock(&q->mmap_lock);
__vb2_queue_free(q, q->num_buffers);
mutex_unlock(&q->mmap_lock);
+ xa_destroy(&q->bufs);
}
EXPORT_SYMBOL_GPL(vb2_core_queue_release);
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 4b6a9d2ea372..77921cf894ef 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -619,7 +619,7 @@ struct vb2_queue {
struct mutex mmap_lock;
unsigned int memory;
enum dma_data_direction dma_dir;
- struct vb2_buffer *bufs[VB2_MAX_FRAME];
+ struct xarray bufs;
unsigned int num_buffers;
struct list_head queued_list;
@@ -1239,9 +1239,7 @@ static inline void vb2_clear_last_buffer_dequeued(struct vb2_queue *q)
static inline struct vb2_buffer *vb2_get_buffer(struct vb2_queue *q,
unsigned int index)
{
- if (index < q->num_buffers)
- return q->bufs[index];
- return NULL;
+ return xa_load(&q->bufs, index);
}
/*
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 02/11] media: videobuf2: Use Xarray instead of static buffers array
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Instead of a static array change bufs to a dynamically allocated array.
This will allow to store more video buffers if needed.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/common/videobuf2/videobuf2-core.c | 38 +++++++++++--------
include/media/videobuf2-core.h | 6 +--
2 files changed, 25 insertions(+), 19 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 42fd3984c2bc..f1ff7af34a9f 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -409,18 +409,24 @@ static void init_buffer_cache_hints(struct vb2_queue *q, struct vb2_buffer *vb)
* vb2_queue_add_buffer() - add a buffer to a queue
* @q: pointer to &struct vb2_queue with videobuf2 queue.
* @vb: pointer to &struct vb2_buffer to be added to the queue.
- * @index: index where add vb2_buffer in the queue
*/
-static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb, int index)
+static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb)
{
- if (index < VB2_MAX_FRAME && !q->bufs[index]) {
- q->bufs[index] = vb;
- vb->index = index;
- vb->vb2_queue = q;
- return true;
- }
+ struct xa_limit range = {
+ .max = UINT_MAX,
+ .min = q->num_buffers,
+ };
+ u32 index;
+ int ret;
- return false;
+ ret = xa_alloc(&q->bufs, &index, vb, range, GFP_KERNEL);
+ if (ret)
+ return false;
+
+ vb->index = index;
+ vb->vb2_queue = q;
+
+ return true;
}
/**
@@ -430,10 +436,8 @@ static bool vb2_queue_add_buffer(struct vb2_queue *q, struct vb2_buffer *vb, int
*/
static void vb2_queue_remove_buffer(struct vb2_queue *q, struct vb2_buffer *vb)
{
- if (vb->index < VB2_MAX_FRAME) {
- q->bufs[vb->index] = NULL;
- vb->vb2_queue = NULL;
- }
+ xa_erase(&q->bufs, vb->index);
+ vb->vb2_queue = NULL;
}
/*
@@ -474,7 +478,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
}
call_void_bufop(q, init_buffer, vb);
- if (!vb2_queue_add_buffer(q, vb, q->num_buffers + buffer)) {
+ if (!vb2_queue_add_buffer(q, vb)) {
dprintk(q, 1, "failed adding buffer %d to queue\n", buffer);
kfree(vb);
break;
@@ -930,7 +934,7 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
}
mutex_lock(&q->mmap_lock);
- q->num_buffers = allocated_buffers;
+ q->num_buffers += allocated_buffers;
if (ret < 0) {
/*
@@ -2547,6 +2551,9 @@ int vb2_core_queue_init(struct vb2_queue *q)
mutex_init(&q->mmap_lock);
init_waitqueue_head(&q->done_wq);
+ xa_init_flags(&q->bufs, XA_FLAGS_ALLOC);
+ q->num_buffers = 0;
+
q->memory = VB2_MEMORY_UNKNOWN;
if (q->buf_struct_size == 0)
@@ -2574,6 +2581,7 @@ void vb2_core_queue_release(struct vb2_queue *q)
mutex_lock(&q->mmap_lock);
__vb2_queue_free(q, q->num_buffers);
mutex_unlock(&q->mmap_lock);
+ xa_destroy(&q->bufs);
}
EXPORT_SYMBOL_GPL(vb2_core_queue_release);
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 4b6a9d2ea372..77921cf894ef 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -619,7 +619,7 @@ struct vb2_queue {
struct mutex mmap_lock;
unsigned int memory;
enum dma_data_direction dma_dir;
- struct vb2_buffer *bufs[VB2_MAX_FRAME];
+ struct xarray bufs;
unsigned int num_buffers;
struct list_head queued_list;
@@ -1239,9 +1239,7 @@ static inline void vb2_clear_last_buffer_dequeued(struct vb2_queue *q)
static inline struct vb2_buffer *vb2_get_buffer(struct vb2_queue *q,
unsigned int index)
{
- if (index < q->num_buffers)
- return q->bufs[index];
- return NULL;
+ return xa_load(&q->bufs, index);
}
/*
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 183+ messages in thread
* [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Remove VB2_MAX_FRAME buffer limit since Xarray allows to store
more than that.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++----------
1 file changed, 5 insertions(+), 10 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index f1ff7af34a9f..86e1e926fa45 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
struct vb2_buffer *vb;
int ret;
- /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
+ /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
num_buffers = min_t(unsigned int, num_buffers,
- VB2_MAX_FRAME - q->num_buffers);
+ UINT_MAX - q->num_buffers);
for (buffer = 0; buffer < num_buffers; ++buffer) {
/* Allocate vb2 buffer structures */
@@ -858,9 +858,9 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
/*
* Make sure the requested values and current defaults are sane.
*/
- WARN_ON(q->min_buffers_needed > VB2_MAX_FRAME);
+ WARN_ON(q->min_buffers_needed > UINT_MAX);
num_buffers = max_t(unsigned int, *count, q->min_buffers_needed);
- num_buffers = min_t(unsigned int, num_buffers, VB2_MAX_FRAME);
+ num_buffers = min_t(unsigned int, num_buffers, UINT_MAX);
memset(q->alloc_devs, 0, sizeof(q->alloc_devs));
/*
* Set this now to ensure that drivers see the correct q->memory value
@@ -976,11 +976,6 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
bool no_previous_buffers = !q->num_buffers;
int ret;
- if (q->num_buffers == VB2_MAX_FRAME) {
- dprintk(q, 1, "maximum number of buffers already allocated\n");
- return -ENOBUFS;
- }
-
if (no_previous_buffers) {
if (q->waiting_in_dqbuf && *count) {
dprintk(q, 1, "another dup()ped fd is waiting for a buffer\n");
@@ -1005,7 +1000,7 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
return -EINVAL;
}
- num_buffers = min(*count, VB2_MAX_FRAME - q->num_buffers);
+ num_buffers = min(*count, UINT_MAX - q->num_buffers);
if (requested_planes && requested_sizes) {
num_planes = requested_planes;
--
2.39.2
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Remove VB2_MAX_FRAME buffer limit since Xarray allows to store
more than that.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++----------
1 file changed, 5 insertions(+), 10 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index f1ff7af34a9f..86e1e926fa45 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
struct vb2_buffer *vb;
int ret;
- /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
+ /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
num_buffers = min_t(unsigned int, num_buffers,
- VB2_MAX_FRAME - q->num_buffers);
+ UINT_MAX - q->num_buffers);
for (buffer = 0; buffer < num_buffers; ++buffer) {
/* Allocate vb2 buffer structures */
@@ -858,9 +858,9 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
/*
* Make sure the requested values and current defaults are sane.
*/
- WARN_ON(q->min_buffers_needed > VB2_MAX_FRAME);
+ WARN_ON(q->min_buffers_needed > UINT_MAX);
num_buffers = max_t(unsigned int, *count, q->min_buffers_needed);
- num_buffers = min_t(unsigned int, num_buffers, VB2_MAX_FRAME);
+ num_buffers = min_t(unsigned int, num_buffers, UINT_MAX);
memset(q->alloc_devs, 0, sizeof(q->alloc_devs));
/*
* Set this now to ensure that drivers see the correct q->memory value
@@ -976,11 +976,6 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
bool no_previous_buffers = !q->num_buffers;
int ret;
- if (q->num_buffers == VB2_MAX_FRAME) {
- dprintk(q, 1, "maximum number of buffers already allocated\n");
- return -ENOBUFS;
- }
-
if (no_previous_buffers) {
if (q->waiting_in_dqbuf && *count) {
dprintk(q, 1, "another dup()ped fd is waiting for a buffer\n");
@@ -1005,7 +1000,7 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
return -EINVAL;
}
- num_buffers = min(*count, VB2_MAX_FRAME - q->num_buffers);
+ num_buffers = min(*count, UINT_MAX - q->num_buffers);
if (requested_planes && requested_sizes) {
num_planes = requested_planes;
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Remove VB2_MAX_FRAME buffer limit since Xarray allows to store
more than that.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++----------
1 file changed, 5 insertions(+), 10 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index f1ff7af34a9f..86e1e926fa45 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
struct vb2_buffer *vb;
int ret;
- /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
+ /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
num_buffers = min_t(unsigned int, num_buffers,
- VB2_MAX_FRAME - q->num_buffers);
+ UINT_MAX - q->num_buffers);
for (buffer = 0; buffer < num_buffers; ++buffer) {
/* Allocate vb2 buffer structures */
@@ -858,9 +858,9 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
/*
* Make sure the requested values and current defaults are sane.
*/
- WARN_ON(q->min_buffers_needed > VB2_MAX_FRAME);
+ WARN_ON(q->min_buffers_needed > UINT_MAX);
num_buffers = max_t(unsigned int, *count, q->min_buffers_needed);
- num_buffers = min_t(unsigned int, num_buffers, VB2_MAX_FRAME);
+ num_buffers = min_t(unsigned int, num_buffers, UINT_MAX);
memset(q->alloc_devs, 0, sizeof(q->alloc_devs));
/*
* Set this now to ensure that drivers see the correct q->memory value
@@ -976,11 +976,6 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
bool no_previous_buffers = !q->num_buffers;
int ret;
- if (q->num_buffers == VB2_MAX_FRAME) {
- dprintk(q, 1, "maximum number of buffers already allocated\n");
- return -ENOBUFS;
- }
-
if (no_previous_buffers) {
if (q->waiting_in_dqbuf && *count) {
dprintk(q, 1, "another dup()ped fd is waiting for a buffer\n");
@@ -1005,7 +1000,7 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
return -EINVAL;
}
- num_buffers = min(*count, VB2_MAX_FRAME - q->num_buffers);
+ num_buffers = min(*count, UINT_MAX - q->num_buffers);
if (requested_planes && requested_sizes) {
num_planes = requested_planes;
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:56 ` Dan Carpenter
-1 siblings, 0 replies; 183+ messages in thread
From: Dan Carpenter @ 2023-06-22 13:56 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne, linux-media,
linux-kernel, linux-arm-kernel, linux-mediatek, linux-arm-msm,
linux-rockchip, linux-staging, kernel
On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index f1ff7af34a9f..86e1e926fa45 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
> struct vb2_buffer *vb;
> int ret;
>
> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
> num_buffers = min_t(unsigned int, num_buffers,
> - VB2_MAX_FRAME - q->num_buffers);
> + UINT_MAX - q->num_buffers);
The UINT_MAX limit adds a level of danger. It would be safer to do what
the vfs layer does for MAX_RW_COUNT and use "INT_MAX - PAGE_SIZE". That
way you can take size + sizeof() and it's only very rarely going to turn
negative. Or at least just INT_MAX. I would keep the VB2_MAX_FRAME and
define it as:
#define VB2_MAX_FRAME (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
>
> for (buffer = 0; buffer < num_buffers; ++buffer) {
> /* Allocate vb2 buffer structures */
> @@ -858,9 +858,9 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
> /*
> * Make sure the requested values and current defaults are sane.
> */
> - WARN_ON(q->min_buffers_needed > VB2_MAX_FRAME);
> + WARN_ON(q->min_buffers_needed > UINT_MAX);
This will trigger a static checker warning because the condition is
impossible.
regards,
dan carpenter
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-22 13:56 ` Dan Carpenter
0 siblings, 0 replies; 183+ messages in thread
From: Dan Carpenter @ 2023-06-22 13:56 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne, linux-media,
linux-kernel, linux-arm-kernel, linux-mediatek, linux-arm-msm,
linux-rockchip, linux-staging, kernel
On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index f1ff7af34a9f..86e1e926fa45 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
> struct vb2_buffer *vb;
> int ret;
>
> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
> num_buffers = min_t(unsigned int, num_buffers,
> - VB2_MAX_FRAME - q->num_buffers);
> + UINT_MAX - q->num_buffers);
The UINT_MAX limit adds a level of danger. It would be safer to do what
the vfs layer does for MAX_RW_COUNT and use "INT_MAX - PAGE_SIZE". That
way you can take size + sizeof() and it's only very rarely going to turn
negative. Or at least just INT_MAX. I would keep the VB2_MAX_FRAME and
define it as:
#define VB2_MAX_FRAME (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
>
> for (buffer = 0; buffer < num_buffers; ++buffer) {
> /* Allocate vb2 buffer structures */
> @@ -858,9 +858,9 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
> /*
> * Make sure the requested values and current defaults are sane.
> */
> - WARN_ON(q->min_buffers_needed > VB2_MAX_FRAME);
> + WARN_ON(q->min_buffers_needed > UINT_MAX);
This will trigger a static checker warning because the condition is
impossible.
regards,
dan carpenter
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-22 13:56 ` Dan Carpenter
0 siblings, 0 replies; 183+ messages in thread
From: Dan Carpenter @ 2023-06-22 13:56 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne, linux-media,
linux-kernel, linux-arm-kernel, linux-mediatek, linux-arm-msm,
linux-rockchip, linux-staging, kernel
On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index f1ff7af34a9f..86e1e926fa45 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
> struct vb2_buffer *vb;
> int ret;
>
> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
> num_buffers = min_t(unsigned int, num_buffers,
> - VB2_MAX_FRAME - q->num_buffers);
> + UINT_MAX - q->num_buffers);
The UINT_MAX limit adds a level of danger. It would be safer to do what
the vfs layer does for MAX_RW_COUNT and use "INT_MAX - PAGE_SIZE". That
way you can take size + sizeof() and it's only very rarely going to turn
negative. Or at least just INT_MAX. I would keep the VB2_MAX_FRAME and
define it as:
#define VB2_MAX_FRAME (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
>
> for (buffer = 0; buffer < num_buffers; ++buffer) {
> /* Allocate vb2 buffer structures */
> @@ -858,9 +858,9 @@ int vb2_core_reqbufs(struct vb2_queue *q, enum vb2_memory memory,
> /*
> * Make sure the requested values and current defaults are sane.
> */
> - WARN_ON(q->min_buffers_needed > VB2_MAX_FRAME);
> + WARN_ON(q->min_buffers_needed > UINT_MAX);
This will trigger a static checker warning because the condition is
impossible.
regards,
dan carpenter
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 14:11 ` Dan Carpenter
-1 siblings, 0 replies; 183+ messages in thread
From: Dan Carpenter @ 2023-06-22 14:11 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne, linux-media,
linux-kernel, linux-arm-kernel, linux-mediatek, linux-arm-msm,
linux-rockchip, linux-staging, kernel
On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index f1ff7af34a9f..86e1e926fa45 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
> struct vb2_buffer *vb;
> int ret;
>
> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
> num_buffers = min_t(unsigned int, num_buffers,
> - VB2_MAX_FRAME - q->num_buffers);
> + UINT_MAX - q->num_buffers);
>
> for (buffer = 0; buffer < num_buffers; ++buffer) {
> /* Allocate vb2 buffer structures */
Ah... Here's one of the integer overflow bugs I was talking about. The
__vb2_queue_alloc() function returns an int so if num_buffers goes over
INT_MAX we are hosed.
regards,
dan carpenter
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-22 14:11 ` Dan Carpenter
0 siblings, 0 replies; 183+ messages in thread
From: Dan Carpenter @ 2023-06-22 14:11 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne, linux-media,
linux-kernel, linux-arm-kernel, linux-mediatek, linux-arm-msm,
linux-rockchip, linux-staging, kernel
On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index f1ff7af34a9f..86e1e926fa45 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
> struct vb2_buffer *vb;
> int ret;
>
> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
> num_buffers = min_t(unsigned int, num_buffers,
> - VB2_MAX_FRAME - q->num_buffers);
> + UINT_MAX - q->num_buffers);
>
> for (buffer = 0; buffer < num_buffers; ++buffer) {
> /* Allocate vb2 buffer structures */
Ah... Here's one of the integer overflow bugs I was talking about. The
__vb2_queue_alloc() function returns an int so if num_buffers goes over
INT_MAX we are hosed.
regards,
dan carpenter
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-22 14:11 ` Dan Carpenter
0 siblings, 0 replies; 183+ messages in thread
From: Dan Carpenter @ 2023-06-22 14:11 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne, linux-media,
linux-kernel, linux-arm-kernel, linux-mediatek, linux-arm-msm,
linux-rockchip, linux-staging, kernel
On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index f1ff7af34a9f..86e1e926fa45 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
> struct vb2_buffer *vb;
> int ret;
>
> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
> num_buffers = min_t(unsigned int, num_buffers,
> - VB2_MAX_FRAME - q->num_buffers);
> + UINT_MAX - q->num_buffers);
>
> for (buffer = 0; buffer < num_buffers; ++buffer) {
> /* Allocate vb2 buffer structures */
Ah... Here's one of the integer overflow bugs I was talking about. The
__vb2_queue_alloc() function returns an int so if num_buffers goes over
INT_MAX we are hosed.
regards,
dan carpenter
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
2023-06-22 14:11 ` Dan Carpenter
(?)
@ 2023-06-22 14:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 14:13 UTC (permalink / raw)
To: Dan Carpenter
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne, linux-media,
linux-kernel, linux-arm-kernel, linux-mediatek, linux-arm-msm,
linux-rockchip, linux-staging, kernel
Le 22/06/2023 à 16:11, Dan Carpenter a écrit :
> On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
>> index f1ff7af34a9f..86e1e926fa45 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
>> struct vb2_buffer *vb;
>> int ret;
>>
>> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
>> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
>> num_buffers = min_t(unsigned int, num_buffers,
>> - VB2_MAX_FRAME - q->num_buffers);
>> + UINT_MAX - q->num_buffers);
>>
>> for (buffer = 0; buffer < num_buffers; ++buffer) {
>> /* Allocate vb2 buffer structures */
> Ah... Here's one of the integer overflow bugs I was talking about. The
> __vb2_queue_alloc() function returns an int so if num_buffers goes over
> INT_MAX we are hosed.
I will limit it to:
#define VB2_QUEUE_MAX_BUFFERS (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
as you have suggest it.
That will be in version 4.
Thanks,
Benjamin
>
> regards,
> dan carpenter
>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-22 14:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 14:13 UTC (permalink / raw)
To: Dan Carpenter
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne, linux-media,
linux-kernel, linux-arm-kernel, linux-mediatek, linux-arm-msm,
linux-rockchip, linux-staging, kernel
Le 22/06/2023 à 16:11, Dan Carpenter a écrit :
> On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
>> index f1ff7af34a9f..86e1e926fa45 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
>> struct vb2_buffer *vb;
>> int ret;
>>
>> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
>> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
>> num_buffers = min_t(unsigned int, num_buffers,
>> - VB2_MAX_FRAME - q->num_buffers);
>> + UINT_MAX - q->num_buffers);
>>
>> for (buffer = 0; buffer < num_buffers; ++buffer) {
>> /* Allocate vb2 buffer structures */
> Ah... Here's one of the integer overflow bugs I was talking about. The
> __vb2_queue_alloc() function returns an int so if num_buffers goes over
> INT_MAX we are hosed.
I will limit it to:
#define VB2_QUEUE_MAX_BUFFERS (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
as you have suggest it.
That will be in version 4.
Thanks,
Benjamin
>
> regards,
> dan carpenter
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-22 14:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 14:13 UTC (permalink / raw)
To: Dan Carpenter
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne, linux-media,
linux-kernel, linux-arm-kernel, linux-mediatek, linux-arm-msm,
linux-rockchip, linux-staging, kernel
Le 22/06/2023 à 16:11, Dan Carpenter a écrit :
> On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
>> index f1ff7af34a9f..86e1e926fa45 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
>> struct vb2_buffer *vb;
>> int ret;
>>
>> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
>> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
>> num_buffers = min_t(unsigned int, num_buffers,
>> - VB2_MAX_FRAME - q->num_buffers);
>> + UINT_MAX - q->num_buffers);
>>
>> for (buffer = 0; buffer < num_buffers; ++buffer) {
>> /* Allocate vb2 buffer structures */
> Ah... Here's one of the integer overflow bugs I was talking about. The
> __vb2_queue_alloc() function returns an int so if num_buffers goes over
> INT_MAX we are hosed.
I will limit it to:
#define VB2_QUEUE_MAX_BUFFERS (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
as you have suggest it.
That will be in version 4.
Thanks,
Benjamin
>
> regards,
> dan carpenter
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
2023-06-22 14:13 ` Benjamin Gaignard
(?)
@ 2023-06-23 7:02 ` Hans Verkuil
-1 siblings, 0 replies; 183+ messages in thread
From: Hans Verkuil @ 2023-06-23 7:02 UTC (permalink / raw)
To: Benjamin Gaignard, Dan Carpenter
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, nicolas.dufresne, linux-media, linux-kernel,
linux-arm-kernel, linux-mediatek, linux-arm-msm, linux-rockchip,
linux-staging, kernel
On 22/06/2023 16:13, Benjamin Gaignard wrote:
>
> Le 22/06/2023 à 16:11, Dan Carpenter a écrit :
>> On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index f1ff7af34a9f..86e1e926fa45 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
>>> struct vb2_buffer *vb;
>>> int ret;
>>> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
>>> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
>>> num_buffers = min_t(unsigned int, num_buffers,
>>> - VB2_MAX_FRAME - q->num_buffers);
>>> + UINT_MAX - q->num_buffers);
>>> for (buffer = 0; buffer < num_buffers; ++buffer) {
>>> /* Allocate vb2 buffer structures */
>> Ah... Here's one of the integer overflow bugs I was talking about. The
>> __vb2_queue_alloc() function returns an int so if num_buffers goes over
>> INT_MAX we are hosed.
>
> I will limit it to:
> #define VB2_QUEUE_MAX_BUFFERS (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
> as you have suggest it.
IMHO INT_MAX is way overkill. How about (1U << 20)? I would like some sort of
sanity check here. 1048576 buffers of 640x480 and 4 bytes per pixel is 1.2 TB.
Since a TB of memory is doable these days, I think this is a reasonable
value for MAX_BUFFERS without allowing just anything.
An alternative is to make this a kernel config.
Regards,
Hans
>
> That will be in version 4.
>
> Thanks,
> Benjamin
>
>>
>> regards,
>> dan carpenter
>>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-23 7:02 ` Hans Verkuil
0 siblings, 0 replies; 183+ messages in thread
From: Hans Verkuil @ 2023-06-23 7:02 UTC (permalink / raw)
To: Benjamin Gaignard, Dan Carpenter
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, nicolas.dufresne, linux-media, linux-kernel,
linux-arm-kernel, linux-mediatek, linux-arm-msm, linux-rockchip,
linux-staging, kernel
On 22/06/2023 16:13, Benjamin Gaignard wrote:
>
> Le 22/06/2023 à 16:11, Dan Carpenter a écrit :
>> On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index f1ff7af34a9f..86e1e926fa45 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
>>> struct vb2_buffer *vb;
>>> int ret;
>>> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
>>> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
>>> num_buffers = min_t(unsigned int, num_buffers,
>>> - VB2_MAX_FRAME - q->num_buffers);
>>> + UINT_MAX - q->num_buffers);
>>> for (buffer = 0; buffer < num_buffers; ++buffer) {
>>> /* Allocate vb2 buffer structures */
>> Ah... Here's one of the integer overflow bugs I was talking about. The
>> __vb2_queue_alloc() function returns an int so if num_buffers goes over
>> INT_MAX we are hosed.
>
> I will limit it to:
> #define VB2_QUEUE_MAX_BUFFERS (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
> as you have suggest it.
IMHO INT_MAX is way overkill. How about (1U << 20)? I would like some sort of
sanity check here. 1048576 buffers of 640x480 and 4 bytes per pixel is 1.2 TB.
Since a TB of memory is doable these days, I think this is a reasonable
value for MAX_BUFFERS without allowing just anything.
An alternative is to make this a kernel config.
Regards,
Hans
>
> That will be in version 4.
>
> Thanks,
> Benjamin
>
>>
>> regards,
>> dan carpenter
>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-23 7:02 ` Hans Verkuil
0 siblings, 0 replies; 183+ messages in thread
From: Hans Verkuil @ 2023-06-23 7:02 UTC (permalink / raw)
To: Benjamin Gaignard, Dan Carpenter
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, nicolas.dufresne, linux-media, linux-kernel,
linux-arm-kernel, linux-mediatek, linux-arm-msm, linux-rockchip,
linux-staging, kernel
On 22/06/2023 16:13, Benjamin Gaignard wrote:
>
> Le 22/06/2023 à 16:11, Dan Carpenter a écrit :
>> On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index f1ff7af34a9f..86e1e926fa45 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
>>> struct vb2_buffer *vb;
>>> int ret;
>>> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
>>> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
>>> num_buffers = min_t(unsigned int, num_buffers,
>>> - VB2_MAX_FRAME - q->num_buffers);
>>> + UINT_MAX - q->num_buffers);
>>> for (buffer = 0; buffer < num_buffers; ++buffer) {
>>> /* Allocate vb2 buffer structures */
>> Ah... Here's one of the integer overflow bugs I was talking about. The
>> __vb2_queue_alloc() function returns an int so if num_buffers goes over
>> INT_MAX we are hosed.
>
> I will limit it to:
> #define VB2_QUEUE_MAX_BUFFERS (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
> as you have suggest it.
IMHO INT_MAX is way overkill. How about (1U << 20)? I would like some sort of
sanity check here. 1048576 buffers of 640x480 and 4 bytes per pixel is 1.2 TB.
Since a TB of memory is doable these days, I think this is a reasonable
value for MAX_BUFFERS without allowing just anything.
An alternative is to make this a kernel config.
Regards,
Hans
>
> That will be in version 4.
>
> Thanks,
> Benjamin
>
>>
>> regards,
>> dan carpenter
>>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
2023-06-23 7:02 ` Hans Verkuil
(?)
@ 2023-06-23 7:51 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-23 7:51 UTC (permalink / raw)
To: Hans Verkuil, Dan Carpenter
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, nicolas.dufresne, linux-media, linux-kernel,
linux-arm-kernel, linux-mediatek, linux-arm-msm, linux-rockchip,
linux-staging, kernel
Le 23/06/2023 à 09:02, Hans Verkuil a écrit :
> On 22/06/2023 16:13, Benjamin Gaignard wrote:
>> Le 22/06/2023 à 16:11, Dan Carpenter a écrit :
>>> On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index f1ff7af34a9f..86e1e926fa45 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
>>>> struct vb2_buffer *vb;
>>>> int ret;
>>>> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
>>>> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
>>>> num_buffers = min_t(unsigned int, num_buffers,
>>>> - VB2_MAX_FRAME - q->num_buffers);
>>>> + UINT_MAX - q->num_buffers);
>>>> for (buffer = 0; buffer < num_buffers; ++buffer) {
>>>> /* Allocate vb2 buffer structures */
>>> Ah... Here's one of the integer overflow bugs I was talking about. The
>>> __vb2_queue_alloc() function returns an int so if num_buffers goes over
>>> INT_MAX we are hosed.
>> I will limit it to:
>> #define VB2_QUEUE_MAX_BUFFERS (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
>> as you have suggest it.
> IMHO INT_MAX is way overkill. How about (1U << 20)? I would like some sort of
> sanity check here. 1048576 buffers of 640x480 and 4 bytes per pixel is 1.2 TB.
I will go for (1U << 20) in next version.
Regards,
Benjamin
>
> Since a TB of memory is doable these days, I think this is a reasonable
> value for MAX_BUFFERS without allowing just anything.
>
> An alternative is to make this a kernel config.
>
> Regards,
>
> Hans
>
>> That will be in version 4.
>>
>> Thanks,
>> Benjamin
>>
>>> regards,
>>> dan carpenter
>>>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-23 7:51 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-23 7:51 UTC (permalink / raw)
To: Hans Verkuil, Dan Carpenter
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, nicolas.dufresne, linux-media, linux-kernel,
linux-arm-kernel, linux-mediatek, linux-arm-msm, linux-rockchip,
linux-staging, kernel
Le 23/06/2023 à 09:02, Hans Verkuil a écrit :
> On 22/06/2023 16:13, Benjamin Gaignard wrote:
>> Le 22/06/2023 à 16:11, Dan Carpenter a écrit :
>>> On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index f1ff7af34a9f..86e1e926fa45 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
>>>> struct vb2_buffer *vb;
>>>> int ret;
>>>> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
>>>> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
>>>> num_buffers = min_t(unsigned int, num_buffers,
>>>> - VB2_MAX_FRAME - q->num_buffers);
>>>> + UINT_MAX - q->num_buffers);
>>>> for (buffer = 0; buffer < num_buffers; ++buffer) {
>>>> /* Allocate vb2 buffer structures */
>>> Ah... Here's one of the integer overflow bugs I was talking about. The
>>> __vb2_queue_alloc() function returns an int so if num_buffers goes over
>>> INT_MAX we are hosed.
>> I will limit it to:
>> #define VB2_QUEUE_MAX_BUFFERS (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
>> as you have suggest it.
> IMHO INT_MAX is way overkill. How about (1U << 20)? I would like some sort of
> sanity check here. 1048576 buffers of 640x480 and 4 bytes per pixel is 1.2 TB.
I will go for (1U << 20) in next version.
Regards,
Benjamin
>
> Since a TB of memory is doable these days, I think this is a reasonable
> value for MAX_BUFFERS without allowing just anything.
>
> An alternative is to make this a kernel config.
>
> Regards,
>
> Hans
>
>> That will be in version 4.
>>
>> Thanks,
>> Benjamin
>>
>>> regards,
>>> dan carpenter
>>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 03/11] media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
@ 2023-06-23 7:51 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-23 7:51 UTC (permalink / raw)
To: Hans Verkuil, Dan Carpenter
Cc: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, nicolas.dufresne, linux-media, linux-kernel,
linux-arm-kernel, linux-mediatek, linux-arm-msm, linux-rockchip,
linux-staging, kernel
Le 23/06/2023 à 09:02, Hans Verkuil a écrit :
> On 22/06/2023 16:13, Benjamin Gaignard wrote:
>> Le 22/06/2023 à 16:11, Dan Carpenter a écrit :
>>> On Thu, Jun 22, 2023 at 03:13:41PM +0200, Benjamin Gaignard wrote:
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index f1ff7af34a9f..86e1e926fa45 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -455,9 +455,9 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
>>>> struct vb2_buffer *vb;
>>>> int ret;
>>>> - /* Ensure that q->num_buffers+num_buffers is below VB2_MAX_FRAME */
>>>> + /* Ensure that q->num_buffers + num_buffers is UINT_MAX */
>>>> num_buffers = min_t(unsigned int, num_buffers,
>>>> - VB2_MAX_FRAME - q->num_buffers);
>>>> + UINT_MAX - q->num_buffers);
>>>> for (buffer = 0; buffer < num_buffers; ++buffer) {
>>>> /* Allocate vb2 buffer structures */
>>> Ah... Here's one of the integer overflow bugs I was talking about. The
>>> __vb2_queue_alloc() function returns an int so if num_buffers goes over
>>> INT_MAX we are hosed.
>> I will limit it to:
>> #define VB2_QUEUE_MAX_BUFFERS (INT_MAX & PAGE_MASK) /* The mask prevents 85% of integer overflows */
>> as you have suggest it.
> IMHO INT_MAX is way overkill. How about (1U << 20)? I would like some sort of
> sanity check here. 1048576 buffers of 640x480 and 4 bytes per pixel is 1.2 TB.
I will go for (1U << 20) in next version.
Regards,
Benjamin
>
> Since a TB of memory is doable these days, I think this is a reasonable
> value for MAX_BUFFERS without allowing just anything.
>
> An alternative is to make this a kernel config.
>
> Regards,
>
> Hans
>
>> That will be in version 4.
>>
>> Thanks,
>> Benjamin
>>
>>> regards,
>>> dan carpenter
>>>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
After changing bufs arrays to a dynamic allocated array
VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
Remove it from the core definitions but keep it for drivers internal
needs.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
drivers/media/platform/amphion/vdec.c | 1 +
.../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
drivers/media/platform/qcom/venus/hfi.h | 2 ++
drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
include/media/videobuf2-core.h | 1 -
include/media/videobuf2-v4l2.h | 4 ----
8 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 86e1e926fa45..899783f67580 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -31,6 +31,8 @@
#include <trace/events/vb2.h>
+#define VB2_MAX_FRAME 32
+
static int debug;
module_param(debug, int, 0644);
diff --git a/drivers/media/platform/amphion/vdec.c b/drivers/media/platform/amphion/vdec.c
index 3fa1a74a2e20..b3219f6d17fa 100644
--- a/drivers/media/platform/amphion/vdec.c
+++ b/drivers/media/platform/amphion/vdec.c
@@ -28,6 +28,7 @@
#define VDEC_MIN_BUFFER_CAP 8
#define VDEC_MIN_BUFFER_OUT 8
+#define VB2_MAX_FRAME 32
struct vdec_fs_info {
char name[8];
diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
index 6532a69f1fa8..a1e0f24bb91c 100644
--- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
+++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
@@ -16,6 +16,8 @@
#include "../vdec_drv_if.h"
#include "../vdec_vpu_if.h"
+#define VB2_MAX_FRAME 32
+
/* reset_frame_context defined in VP9 spec */
#define VP9_RESET_FRAME_CONTEXT_NONE0 0
#define VP9_RESET_FRAME_CONTEXT_NONE1 1
diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/venus/hfi.h
index f25d412d6553..bd5ca5a8b945 100644
--- a/drivers/media/platform/qcom/venus/hfi.h
+++ b/drivers/media/platform/qcom/venus/hfi.h
@@ -10,6 +10,8 @@
#include "hfi_helper.h"
+#define VB2_MAX_FRAME 32
+
#define VIDC_SESSION_TYPE_VPE 0
#define VIDC_SESSION_TYPE_ENC 1
#define VIDC_SESSION_TYPE_DEC 2
diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
index e83f0c523a30..9e8faf7ba6fb 100644
--- a/drivers/media/platform/verisilicon/hantro_hw.h
+++ b/drivers/media/platform/verisilicon/hantro_hw.h
@@ -15,6 +15,8 @@
#include <media/v4l2-vp9.h>
#include <media/videobuf2-core.h>
+#define VB2_MAX_FRAME 32
+
#define DEC_8190_ALIGN_MASK 0x07U
#define MB_DIM 16
diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c b/drivers/staging/media/ipu3/ipu3-v4l2.c
index e530767e80a5..6627b5c2d4d6 100644
--- a/drivers/staging/media/ipu3/ipu3-v4l2.c
+++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
@@ -10,6 +10,8 @@
#include "ipu3.h"
#include "ipu3-dmamap.h"
+#define VB2_MAX_FRAME 32
+
/******************** v4l2_subdev_ops ********************/
#define IPU3_RUNNING_MODE_VIDEO 0
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 77921cf894ef..080b783d608d 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -20,7 +20,6 @@
#include <media/media-request.h>
#include <media/frame_vector.h>
-#define VB2_MAX_FRAME (32)
#define VB2_MAX_PLANES (8)
/**
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 5a845887850b..88a7a565170e 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -15,10 +15,6 @@
#include <linux/videodev2.h>
#include <media/videobuf2-core.h>
-#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
-#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
-#endif
-
#if VB2_MAX_PLANES != VIDEO_MAX_PLANES
#error VB2_MAX_PLANES != VIDEO_MAX_PLANES
#endif
--
2.39.2
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
After changing bufs arrays to a dynamic allocated array
VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
Remove it from the core definitions but keep it for drivers internal
needs.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
drivers/media/platform/amphion/vdec.c | 1 +
.../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
drivers/media/platform/qcom/venus/hfi.h | 2 ++
drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
include/media/videobuf2-core.h | 1 -
include/media/videobuf2-v4l2.h | 4 ----
8 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 86e1e926fa45..899783f67580 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -31,6 +31,8 @@
#include <trace/events/vb2.h>
+#define VB2_MAX_FRAME 32
+
static int debug;
module_param(debug, int, 0644);
diff --git a/drivers/media/platform/amphion/vdec.c b/drivers/media/platform/amphion/vdec.c
index 3fa1a74a2e20..b3219f6d17fa 100644
--- a/drivers/media/platform/amphion/vdec.c
+++ b/drivers/media/platform/amphion/vdec.c
@@ -28,6 +28,7 @@
#define VDEC_MIN_BUFFER_CAP 8
#define VDEC_MIN_BUFFER_OUT 8
+#define VB2_MAX_FRAME 32
struct vdec_fs_info {
char name[8];
diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
index 6532a69f1fa8..a1e0f24bb91c 100644
--- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
+++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
@@ -16,6 +16,8 @@
#include "../vdec_drv_if.h"
#include "../vdec_vpu_if.h"
+#define VB2_MAX_FRAME 32
+
/* reset_frame_context defined in VP9 spec */
#define VP9_RESET_FRAME_CONTEXT_NONE0 0
#define VP9_RESET_FRAME_CONTEXT_NONE1 1
diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/venus/hfi.h
index f25d412d6553..bd5ca5a8b945 100644
--- a/drivers/media/platform/qcom/venus/hfi.h
+++ b/drivers/media/platform/qcom/venus/hfi.h
@@ -10,6 +10,8 @@
#include "hfi_helper.h"
+#define VB2_MAX_FRAME 32
+
#define VIDC_SESSION_TYPE_VPE 0
#define VIDC_SESSION_TYPE_ENC 1
#define VIDC_SESSION_TYPE_DEC 2
diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
index e83f0c523a30..9e8faf7ba6fb 100644
--- a/drivers/media/platform/verisilicon/hantro_hw.h
+++ b/drivers/media/platform/verisilicon/hantro_hw.h
@@ -15,6 +15,8 @@
#include <media/v4l2-vp9.h>
#include <media/videobuf2-core.h>
+#define VB2_MAX_FRAME 32
+
#define DEC_8190_ALIGN_MASK 0x07U
#define MB_DIM 16
diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c b/drivers/staging/media/ipu3/ipu3-v4l2.c
index e530767e80a5..6627b5c2d4d6 100644
--- a/drivers/staging/media/ipu3/ipu3-v4l2.c
+++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
@@ -10,6 +10,8 @@
#include "ipu3.h"
#include "ipu3-dmamap.h"
+#define VB2_MAX_FRAME 32
+
/******************** v4l2_subdev_ops ********************/
#define IPU3_RUNNING_MODE_VIDEO 0
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 77921cf894ef..080b783d608d 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -20,7 +20,6 @@
#include <media/media-request.h>
#include <media/frame_vector.h>
-#define VB2_MAX_FRAME (32)
#define VB2_MAX_PLANES (8)
/**
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 5a845887850b..88a7a565170e 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -15,10 +15,6 @@
#include <linux/videodev2.h>
#include <media/videobuf2-core.h>
-#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
-#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
-#endif
-
#if VB2_MAX_PLANES != VIDEO_MAX_PLANES
#error VB2_MAX_PLANES != VIDEO_MAX_PLANES
#endif
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
After changing bufs arrays to a dynamic allocated array
VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
Remove it from the core definitions but keep it for drivers internal
needs.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
drivers/media/platform/amphion/vdec.c | 1 +
.../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
drivers/media/platform/qcom/venus/hfi.h | 2 ++
drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
include/media/videobuf2-core.h | 1 -
include/media/videobuf2-v4l2.h | 4 ----
8 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 86e1e926fa45..899783f67580 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -31,6 +31,8 @@
#include <trace/events/vb2.h>
+#define VB2_MAX_FRAME 32
+
static int debug;
module_param(debug, int, 0644);
diff --git a/drivers/media/platform/amphion/vdec.c b/drivers/media/platform/amphion/vdec.c
index 3fa1a74a2e20..b3219f6d17fa 100644
--- a/drivers/media/platform/amphion/vdec.c
+++ b/drivers/media/platform/amphion/vdec.c
@@ -28,6 +28,7 @@
#define VDEC_MIN_BUFFER_CAP 8
#define VDEC_MIN_BUFFER_OUT 8
+#define VB2_MAX_FRAME 32
struct vdec_fs_info {
char name[8];
diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
index 6532a69f1fa8..a1e0f24bb91c 100644
--- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
+++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
@@ -16,6 +16,8 @@
#include "../vdec_drv_if.h"
#include "../vdec_vpu_if.h"
+#define VB2_MAX_FRAME 32
+
/* reset_frame_context defined in VP9 spec */
#define VP9_RESET_FRAME_CONTEXT_NONE0 0
#define VP9_RESET_FRAME_CONTEXT_NONE1 1
diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/venus/hfi.h
index f25d412d6553..bd5ca5a8b945 100644
--- a/drivers/media/platform/qcom/venus/hfi.h
+++ b/drivers/media/platform/qcom/venus/hfi.h
@@ -10,6 +10,8 @@
#include "hfi_helper.h"
+#define VB2_MAX_FRAME 32
+
#define VIDC_SESSION_TYPE_VPE 0
#define VIDC_SESSION_TYPE_ENC 1
#define VIDC_SESSION_TYPE_DEC 2
diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
index e83f0c523a30..9e8faf7ba6fb 100644
--- a/drivers/media/platform/verisilicon/hantro_hw.h
+++ b/drivers/media/platform/verisilicon/hantro_hw.h
@@ -15,6 +15,8 @@
#include <media/v4l2-vp9.h>
#include <media/videobuf2-core.h>
+#define VB2_MAX_FRAME 32
+
#define DEC_8190_ALIGN_MASK 0x07U
#define MB_DIM 16
diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c b/drivers/staging/media/ipu3/ipu3-v4l2.c
index e530767e80a5..6627b5c2d4d6 100644
--- a/drivers/staging/media/ipu3/ipu3-v4l2.c
+++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
@@ -10,6 +10,8 @@
#include "ipu3.h"
#include "ipu3-dmamap.h"
+#define VB2_MAX_FRAME 32
+
/******************** v4l2_subdev_ops ********************/
#define IPU3_RUNNING_MODE_VIDEO 0
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 77921cf894ef..080b783d608d 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -20,7 +20,6 @@
#include <media/media-request.h>
#include <media/frame_vector.h>
-#define VB2_MAX_FRAME (32)
#define VB2_MAX_PLANES (8)
/**
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 5a845887850b..88a7a565170e 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -15,10 +15,6 @@
#include <linux/videodev2.h>
#include <media/videobuf2-core.h>
-#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
-#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
-#endif
-
#if VB2_MAX_PLANES != VIDEO_MAX_PLANES
#error VB2_MAX_PLANES != VIDEO_MAX_PLANES
#endif
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-30 9:51 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-30 9:51 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
On 6/22/23 21:13, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> After changing bufs arrays to a dynamic allocated array
> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
more reasonable.
It would be hard to iterate the whole array, it would go worse with a
filter. Such iterate may need to go twice because you mix
post-processing buffer and decoding buffer(with MV) in the same array.
> Remove it from the core definitions but keep it for drivers internal
> needs.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> drivers/media/platform/amphion/vdec.c | 1 +
> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> include/media/videobuf2-core.h | 1 -
> include/media/videobuf2-v4l2.h | 4 ----
> 8 files changed, 11 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 86e1e926fa45..899783f67580 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -31,6 +31,8 @@
>
> #include <trace/events/vb2.h>
>
> +#define VB2_MAX_FRAME 32
> +
> static int debug;
> module_param(debug, int, 0644);
>
> diff --git a/drivers/media/platform/amphion/vdec.c b/drivers/media/platform/amphion/vdec.c
> index 3fa1a74a2e20..b3219f6d17fa 100644
> --- a/drivers/media/platform/amphion/vdec.c
> +++ b/drivers/media/platform/amphion/vdec.c
> @@ -28,6 +28,7 @@
>
> #define VDEC_MIN_BUFFER_CAP 8
> #define VDEC_MIN_BUFFER_OUT 8
> +#define VB2_MAX_FRAME 32
>
> struct vdec_fs_info {
> char name[8];
> diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> index 6532a69f1fa8..a1e0f24bb91c 100644
> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> @@ -16,6 +16,8 @@
> #include "../vdec_drv_if.h"
> #include "../vdec_vpu_if.h"
>
> +#define VB2_MAX_FRAME 32
> +
> /* reset_frame_context defined in VP9 spec */
> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/venus/hfi.h
> index f25d412d6553..bd5ca5a8b945 100644
> --- a/drivers/media/platform/qcom/venus/hfi.h
> +++ b/drivers/media/platform/qcom/venus/hfi.h
> @@ -10,6 +10,8 @@
>
> #include "hfi_helper.h"
>
> +#define VB2_MAX_FRAME 32
> +
> #define VIDC_SESSION_TYPE_VPE 0
> #define VIDC_SESSION_TYPE_ENC 1
> #define VIDC_SESSION_TYPE_DEC 2
> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
> index e83f0c523a30..9e8faf7ba6fb 100644
> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> @@ -15,6 +15,8 @@
> #include <media/v4l2-vp9.h>
> #include <media/videobuf2-core.h>
>
> +#define VB2_MAX_FRAME 32
> +
> #define DEC_8190_ALIGN_MASK 0x07U
>
> #define MB_DIM 16
> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c b/drivers/staging/media/ipu3/ipu3-v4l2.c
> index e530767e80a5..6627b5c2d4d6 100644
> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> @@ -10,6 +10,8 @@
> #include "ipu3.h"
> #include "ipu3-dmamap.h"
>
> +#define VB2_MAX_FRAME 32
> +
> /******************** v4l2_subdev_ops ********************/
>
> #define IPU3_RUNNING_MODE_VIDEO 0
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 77921cf894ef..080b783d608d 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -20,7 +20,6 @@
> #include <media/media-request.h>
> #include <media/frame_vector.h>
>
> -#define VB2_MAX_FRAME (32)
> #define VB2_MAX_PLANES (8)
>
> /**
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 5a845887850b..88a7a565170e 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -15,10 +15,6 @@
> #include <linux/videodev2.h>
> #include <media/videobuf2-core.h>
>
> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> -#endif
> -
> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> #endif
> --
> 2.39.2
>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-06-30 9:51 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-30 9:51 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
On 6/22/23 21:13, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> After changing bufs arrays to a dynamic allocated array
> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
more reasonable.
It would be hard to iterate the whole array, it would go worse with a
filter. Such iterate may need to go twice because you mix
post-processing buffer and decoding buffer(with MV) in the same array.
> Remove it from the core definitions but keep it for drivers internal
> needs.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> drivers/media/platform/amphion/vdec.c | 1 +
> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> include/media/videobuf2-core.h | 1 -
> include/media/videobuf2-v4l2.h | 4 ----
> 8 files changed, 11 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 86e1e926fa45..899783f67580 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -31,6 +31,8 @@
>
> #include <trace/events/vb2.h>
>
> +#define VB2_MAX_FRAME 32
> +
> static int debug;
> module_param(debug, int, 0644);
>
> diff --git a/drivers/media/platform/amphion/vdec.c b/drivers/media/platform/amphion/vdec.c
> index 3fa1a74a2e20..b3219f6d17fa 100644
> --- a/drivers/media/platform/amphion/vdec.c
> +++ b/drivers/media/platform/amphion/vdec.c
> @@ -28,6 +28,7 @@
>
> #define VDEC_MIN_BUFFER_CAP 8
> #define VDEC_MIN_BUFFER_OUT 8
> +#define VB2_MAX_FRAME 32
>
> struct vdec_fs_info {
> char name[8];
> diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> index 6532a69f1fa8..a1e0f24bb91c 100644
> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> @@ -16,6 +16,8 @@
> #include "../vdec_drv_if.h"
> #include "../vdec_vpu_if.h"
>
> +#define VB2_MAX_FRAME 32
> +
> /* reset_frame_context defined in VP9 spec */
> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/venus/hfi.h
> index f25d412d6553..bd5ca5a8b945 100644
> --- a/drivers/media/platform/qcom/venus/hfi.h
> +++ b/drivers/media/platform/qcom/venus/hfi.h
> @@ -10,6 +10,8 @@
>
> #include "hfi_helper.h"
>
> +#define VB2_MAX_FRAME 32
> +
> #define VIDC_SESSION_TYPE_VPE 0
> #define VIDC_SESSION_TYPE_ENC 1
> #define VIDC_SESSION_TYPE_DEC 2
> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
> index e83f0c523a30..9e8faf7ba6fb 100644
> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> @@ -15,6 +15,8 @@
> #include <media/v4l2-vp9.h>
> #include <media/videobuf2-core.h>
>
> +#define VB2_MAX_FRAME 32
> +
> #define DEC_8190_ALIGN_MASK 0x07U
>
> #define MB_DIM 16
> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c b/drivers/staging/media/ipu3/ipu3-v4l2.c
> index e530767e80a5..6627b5c2d4d6 100644
> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> @@ -10,6 +10,8 @@
> #include "ipu3.h"
> #include "ipu3-dmamap.h"
>
> +#define VB2_MAX_FRAME 32
> +
> /******************** v4l2_subdev_ops ********************/
>
> #define IPU3_RUNNING_MODE_VIDEO 0
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 77921cf894ef..080b783d608d 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -20,7 +20,6 @@
> #include <media/media-request.h>
> #include <media/frame_vector.h>
>
> -#define VB2_MAX_FRAME (32)
> #define VB2_MAX_PLANES (8)
>
> /**
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 5a845887850b..88a7a565170e 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -15,10 +15,6 @@
> #include <linux/videodev2.h>
> #include <media/videobuf2-core.h>
>
> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> -#endif
> -
> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> #endif
> --
> 2.39.2
>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-06-30 9:51 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-30 9:51 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
On 6/22/23 21:13, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> After changing bufs arrays to a dynamic allocated array
> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
more reasonable.
It would be hard to iterate the whole array, it would go worse with a
filter. Such iterate may need to go twice because you mix
post-processing buffer and decoding buffer(with MV) in the same array.
> Remove it from the core definitions but keep it for drivers internal
> needs.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> drivers/media/platform/amphion/vdec.c | 1 +
> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> include/media/videobuf2-core.h | 1 -
> include/media/videobuf2-v4l2.h | 4 ----
> 8 files changed, 11 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 86e1e926fa45..899783f67580 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -31,6 +31,8 @@
>
> #include <trace/events/vb2.h>
>
> +#define VB2_MAX_FRAME 32
> +
> static int debug;
> module_param(debug, int, 0644);
>
> diff --git a/drivers/media/platform/amphion/vdec.c b/drivers/media/platform/amphion/vdec.c
> index 3fa1a74a2e20..b3219f6d17fa 100644
> --- a/drivers/media/platform/amphion/vdec.c
> +++ b/drivers/media/platform/amphion/vdec.c
> @@ -28,6 +28,7 @@
>
> #define VDEC_MIN_BUFFER_CAP 8
> #define VDEC_MIN_BUFFER_OUT 8
> +#define VB2_MAX_FRAME 32
>
> struct vdec_fs_info {
> char name[8];
> diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> index 6532a69f1fa8..a1e0f24bb91c 100644
> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> @@ -16,6 +16,8 @@
> #include "../vdec_drv_if.h"
> #include "../vdec_vpu_if.h"
>
> +#define VB2_MAX_FRAME 32
> +
> /* reset_frame_context defined in VP9 spec */
> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/venus/hfi.h
> index f25d412d6553..bd5ca5a8b945 100644
> --- a/drivers/media/platform/qcom/venus/hfi.h
> +++ b/drivers/media/platform/qcom/venus/hfi.h
> @@ -10,6 +10,8 @@
>
> #include "hfi_helper.h"
>
> +#define VB2_MAX_FRAME 32
> +
> #define VIDC_SESSION_TYPE_VPE 0
> #define VIDC_SESSION_TYPE_ENC 1
> #define VIDC_SESSION_TYPE_DEC 2
> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
> index e83f0c523a30..9e8faf7ba6fb 100644
> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> @@ -15,6 +15,8 @@
> #include <media/v4l2-vp9.h>
> #include <media/videobuf2-core.h>
>
> +#define VB2_MAX_FRAME 32
> +
> #define DEC_8190_ALIGN_MASK 0x07U
>
> #define MB_DIM 16
> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c b/drivers/staging/media/ipu3/ipu3-v4l2.c
> index e530767e80a5..6627b5c2d4d6 100644
> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> @@ -10,6 +10,8 @@
> #include "ipu3.h"
> #include "ipu3-dmamap.h"
>
> +#define VB2_MAX_FRAME 32
> +
> /******************** v4l2_subdev_ops ********************/
>
> #define IPU3_RUNNING_MODE_VIDEO 0
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 77921cf894ef..080b783d608d 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -20,7 +20,6 @@
> #include <media/media-request.h>
> #include <media/frame_vector.h>
>
> -#define VB2_MAX_FRAME (32)
> #define VB2_MAX_PLANES (8)
>
> /**
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 5a845887850b..88a7a565170e 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -15,10 +15,6 @@
> #include <linux/videodev2.h>
> #include <media/videobuf2-core.h>
>
> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> -#endif
> -
> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> #endif
> --
> 2.39.2
>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-06-30 9:51 ` Hsia-Jun Li
(?)
@ 2023-07-03 8:09 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 8:09 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>
> On 6/22/23 21:13, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> After changing bufs arrays to a dynamic allocated array
>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>
> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> more reasonable.
>
> It would be hard to iterate the whole array, it would go worse with a
> filter. Such iterate may need to go twice because you mix
> post-processing buffer and decoding buffer(with MV) in the same array.
Here I don't want to change drivers behavior so I keep the same value.
If it happens that they need more buffers, like for dynamic resolution change
feature for Verisilicon VP9 decoder, case by case patches will be needed.
>
>> Remove it from the core definitions but keep it for drivers internal
>> needs.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>> drivers/media/platform/amphion/vdec.c | 1 +
>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>> include/media/videobuf2-core.h | 1 -
>> include/media/videobuf2-v4l2.h | 4 ----
>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index 86e1e926fa45..899783f67580 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -31,6 +31,8 @@
>>
>> #include <trace/events/vb2.h>
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> static int debug;
>> module_param(debug, int, 0644);
>>
>> diff --git a/drivers/media/platform/amphion/vdec.c
>> b/drivers/media/platform/amphion/vdec.c
>> index 3fa1a74a2e20..b3219f6d17fa 100644
>> --- a/drivers/media/platform/amphion/vdec.c
>> +++ b/drivers/media/platform/amphion/vdec.c
>> @@ -28,6 +28,7 @@
>>
>> #define VDEC_MIN_BUFFER_CAP 8
>> #define VDEC_MIN_BUFFER_OUT 8
>> +#define VB2_MAX_FRAME 32
>>
>> struct vdec_fs_info {
>> char name[8];
>> diff --git
>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> index 6532a69f1fa8..a1e0f24bb91c 100644
>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> @@ -16,6 +16,8 @@
>> #include "../vdec_drv_if.h"
>> #include "../vdec_vpu_if.h"
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> /* reset_frame_context defined in VP9 spec */
>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>> b/drivers/media/platform/qcom/venus/hfi.h
>> index f25d412d6553..bd5ca5a8b945 100644
>> --- a/drivers/media/platform/qcom/venus/hfi.h
>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>> @@ -10,6 +10,8 @@
>>
>> #include "hfi_helper.h"
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> #define VIDC_SESSION_TYPE_VPE 0
>> #define VIDC_SESSION_TYPE_ENC 1
>> #define VIDC_SESSION_TYPE_DEC 2
>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>> b/drivers/media/platform/verisilicon/hantro_hw.h
>> index e83f0c523a30..9e8faf7ba6fb 100644
>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>> @@ -15,6 +15,8 @@
>> #include <media/v4l2-vp9.h>
>> #include <media/videobuf2-core.h>
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> #define DEC_8190_ALIGN_MASK 0x07U
>>
>> #define MB_DIM 16
>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>> index e530767e80a5..6627b5c2d4d6 100644
>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>> @@ -10,6 +10,8 @@
>> #include "ipu3.h"
>> #include "ipu3-dmamap.h"
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> /******************** v4l2_subdev_ops ********************/
>>
>> #define IPU3_RUNNING_MODE_VIDEO 0
>> diff --git a/include/media/videobuf2-core.h
>> b/include/media/videobuf2-core.h
>> index 77921cf894ef..080b783d608d 100644
>> --- a/include/media/videobuf2-core.h
>> +++ b/include/media/videobuf2-core.h
>> @@ -20,7 +20,6 @@
>> #include <media/media-request.h>
>> #include <media/frame_vector.h>
>>
>> -#define VB2_MAX_FRAME (32)
>> #define VB2_MAX_PLANES (8)
>>
>> /**
>> diff --git a/include/media/videobuf2-v4l2.h
>> b/include/media/videobuf2-v4l2.h
>> index 5a845887850b..88a7a565170e 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -15,10 +15,6 @@
>> #include <linux/videodev2.h>
>> #include <media/videobuf2-core.h>
>>
>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>> -#endif
>> -
>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>> #endif
>> --
>> 2.39.2
>>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-03 8:09 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 8:09 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>
> On 6/22/23 21:13, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> After changing bufs arrays to a dynamic allocated array
>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>
> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> more reasonable.
>
> It would be hard to iterate the whole array, it would go worse with a
> filter. Such iterate may need to go twice because you mix
> post-processing buffer and decoding buffer(with MV) in the same array.
Here I don't want to change drivers behavior so I keep the same value.
If it happens that they need more buffers, like for dynamic resolution change
feature for Verisilicon VP9 decoder, case by case patches will be needed.
>
>> Remove it from the core definitions but keep it for drivers internal
>> needs.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>> drivers/media/platform/amphion/vdec.c | 1 +
>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>> include/media/videobuf2-core.h | 1 -
>> include/media/videobuf2-v4l2.h | 4 ----
>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index 86e1e926fa45..899783f67580 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -31,6 +31,8 @@
>>
>> #include <trace/events/vb2.h>
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> static int debug;
>> module_param(debug, int, 0644);
>>
>> diff --git a/drivers/media/platform/amphion/vdec.c
>> b/drivers/media/platform/amphion/vdec.c
>> index 3fa1a74a2e20..b3219f6d17fa 100644
>> --- a/drivers/media/platform/amphion/vdec.c
>> +++ b/drivers/media/platform/amphion/vdec.c
>> @@ -28,6 +28,7 @@
>>
>> #define VDEC_MIN_BUFFER_CAP 8
>> #define VDEC_MIN_BUFFER_OUT 8
>> +#define VB2_MAX_FRAME 32
>>
>> struct vdec_fs_info {
>> char name[8];
>> diff --git
>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> index 6532a69f1fa8..a1e0f24bb91c 100644
>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> @@ -16,6 +16,8 @@
>> #include "../vdec_drv_if.h"
>> #include "../vdec_vpu_if.h"
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> /* reset_frame_context defined in VP9 spec */
>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>> b/drivers/media/platform/qcom/venus/hfi.h
>> index f25d412d6553..bd5ca5a8b945 100644
>> --- a/drivers/media/platform/qcom/venus/hfi.h
>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>> @@ -10,6 +10,8 @@
>>
>> #include "hfi_helper.h"
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> #define VIDC_SESSION_TYPE_VPE 0
>> #define VIDC_SESSION_TYPE_ENC 1
>> #define VIDC_SESSION_TYPE_DEC 2
>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>> b/drivers/media/platform/verisilicon/hantro_hw.h
>> index e83f0c523a30..9e8faf7ba6fb 100644
>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>> @@ -15,6 +15,8 @@
>> #include <media/v4l2-vp9.h>
>> #include <media/videobuf2-core.h>
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> #define DEC_8190_ALIGN_MASK 0x07U
>>
>> #define MB_DIM 16
>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>> index e530767e80a5..6627b5c2d4d6 100644
>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>> @@ -10,6 +10,8 @@
>> #include "ipu3.h"
>> #include "ipu3-dmamap.h"
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> /******************** v4l2_subdev_ops ********************/
>>
>> #define IPU3_RUNNING_MODE_VIDEO 0
>> diff --git a/include/media/videobuf2-core.h
>> b/include/media/videobuf2-core.h
>> index 77921cf894ef..080b783d608d 100644
>> --- a/include/media/videobuf2-core.h
>> +++ b/include/media/videobuf2-core.h
>> @@ -20,7 +20,6 @@
>> #include <media/media-request.h>
>> #include <media/frame_vector.h>
>>
>> -#define VB2_MAX_FRAME (32)
>> #define VB2_MAX_PLANES (8)
>>
>> /**
>> diff --git a/include/media/videobuf2-v4l2.h
>> b/include/media/videobuf2-v4l2.h
>> index 5a845887850b..88a7a565170e 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -15,10 +15,6 @@
>> #include <linux/videodev2.h>
>> #include <media/videobuf2-core.h>
>>
>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>> -#endif
>> -
>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>> #endif
>> --
>> 2.39.2
>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-03 8:09 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 8:09 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>
> On 6/22/23 21:13, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> After changing bufs arrays to a dynamic allocated array
>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>
> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> more reasonable.
>
> It would be hard to iterate the whole array, it would go worse with a
> filter. Such iterate may need to go twice because you mix
> post-processing buffer and decoding buffer(with MV) in the same array.
Here I don't want to change drivers behavior so I keep the same value.
If it happens that they need more buffers, like for dynamic resolution change
feature for Verisilicon VP9 decoder, case by case patches will be needed.
>
>> Remove it from the core definitions but keep it for drivers internal
>> needs.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>> drivers/media/platform/amphion/vdec.c | 1 +
>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>> include/media/videobuf2-core.h | 1 -
>> include/media/videobuf2-v4l2.h | 4 ----
>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index 86e1e926fa45..899783f67580 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -31,6 +31,8 @@
>>
>> #include <trace/events/vb2.h>
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> static int debug;
>> module_param(debug, int, 0644);
>>
>> diff --git a/drivers/media/platform/amphion/vdec.c
>> b/drivers/media/platform/amphion/vdec.c
>> index 3fa1a74a2e20..b3219f6d17fa 100644
>> --- a/drivers/media/platform/amphion/vdec.c
>> +++ b/drivers/media/platform/amphion/vdec.c
>> @@ -28,6 +28,7 @@
>>
>> #define VDEC_MIN_BUFFER_CAP 8
>> #define VDEC_MIN_BUFFER_OUT 8
>> +#define VB2_MAX_FRAME 32
>>
>> struct vdec_fs_info {
>> char name[8];
>> diff --git
>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> index 6532a69f1fa8..a1e0f24bb91c 100644
>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>> @@ -16,6 +16,8 @@
>> #include "../vdec_drv_if.h"
>> #include "../vdec_vpu_if.h"
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> /* reset_frame_context defined in VP9 spec */
>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>> b/drivers/media/platform/qcom/venus/hfi.h
>> index f25d412d6553..bd5ca5a8b945 100644
>> --- a/drivers/media/platform/qcom/venus/hfi.h
>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>> @@ -10,6 +10,8 @@
>>
>> #include "hfi_helper.h"
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> #define VIDC_SESSION_TYPE_VPE 0
>> #define VIDC_SESSION_TYPE_ENC 1
>> #define VIDC_SESSION_TYPE_DEC 2
>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>> b/drivers/media/platform/verisilicon/hantro_hw.h
>> index e83f0c523a30..9e8faf7ba6fb 100644
>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>> @@ -15,6 +15,8 @@
>> #include <media/v4l2-vp9.h>
>> #include <media/videobuf2-core.h>
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> #define DEC_8190_ALIGN_MASK 0x07U
>>
>> #define MB_DIM 16
>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>> index e530767e80a5..6627b5c2d4d6 100644
>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>> @@ -10,6 +10,8 @@
>> #include "ipu3.h"
>> #include "ipu3-dmamap.h"
>>
>> +#define VB2_MAX_FRAME 32
>> +
>> /******************** v4l2_subdev_ops ********************/
>>
>> #define IPU3_RUNNING_MODE_VIDEO 0
>> diff --git a/include/media/videobuf2-core.h
>> b/include/media/videobuf2-core.h
>> index 77921cf894ef..080b783d608d 100644
>> --- a/include/media/videobuf2-core.h
>> +++ b/include/media/videobuf2-core.h
>> @@ -20,7 +20,6 @@
>> #include <media/media-request.h>
>> #include <media/frame_vector.h>
>>
>> -#define VB2_MAX_FRAME (32)
>> #define VB2_MAX_PLANES (8)
>>
>> /**
>> diff --git a/include/media/videobuf2-v4l2.h
>> b/include/media/videobuf2-v4l2.h
>> index 5a845887850b..88a7a565170e 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -15,10 +15,6 @@
>> #include <linux/videodev2.h>
>> #include <media/videobuf2-core.h>
>>
>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>> -#endif
>> -
>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>> #endif
>> --
>> 2.39.2
>>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-07-03 8:09 ` Benjamin Gaignard
(?)
@ 2023-07-03 8:35 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 8:35 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
On 7/3/23 16:09, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>
>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> After changing bufs arrays to a dynamic allocated array
>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>
>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>> more reasonable.
>>
>> It would be hard to iterate the whole array, it would go worse with a
>> filter. Such iterate may need to go twice because you mix
>> post-processing buffer and decoding buffer(with MV) in the same array.
>
> Here I don't want to change drivers behavior so I keep the same value.
> If it happens that they need more buffers, like for dynamic resolution
> change
> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>
I just don't like the idea that using a variant length array here.
And I could explain why you won't need so many buffers for the
performance of decoding.
VP9 could support 10 reference frames in dpb.
Even for those frequent resolution changing test set, it would only
happen to two resolutions,
32 would be enough for 20 buffers of two resolution plus golden frames.
It also leaves enough slots for re-order latency.
If your case had more two resolutions, likes low->medium->high.
I would suggest just skip the medium resolutions, just allocate the
lower one first for fast playback then the highest for all the possible
medium cases. Reallocation happens frequently would only cause memory
fragment, nothing benefits your performance.
>
>>
>>> Remove it from the core definitions but keep it for drivers internal
>>> needs.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>> drivers/media/platform/amphion/vdec.c | 1 +
>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>> include/media/videobuf2-core.h | 1 -
>>> include/media/videobuf2-v4l2.h | 4 ----
>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 86e1e926fa45..899783f67580 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -31,6 +31,8 @@
>>>
>>> #include <trace/events/vb2.h>
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> static int debug;
>>> module_param(debug, int, 0644);
>>>
>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>> b/drivers/media/platform/amphion/vdec.c
>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>> --- a/drivers/media/platform/amphion/vdec.c
>>> +++ b/drivers/media/platform/amphion/vdec.c
>>> @@ -28,6 +28,7 @@
>>>
>>> #define VDEC_MIN_BUFFER_CAP 8
>>> #define VDEC_MIN_BUFFER_OUT 8
>>> +#define VB2_MAX_FRAME 32
>>>
>>> struct vdec_fs_info {
>>> char name[8];
>>> diff --git
>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> @@ -16,6 +16,8 @@
>>> #include "../vdec_drv_if.h"
>>> #include "../vdec_vpu_if.h"
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> /* reset_frame_context defined in VP9 spec */
>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>> b/drivers/media/platform/qcom/venus/hfi.h
>>> index f25d412d6553..bd5ca5a8b945 100644
>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>> @@ -10,6 +10,8 @@
>>>
>>> #include "hfi_helper.h"
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> #define VIDC_SESSION_TYPE_VPE 0
>>> #define VIDC_SESSION_TYPE_ENC 1
>>> #define VIDC_SESSION_TYPE_DEC 2
>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>> @@ -15,6 +15,8 @@
>>> #include <media/v4l2-vp9.h>
>>> #include <media/videobuf2-core.h>
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>
>>> #define MB_DIM 16
>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> index e530767e80a5..6627b5c2d4d6 100644
>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> @@ -10,6 +10,8 @@
>>> #include "ipu3.h"
>>> #include "ipu3-dmamap.h"
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> /******************** v4l2_subdev_ops ********************/
>>>
>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>> diff --git a/include/media/videobuf2-core.h
>>> b/include/media/videobuf2-core.h
>>> index 77921cf894ef..080b783d608d 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -20,7 +20,6 @@
>>> #include <media/media-request.h>
>>> #include <media/frame_vector.h>
>>>
>>> -#define VB2_MAX_FRAME (32)
>>> #define VB2_MAX_PLANES (8)
>>>
>>> /**
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h
>>> index 5a845887850b..88a7a565170e 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -15,10 +15,6 @@
>>> #include <linux/videodev2.h>
>>> #include <media/videobuf2-core.h>
>>>
>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>> -#endif
>>> -
>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>> #endif
>>> --
>>> 2.39.2
>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-03 8:35 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 8:35 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
On 7/3/23 16:09, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>
>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> After changing bufs arrays to a dynamic allocated array
>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>
>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>> more reasonable.
>>
>> It would be hard to iterate the whole array, it would go worse with a
>> filter. Such iterate may need to go twice because you mix
>> post-processing buffer and decoding buffer(with MV) in the same array.
>
> Here I don't want to change drivers behavior so I keep the same value.
> If it happens that they need more buffers, like for dynamic resolution
> change
> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>
I just don't like the idea that using a variant length array here.
And I could explain why you won't need so many buffers for the
performance of decoding.
VP9 could support 10 reference frames in dpb.
Even for those frequent resolution changing test set, it would only
happen to two resolutions,
32 would be enough for 20 buffers of two resolution plus golden frames.
It also leaves enough slots for re-order latency.
If your case had more two resolutions, likes low->medium->high.
I would suggest just skip the medium resolutions, just allocate the
lower one first for fast playback then the highest for all the possible
medium cases. Reallocation happens frequently would only cause memory
fragment, nothing benefits your performance.
>
>>
>>> Remove it from the core definitions but keep it for drivers internal
>>> needs.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>> drivers/media/platform/amphion/vdec.c | 1 +
>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>> include/media/videobuf2-core.h | 1 -
>>> include/media/videobuf2-v4l2.h | 4 ----
>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 86e1e926fa45..899783f67580 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -31,6 +31,8 @@
>>>
>>> #include <trace/events/vb2.h>
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> static int debug;
>>> module_param(debug, int, 0644);
>>>
>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>> b/drivers/media/platform/amphion/vdec.c
>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>> --- a/drivers/media/platform/amphion/vdec.c
>>> +++ b/drivers/media/platform/amphion/vdec.c
>>> @@ -28,6 +28,7 @@
>>>
>>> #define VDEC_MIN_BUFFER_CAP 8
>>> #define VDEC_MIN_BUFFER_OUT 8
>>> +#define VB2_MAX_FRAME 32
>>>
>>> struct vdec_fs_info {
>>> char name[8];
>>> diff --git
>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> @@ -16,6 +16,8 @@
>>> #include "../vdec_drv_if.h"
>>> #include "../vdec_vpu_if.h"
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> /* reset_frame_context defined in VP9 spec */
>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>> b/drivers/media/platform/qcom/venus/hfi.h
>>> index f25d412d6553..bd5ca5a8b945 100644
>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>> @@ -10,6 +10,8 @@
>>>
>>> #include "hfi_helper.h"
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> #define VIDC_SESSION_TYPE_VPE 0
>>> #define VIDC_SESSION_TYPE_ENC 1
>>> #define VIDC_SESSION_TYPE_DEC 2
>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>> @@ -15,6 +15,8 @@
>>> #include <media/v4l2-vp9.h>
>>> #include <media/videobuf2-core.h>
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>
>>> #define MB_DIM 16
>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> index e530767e80a5..6627b5c2d4d6 100644
>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> @@ -10,6 +10,8 @@
>>> #include "ipu3.h"
>>> #include "ipu3-dmamap.h"
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> /******************** v4l2_subdev_ops ********************/
>>>
>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>> diff --git a/include/media/videobuf2-core.h
>>> b/include/media/videobuf2-core.h
>>> index 77921cf894ef..080b783d608d 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -20,7 +20,6 @@
>>> #include <media/media-request.h>
>>> #include <media/frame_vector.h>
>>>
>>> -#define VB2_MAX_FRAME (32)
>>> #define VB2_MAX_PLANES (8)
>>>
>>> /**
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h
>>> index 5a845887850b..88a7a565170e 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -15,10 +15,6 @@
>>> #include <linux/videodev2.h>
>>> #include <media/videobuf2-core.h>
>>>
>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>> -#endif
>>> -
>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>> #endif
>>> --
>>> 2.39.2
>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-03 8:35 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 8:35 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
On 7/3/23 16:09, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>
>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> After changing bufs arrays to a dynamic allocated array
>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>
>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>> more reasonable.
>>
>> It would be hard to iterate the whole array, it would go worse with a
>> filter. Such iterate may need to go twice because you mix
>> post-processing buffer and decoding buffer(with MV) in the same array.
>
> Here I don't want to change drivers behavior so I keep the same value.
> If it happens that they need more buffers, like for dynamic resolution
> change
> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>
I just don't like the idea that using a variant length array here.
And I could explain why you won't need so many buffers for the
performance of decoding.
VP9 could support 10 reference frames in dpb.
Even for those frequent resolution changing test set, it would only
happen to two resolutions,
32 would be enough for 20 buffers of two resolution plus golden frames.
It also leaves enough slots for re-order latency.
If your case had more two resolutions, likes low->medium->high.
I would suggest just skip the medium resolutions, just allocate the
lower one first for fast playback then the highest for all the possible
medium cases. Reallocation happens frequently would only cause memory
fragment, nothing benefits your performance.
>
>>
>>> Remove it from the core definitions but keep it for drivers internal
>>> needs.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>> drivers/media/platform/amphion/vdec.c | 1 +
>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>> include/media/videobuf2-core.h | 1 -
>>> include/media/videobuf2-v4l2.h | 4 ----
>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 86e1e926fa45..899783f67580 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -31,6 +31,8 @@
>>>
>>> #include <trace/events/vb2.h>
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> static int debug;
>>> module_param(debug, int, 0644);
>>>
>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>> b/drivers/media/platform/amphion/vdec.c
>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>> --- a/drivers/media/platform/amphion/vdec.c
>>> +++ b/drivers/media/platform/amphion/vdec.c
>>> @@ -28,6 +28,7 @@
>>>
>>> #define VDEC_MIN_BUFFER_CAP 8
>>> #define VDEC_MIN_BUFFER_OUT 8
>>> +#define VB2_MAX_FRAME 32
>>>
>>> struct vdec_fs_info {
>>> char name[8];
>>> diff --git
>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>> @@ -16,6 +16,8 @@
>>> #include "../vdec_drv_if.h"
>>> #include "../vdec_vpu_if.h"
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> /* reset_frame_context defined in VP9 spec */
>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>> b/drivers/media/platform/qcom/venus/hfi.h
>>> index f25d412d6553..bd5ca5a8b945 100644
>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>> @@ -10,6 +10,8 @@
>>>
>>> #include "hfi_helper.h"
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> #define VIDC_SESSION_TYPE_VPE 0
>>> #define VIDC_SESSION_TYPE_ENC 1
>>> #define VIDC_SESSION_TYPE_DEC 2
>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>> @@ -15,6 +15,8 @@
>>> #include <media/v4l2-vp9.h>
>>> #include <media/videobuf2-core.h>
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>
>>> #define MB_DIM 16
>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> index e530767e80a5..6627b5c2d4d6 100644
>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>> @@ -10,6 +10,8 @@
>>> #include "ipu3.h"
>>> #include "ipu3-dmamap.h"
>>>
>>> +#define VB2_MAX_FRAME 32
>>> +
>>> /******************** v4l2_subdev_ops ********************/
>>>
>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>> diff --git a/include/media/videobuf2-core.h
>>> b/include/media/videobuf2-core.h
>>> index 77921cf894ef..080b783d608d 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -20,7 +20,6 @@
>>> #include <media/media-request.h>
>>> #include <media/frame_vector.h>
>>>
>>> -#define VB2_MAX_FRAME (32)
>>> #define VB2_MAX_PLANES (8)
>>>
>>> /**
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h
>>> index 5a845887850b..88a7a565170e 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -15,10 +15,6 @@
>>> #include <linux/videodev2.h>
>>> #include <media/videobuf2-core.h>
>>>
>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>> -#endif
>>> -
>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>> #endif
>>> --
>>> 2.39.2
>>>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-07-03 8:35 ` Hsia-Jun Li
(?)
@ 2023-07-03 10:53 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 10:53 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
Le 03/07/2023 à 10:35, Hsia-Jun Li a écrit :
>
> On 7/3/23 16:09, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>
>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> After changing bufs arrays to a dynamic allocated array
>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>
>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>> more reasonable.
>>>
>>> It would be hard to iterate the whole array, it would go worse with a
>>> filter. Such iterate may need to go twice because you mix
>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>
>> Here I don't want to change drivers behavior so I keep the same value.
>> If it happens that they need more buffers, like for dynamic
>> resolution change
>> feature for Verisilicon VP9 decoder, case by case patches will be
>> needed.
>>
> I just don't like the idea that using a variant length array here.
As far I have understand Hans and/or Laurent were happy to use that but I may have misunderstood them.
>
> And I could explain why you won't need so many buffers for the
> performance of decoding.
>
> VP9 could support 10 reference frames in dpb.
>
> Even for those frequent resolution changing test set, it would only
> happen to two resolutions,
>
> 32 would be enough for 20 buffers of two resolution plus golden
> frames. It also leaves enough slots for re-order latency.
>
> If your case had more two resolutions, likes low->medium->high.
VP9 header doesn't tell you that video resolution will change and in which way.
It cloud be from factor x1/2 to x16 and multiple time so you can use lot of (too much) buffers.
>
> I would suggest just skip the medium resolutions, just allocate the
> lower one first for fast playback then the highest for all the possible
>
> medium cases. Reallocation happens frequently would only cause memory
> fragment, nothing benefits your performance.
>
>>
>>>
>>>> Remove it from the core definitions but keep it for drivers internal
>>>> needs.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> ---
>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>> include/media/videobuf2-core.h | 1 -
>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index 86e1e926fa45..899783f67580 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -31,6 +31,8 @@
>>>>
>>>> #include <trace/events/vb2.h>
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> static int debug;
>>>> module_param(debug, int, 0644);
>>>>
>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>> b/drivers/media/platform/amphion/vdec.c
>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>> @@ -28,6 +28,7 @@
>>>>
>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>> +#define VB2_MAX_FRAME 32
>>>>
>>>> struct vdec_fs_info {
>>>> char name[8];
>>>> diff --git
>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>> ---
>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> +++
>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> @@ -16,6 +16,8 @@
>>>> #include "../vdec_drv_if.h"
>>>> #include "../vdec_vpu_if.h"
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> /* reset_frame_context defined in VP9 spec */
>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>> @@ -10,6 +10,8 @@
>>>>
>>>> #include "hfi_helper.h"
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>> @@ -15,6 +15,8 @@
>>>> #include <media/v4l2-vp9.h>
>>>> #include <media/videobuf2-core.h>
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>
>>>> #define MB_DIM 16
>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> @@ -10,6 +10,8 @@
>>>> #include "ipu3.h"
>>>> #include "ipu3-dmamap.h"
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> /******************** v4l2_subdev_ops ********************/
>>>>
>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>> diff --git a/include/media/videobuf2-core.h
>>>> b/include/media/videobuf2-core.h
>>>> index 77921cf894ef..080b783d608d 100644
>>>> --- a/include/media/videobuf2-core.h
>>>> +++ b/include/media/videobuf2-core.h
>>>> @@ -20,7 +20,6 @@
>>>> #include <media/media-request.h>
>>>> #include <media/frame_vector.h>
>>>>
>>>> -#define VB2_MAX_FRAME (32)
>>>> #define VB2_MAX_PLANES (8)
>>>>
>>>> /**
>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>> b/include/media/videobuf2-v4l2.h
>>>> index 5a845887850b..88a7a565170e 100644
>>>> --- a/include/media/videobuf2-v4l2.h
>>>> +++ b/include/media/videobuf2-v4l2.h
>>>> @@ -15,10 +15,6 @@
>>>> #include <linux/videodev2.h>
>>>> #include <media/videobuf2-core.h>
>>>>
>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>> -#endif
>>>> -
>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>> #endif
>>>> --
>>>> 2.39.2
>>>>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-03 10:53 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 10:53 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
Le 03/07/2023 à 10:35, Hsia-Jun Li a écrit :
>
> On 7/3/23 16:09, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>
>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> After changing bufs arrays to a dynamic allocated array
>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>
>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>> more reasonable.
>>>
>>> It would be hard to iterate the whole array, it would go worse with a
>>> filter. Such iterate may need to go twice because you mix
>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>
>> Here I don't want to change drivers behavior so I keep the same value.
>> If it happens that they need more buffers, like for dynamic
>> resolution change
>> feature for Verisilicon VP9 decoder, case by case patches will be
>> needed.
>>
> I just don't like the idea that using a variant length array here.
As far I have understand Hans and/or Laurent were happy to use that but I may have misunderstood them.
>
> And I could explain why you won't need so many buffers for the
> performance of decoding.
>
> VP9 could support 10 reference frames in dpb.
>
> Even for those frequent resolution changing test set, it would only
> happen to two resolutions,
>
> 32 would be enough for 20 buffers of two resolution plus golden
> frames. It also leaves enough slots for re-order latency.
>
> If your case had more two resolutions, likes low->medium->high.
VP9 header doesn't tell you that video resolution will change and in which way.
It cloud be from factor x1/2 to x16 and multiple time so you can use lot of (too much) buffers.
>
> I would suggest just skip the medium resolutions, just allocate the
> lower one first for fast playback then the highest for all the possible
>
> medium cases. Reallocation happens frequently would only cause memory
> fragment, nothing benefits your performance.
>
>>
>>>
>>>> Remove it from the core definitions but keep it for drivers internal
>>>> needs.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> ---
>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>> include/media/videobuf2-core.h | 1 -
>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index 86e1e926fa45..899783f67580 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -31,6 +31,8 @@
>>>>
>>>> #include <trace/events/vb2.h>
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> static int debug;
>>>> module_param(debug, int, 0644);
>>>>
>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>> b/drivers/media/platform/amphion/vdec.c
>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>> @@ -28,6 +28,7 @@
>>>>
>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>> +#define VB2_MAX_FRAME 32
>>>>
>>>> struct vdec_fs_info {
>>>> char name[8];
>>>> diff --git
>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>> ---
>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> +++
>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> @@ -16,6 +16,8 @@
>>>> #include "../vdec_drv_if.h"
>>>> #include "../vdec_vpu_if.h"
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> /* reset_frame_context defined in VP9 spec */
>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>> @@ -10,6 +10,8 @@
>>>>
>>>> #include "hfi_helper.h"
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>> @@ -15,6 +15,8 @@
>>>> #include <media/v4l2-vp9.h>
>>>> #include <media/videobuf2-core.h>
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>
>>>> #define MB_DIM 16
>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> @@ -10,6 +10,8 @@
>>>> #include "ipu3.h"
>>>> #include "ipu3-dmamap.h"
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> /******************** v4l2_subdev_ops ********************/
>>>>
>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>> diff --git a/include/media/videobuf2-core.h
>>>> b/include/media/videobuf2-core.h
>>>> index 77921cf894ef..080b783d608d 100644
>>>> --- a/include/media/videobuf2-core.h
>>>> +++ b/include/media/videobuf2-core.h
>>>> @@ -20,7 +20,6 @@
>>>> #include <media/media-request.h>
>>>> #include <media/frame_vector.h>
>>>>
>>>> -#define VB2_MAX_FRAME (32)
>>>> #define VB2_MAX_PLANES (8)
>>>>
>>>> /**
>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>> b/include/media/videobuf2-v4l2.h
>>>> index 5a845887850b..88a7a565170e 100644
>>>> --- a/include/media/videobuf2-v4l2.h
>>>> +++ b/include/media/videobuf2-v4l2.h
>>>> @@ -15,10 +15,6 @@
>>>> #include <linux/videodev2.h>
>>>> #include <media/videobuf2-core.h>
>>>>
>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>> -#endif
>>>> -
>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>> #endif
>>>> --
>>>> 2.39.2
>>>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-03 10:53 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 10:53 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
Le 03/07/2023 à 10:35, Hsia-Jun Li a écrit :
>
> On 7/3/23 16:09, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>
>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> After changing bufs arrays to a dynamic allocated array
>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>
>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>> more reasonable.
>>>
>>> It would be hard to iterate the whole array, it would go worse with a
>>> filter. Such iterate may need to go twice because you mix
>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>
>> Here I don't want to change drivers behavior so I keep the same value.
>> If it happens that they need more buffers, like for dynamic
>> resolution change
>> feature for Verisilicon VP9 decoder, case by case patches will be
>> needed.
>>
> I just don't like the idea that using a variant length array here.
As far I have understand Hans and/or Laurent were happy to use that but I may have misunderstood them.
>
> And I could explain why you won't need so many buffers for the
> performance of decoding.
>
> VP9 could support 10 reference frames in dpb.
>
> Even for those frequent resolution changing test set, it would only
> happen to two resolutions,
>
> 32 would be enough for 20 buffers of two resolution plus golden
> frames. It also leaves enough slots for re-order latency.
>
> If your case had more two resolutions, likes low->medium->high.
VP9 header doesn't tell you that video resolution will change and in which way.
It cloud be from factor x1/2 to x16 and multiple time so you can use lot of (too much) buffers.
>
> I would suggest just skip the medium resolutions, just allocate the
> lower one first for fast playback then the highest for all the possible
>
> medium cases. Reallocation happens frequently would only cause memory
> fragment, nothing benefits your performance.
>
>>
>>>
>>>> Remove it from the core definitions but keep it for drivers internal
>>>> needs.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> ---
>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>> include/media/videobuf2-core.h | 1 -
>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index 86e1e926fa45..899783f67580 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -31,6 +31,8 @@
>>>>
>>>> #include <trace/events/vb2.h>
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> static int debug;
>>>> module_param(debug, int, 0644);
>>>>
>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>> b/drivers/media/platform/amphion/vdec.c
>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>> @@ -28,6 +28,7 @@
>>>>
>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>> +#define VB2_MAX_FRAME 32
>>>>
>>>> struct vdec_fs_info {
>>>> char name[8];
>>>> diff --git
>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>> ---
>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> +++
>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>> @@ -16,6 +16,8 @@
>>>> #include "../vdec_drv_if.h"
>>>> #include "../vdec_vpu_if.h"
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> /* reset_frame_context defined in VP9 spec */
>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>> @@ -10,6 +10,8 @@
>>>>
>>>> #include "hfi_helper.h"
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>> @@ -15,6 +15,8 @@
>>>> #include <media/v4l2-vp9.h>
>>>> #include <media/videobuf2-core.h>
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>
>>>> #define MB_DIM 16
>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>> @@ -10,6 +10,8 @@
>>>> #include "ipu3.h"
>>>> #include "ipu3-dmamap.h"
>>>>
>>>> +#define VB2_MAX_FRAME 32
>>>> +
>>>> /******************** v4l2_subdev_ops ********************/
>>>>
>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>> diff --git a/include/media/videobuf2-core.h
>>>> b/include/media/videobuf2-core.h
>>>> index 77921cf894ef..080b783d608d 100644
>>>> --- a/include/media/videobuf2-core.h
>>>> +++ b/include/media/videobuf2-core.h
>>>> @@ -20,7 +20,6 @@
>>>> #include <media/media-request.h>
>>>> #include <media/frame_vector.h>
>>>>
>>>> -#define VB2_MAX_FRAME (32)
>>>> #define VB2_MAX_PLANES (8)
>>>>
>>>> /**
>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>> b/include/media/videobuf2-v4l2.h
>>>> index 5a845887850b..88a7a565170e 100644
>>>> --- a/include/media/videobuf2-v4l2.h
>>>> +++ b/include/media/videobuf2-v4l2.h
>>>> @@ -15,10 +15,6 @@
>>>> #include <linux/videodev2.h>
>>>> #include <media/videobuf2-core.h>
>>>>
>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>> -#endif
>>>> -
>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>> #endif
>>>> --
>>>> 2.39.2
>>>>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-07-03 10:53 ` Benjamin Gaignard
(?)
@ 2023-07-03 11:05 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 11:05 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
On 7/3/23 18:53, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 03/07/2023 à 10:35, Hsia-Jun Li a écrit :
>>
>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>
>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> After changing bufs arrays to a dynamic allocated array
>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>
>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>> more reasonable.
>>>>
>>>> It would be hard to iterate the whole array, it would go worse with a
>>>> filter. Such iterate may need to go twice because you mix
>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>
>>> Here I don't want to change drivers behavior so I keep the same value.
>>> If it happens that they need more buffers, like for dynamic
>>> resolution change
>>> feature for Verisilicon VP9 decoder, case by case patches will be
>>> needed.
>>>
>> I just don't like the idea that using a variant length array here.
>
> As far I have understand Hans and/or Laurent were happy to use that
> but I may have misunderstood them.
>
>>
>> And I could explain why you won't need so many buffers for the
>> performance of decoding.
>>
>> VP9 could support 10 reference frames in dpb.
>>
>> Even for those frequent resolution changing test set, it would only
>> happen to two resolutions,
>>
>> 32 would be enough for 20 buffers of two resolution plus golden
>> frames. It also leaves enough slots for re-order latency.
>>
>> If your case had more two resolutions, likes low->medium->high.
>
> VP9 header doesn't tell you that video resolution will change and in
> which way.
> It cloud be from factor x1/2 to x16 and multiple time so you can use
> lot of (too much) buffers.
I know VP9 doesn't have sequence header. I think you are talking about
scaling frame, which should be allocated when it is requested.
And this can't break the max reference capability requirement of VP9.
>
>>
>> I would suggest just skip the medium resolutions, just allocate the
>> lower one first for fast playback then the highest for all the possible
>>
>> medium cases. Reallocation happens frequently would only cause memory
>> fragment, nothing benefits your performance.
>>
>>>
>>>>
>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>> needs.
>>>>>
>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>> ---
>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>> include/media/videobuf2-core.h | 1 -
>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> index 86e1e926fa45..899783f67580 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> @@ -31,6 +31,8 @@
>>>>>
>>>>> #include <trace/events/vb2.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> static int debug;
>>>>> module_param(debug, int, 0644);
>>>>>
>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>> @@ -28,6 +28,7 @@
>>>>>
>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>> +#define VB2_MAX_FRAME 32
>>>>>
>>>>> struct vdec_fs_info {
>>>>> char name[8];
>>>>> diff --git
>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>> ---
>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> +++
>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> @@ -16,6 +16,8 @@
>>>>> #include "../vdec_drv_if.h"
>>>>> #include "../vdec_vpu_if.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /* reset_frame_context defined in VP9 spec */
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>> @@ -10,6 +10,8 @@
>>>>>
>>>>> #include "hfi_helper.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> @@ -15,6 +15,8 @@
>>>>> #include <media/v4l2-vp9.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>
>>>>> #define MB_DIM 16
>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> @@ -10,6 +10,8 @@
>>>>> #include "ipu3.h"
>>>>> #include "ipu3-dmamap.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>
>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>> diff --git a/include/media/videobuf2-core.h
>>>>> b/include/media/videobuf2-core.h
>>>>> index 77921cf894ef..080b783d608d 100644
>>>>> --- a/include/media/videobuf2-core.h
>>>>> +++ b/include/media/videobuf2-core.h
>>>>> @@ -20,7 +20,6 @@
>>>>> #include <media/media-request.h>
>>>>> #include <media/frame_vector.h>
>>>>>
>>>>> -#define VB2_MAX_FRAME (32)
>>>>> #define VB2_MAX_PLANES (8)
>>>>>
>>>>> /**
>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>> b/include/media/videobuf2-v4l2.h
>>>>> index 5a845887850b..88a7a565170e 100644
>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>> @@ -15,10 +15,6 @@
>>>>> #include <linux/videodev2.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#endif
>>>>> -
>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #endif
>>>>> --
>>>>> 2.39.2
>>>>>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-03 11:05 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 11:05 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
On 7/3/23 18:53, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 03/07/2023 à 10:35, Hsia-Jun Li a écrit :
>>
>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>
>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> After changing bufs arrays to a dynamic allocated array
>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>
>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>> more reasonable.
>>>>
>>>> It would be hard to iterate the whole array, it would go worse with a
>>>> filter. Such iterate may need to go twice because you mix
>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>
>>> Here I don't want to change drivers behavior so I keep the same value.
>>> If it happens that they need more buffers, like for dynamic
>>> resolution change
>>> feature for Verisilicon VP9 decoder, case by case patches will be
>>> needed.
>>>
>> I just don't like the idea that using a variant length array here.
>
> As far I have understand Hans and/or Laurent were happy to use that
> but I may have misunderstood them.
>
>>
>> And I could explain why you won't need so many buffers for the
>> performance of decoding.
>>
>> VP9 could support 10 reference frames in dpb.
>>
>> Even for those frequent resolution changing test set, it would only
>> happen to two resolutions,
>>
>> 32 would be enough for 20 buffers of two resolution plus golden
>> frames. It also leaves enough slots for re-order latency.
>>
>> If your case had more two resolutions, likes low->medium->high.
>
> VP9 header doesn't tell you that video resolution will change and in
> which way.
> It cloud be from factor x1/2 to x16 and multiple time so you can use
> lot of (too much) buffers.
I know VP9 doesn't have sequence header. I think you are talking about
scaling frame, which should be allocated when it is requested.
And this can't break the max reference capability requirement of VP9.
>
>>
>> I would suggest just skip the medium resolutions, just allocate the
>> lower one first for fast playback then the highest for all the possible
>>
>> medium cases. Reallocation happens frequently would only cause memory
>> fragment, nothing benefits your performance.
>>
>>>
>>>>
>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>> needs.
>>>>>
>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>> ---
>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>> include/media/videobuf2-core.h | 1 -
>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> index 86e1e926fa45..899783f67580 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> @@ -31,6 +31,8 @@
>>>>>
>>>>> #include <trace/events/vb2.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> static int debug;
>>>>> module_param(debug, int, 0644);
>>>>>
>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>> @@ -28,6 +28,7 @@
>>>>>
>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>> +#define VB2_MAX_FRAME 32
>>>>>
>>>>> struct vdec_fs_info {
>>>>> char name[8];
>>>>> diff --git
>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>> ---
>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> +++
>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> @@ -16,6 +16,8 @@
>>>>> #include "../vdec_drv_if.h"
>>>>> #include "../vdec_vpu_if.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /* reset_frame_context defined in VP9 spec */
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>> @@ -10,6 +10,8 @@
>>>>>
>>>>> #include "hfi_helper.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> @@ -15,6 +15,8 @@
>>>>> #include <media/v4l2-vp9.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>
>>>>> #define MB_DIM 16
>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> @@ -10,6 +10,8 @@
>>>>> #include "ipu3.h"
>>>>> #include "ipu3-dmamap.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>
>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>> diff --git a/include/media/videobuf2-core.h
>>>>> b/include/media/videobuf2-core.h
>>>>> index 77921cf894ef..080b783d608d 100644
>>>>> --- a/include/media/videobuf2-core.h
>>>>> +++ b/include/media/videobuf2-core.h
>>>>> @@ -20,7 +20,6 @@
>>>>> #include <media/media-request.h>
>>>>> #include <media/frame_vector.h>
>>>>>
>>>>> -#define VB2_MAX_FRAME (32)
>>>>> #define VB2_MAX_PLANES (8)
>>>>>
>>>>> /**
>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>> b/include/media/videobuf2-v4l2.h
>>>>> index 5a845887850b..88a7a565170e 100644
>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>> @@ -15,10 +15,6 @@
>>>>> #include <linux/videodev2.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#endif
>>>>> -
>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #endif
>>>>> --
>>>>> 2.39.2
>>>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-03 11:05 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 11:05 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, m.szyprowski,
linux-mediatek, linux-arm-msm, hverkuil-cisco, ezequiel, p.zabel,
linux-rockchip, mchehab, linux-staging, ming.qian, kernel,
gregkh, tfiga, nicolas.dufresne
On 7/3/23 18:53, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 03/07/2023 à 10:35, Hsia-Jun Li a écrit :
>>
>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>
>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> After changing bufs arrays to a dynamic allocated array
>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>
>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>> more reasonable.
>>>>
>>>> It would be hard to iterate the whole array, it would go worse with a
>>>> filter. Such iterate may need to go twice because you mix
>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>
>>> Here I don't want to change drivers behavior so I keep the same value.
>>> If it happens that they need more buffers, like for dynamic
>>> resolution change
>>> feature for Verisilicon VP9 decoder, case by case patches will be
>>> needed.
>>>
>> I just don't like the idea that using a variant length array here.
>
> As far I have understand Hans and/or Laurent were happy to use that
> but I may have misunderstood them.
>
>>
>> And I could explain why you won't need so many buffers for the
>> performance of decoding.
>>
>> VP9 could support 10 reference frames in dpb.
>>
>> Even for those frequent resolution changing test set, it would only
>> happen to two resolutions,
>>
>> 32 would be enough for 20 buffers of two resolution plus golden
>> frames. It also leaves enough slots for re-order latency.
>>
>> If your case had more two resolutions, likes low->medium->high.
>
> VP9 header doesn't tell you that video resolution will change and in
> which way.
> It cloud be from factor x1/2 to x16 and multiple time so you can use
> lot of (too much) buffers.
I know VP9 doesn't have sequence header. I think you are talking about
scaling frame, which should be allocated when it is requested.
And this can't break the max reference capability requirement of VP9.
>
>>
>> I would suggest just skip the medium resolutions, just allocate the
>> lower one first for fast playback then the highest for all the possible
>>
>> medium cases. Reallocation happens frequently would only cause memory
>> fragment, nothing benefits your performance.
>>
>>>
>>>>
>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>> needs.
>>>>>
>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>> ---
>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>> include/media/videobuf2-core.h | 1 -
>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> index 86e1e926fa45..899783f67580 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> @@ -31,6 +31,8 @@
>>>>>
>>>>> #include <trace/events/vb2.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> static int debug;
>>>>> module_param(debug, int, 0644);
>>>>>
>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>> @@ -28,6 +28,7 @@
>>>>>
>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>> +#define VB2_MAX_FRAME 32
>>>>>
>>>>> struct vdec_fs_info {
>>>>> char name[8];
>>>>> diff --git
>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>> ---
>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> +++
>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> @@ -16,6 +16,8 @@
>>>>> #include "../vdec_drv_if.h"
>>>>> #include "../vdec_vpu_if.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /* reset_frame_context defined in VP9 spec */
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>> @@ -10,6 +10,8 @@
>>>>>
>>>>> #include "hfi_helper.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> @@ -15,6 +15,8 @@
>>>>> #include <media/v4l2-vp9.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>
>>>>> #define MB_DIM 16
>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> @@ -10,6 +10,8 @@
>>>>> #include "ipu3.h"
>>>>> #include "ipu3-dmamap.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>
>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>> diff --git a/include/media/videobuf2-core.h
>>>>> b/include/media/videobuf2-core.h
>>>>> index 77921cf894ef..080b783d608d 100644
>>>>> --- a/include/media/videobuf2-core.h
>>>>> +++ b/include/media/videobuf2-core.h
>>>>> @@ -20,7 +20,6 @@
>>>>> #include <media/media-request.h>
>>>>> #include <media/frame_vector.h>
>>>>>
>>>>> -#define VB2_MAX_FRAME (32)
>>>>> #define VB2_MAX_PLANES (8)
>>>>>
>>>>> /**
>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>> b/include/media/videobuf2-v4l2.h
>>>>> index 5a845887850b..88a7a565170e 100644
>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>> @@ -15,10 +15,6 @@
>>>>> #include <linux/videodev2.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#endif
>>>>> -
>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #endif
>>>>> --
>>>>> 2.39.2
>>>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-07-03 8:35 ` Hsia-Jun Li
(?)
@ 2023-07-12 10:48 ` Tomasz Figa
-1 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-12 10:48 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>
> On 7/3/23 16:09, Benjamin Gaignard wrote:
> > CAUTION: Email originated externally, do not click links or open
> > attachments unless you recognize the sender and know the content is
> > safe.
> >
> >
> > Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> > >
> > > On 6/22/23 21:13, Benjamin Gaignard wrote:
> > > > CAUTION: Email originated externally, do not click links or open
> > > > attachments unless you recognize the sender and know the content is
> > > > safe.
> > > >
> > > >
> > > > After changing bufs arrays to a dynamic allocated array
> > > > VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> > >
> > > I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> > > more reasonable.
> > >
> > > It would be hard to iterate the whole array, it would go worse with a
> > > filter. Such iterate may need to go twice because you mix
> > > post-processing buffer and decoding buffer(with MV) in the same array.
> >
> > Here I don't want to change drivers behavior so I keep the same value.
> > If it happens that they need more buffers, like for dynamic resolution
> > change
> > feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >
> I just don't like the idea that using a variant length array here.
>
"I don't like" is not an argument. We had a number of arguments for
using a generic helper (originally idr, but later decided to go with
XArray, because the former is now deprecated) that we pointed out in
our review comments for previous revisions. It wasn't really about the
size being variable, but rather avoiding open coding things in vb2 and
duplicating what's already implemented in generic code.
> And I could explain why you won't need so many buffers for the performance
> of decoding.
>
> VP9 could support 10 reference frames in dpb.
>
> Even for those frequent resolution changing test set, it would only happen
> to two resolutions,
>
> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> also leaves enough slots for re-order latency.
>
> If your case had more two resolutions, likes low->medium->high.
>
> I would suggest just skip the medium resolutions, just allocate the lower
> one first for fast playback then the highest for all the possible
>
> medium cases. Reallocation happens frequently would only cause memory
> fragment, nothing benefits your performance.
>
We have mechanisms in the kernel to deal with memory fragmentation
(migration/compaction) and it would still only matters for the
pathologic cases of hardware that require physically contiguous memory.
Modern hardware with proper DMA capabilities can either scatter-gather
or are equipped with an IOMMU, so the allocation always happens in page
granularity and fragmentation is avoided.
Best regards,
Tomasz
> >
> > >
> > > > Remove it from the core definitions but keep it for drivers internal
> > > > needs.
> > > >
> > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > > ---
> > > > drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> > > > drivers/media/platform/amphion/vdec.c | 1 +
> > > > .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> > > > drivers/media/platform/qcom/venus/hfi.h | 2 ++
> > > > drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> > > > drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> > > > include/media/videobuf2-core.h | 1 -
> > > > include/media/videobuf2-v4l2.h | 4 ----
> > > > 8 files changed, 11 insertions(+), 5 deletions(-)
> > > >
> > > > diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > index 86e1e926fa45..899783f67580 100644
> > > > --- a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > @@ -31,6 +31,8 @@
> > > >
> > > > #include <trace/events/vb2.h>
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > static int debug;
> > > > module_param(debug, int, 0644);
> > > >
> > > > diff --git a/drivers/media/platform/amphion/vdec.c
> > > > b/drivers/media/platform/amphion/vdec.c
> > > > index 3fa1a74a2e20..b3219f6d17fa 100644
> > > > --- a/drivers/media/platform/amphion/vdec.c
> > > > +++ b/drivers/media/platform/amphion/vdec.c
> > > > @@ -28,6 +28,7 @@
> > > >
> > > > #define VDEC_MIN_BUFFER_CAP 8
> > > > #define VDEC_MIN_BUFFER_OUT 8
> > > > +#define VB2_MAX_FRAME 32
> > > >
> > > > struct vdec_fs_info {
> > > > char name[8];
> > > > diff --git
> > > > a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > index 6532a69f1fa8..a1e0f24bb91c 100644
> > > > --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > @@ -16,6 +16,8 @@
> > > > #include "../vdec_drv_if.h"
> > > > #include "../vdec_vpu_if.h"
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > /* reset_frame_context defined in VP9 spec */
> > > > #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> > > > #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> > > > diff --git a/drivers/media/platform/qcom/venus/hfi.h
> > > > b/drivers/media/platform/qcom/venus/hfi.h
> > > > index f25d412d6553..bd5ca5a8b945 100644
> > > > --- a/drivers/media/platform/qcom/venus/hfi.h
> > > > +++ b/drivers/media/platform/qcom/venus/hfi.h
> > > > @@ -10,6 +10,8 @@
> > > >
> > > > #include "hfi_helper.h"
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > #define VIDC_SESSION_TYPE_VPE 0
> > > > #define VIDC_SESSION_TYPE_ENC 1
> > > > #define VIDC_SESSION_TYPE_DEC 2
> > > > diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> > > > b/drivers/media/platform/verisilicon/hantro_hw.h
> > > > index e83f0c523a30..9e8faf7ba6fb 100644
> > > > --- a/drivers/media/platform/verisilicon/hantro_hw.h
> > > > +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> > > > @@ -15,6 +15,8 @@
> > > > #include <media/v4l2-vp9.h>
> > > > #include <media/videobuf2-core.h>
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > #define DEC_8190_ALIGN_MASK 0x07U
> > > >
> > > > #define MB_DIM 16
> > > > diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > b/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > index e530767e80a5..6627b5c2d4d6 100644
> > > > --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > @@ -10,6 +10,8 @@
> > > > #include "ipu3.h"
> > > > #include "ipu3-dmamap.h"
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > /******************** v4l2_subdev_ops ********************/
> > > >
> > > > #define IPU3_RUNNING_MODE_VIDEO 0
> > > > diff --git a/include/media/videobuf2-core.h
> > > > b/include/media/videobuf2-core.h
> > > > index 77921cf894ef..080b783d608d 100644
> > > > --- a/include/media/videobuf2-core.h
> > > > +++ b/include/media/videobuf2-core.h
> > > > @@ -20,7 +20,6 @@
> > > > #include <media/media-request.h>
> > > > #include <media/frame_vector.h>
> > > >
> > > > -#define VB2_MAX_FRAME (32)
> > > > #define VB2_MAX_PLANES (8)
> > > >
> > > > /**
> > > > diff --git a/include/media/videobuf2-v4l2.h
> > > > b/include/media/videobuf2-v4l2.h
> > > > index 5a845887850b..88a7a565170e 100644
> > > > --- a/include/media/videobuf2-v4l2.h
> > > > +++ b/include/media/videobuf2-v4l2.h
> > > > @@ -15,10 +15,6 @@
> > > > #include <linux/videodev2.h>
> > > > #include <media/videobuf2-core.h>
> > > >
> > > > -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> > > > -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> > > > -#endif
> > > > -
> > > > #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> > > > #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> > > > #endif
> > > > --
> > > > 2.39.2
> > > >
> --
> Hsia-Jun(Randy) Li
>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-12 10:48 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-12 10:48 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>
> On 7/3/23 16:09, Benjamin Gaignard wrote:
> > CAUTION: Email originated externally, do not click links or open
> > attachments unless you recognize the sender and know the content is
> > safe.
> >
> >
> > Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> > >
> > > On 6/22/23 21:13, Benjamin Gaignard wrote:
> > > > CAUTION: Email originated externally, do not click links or open
> > > > attachments unless you recognize the sender and know the content is
> > > > safe.
> > > >
> > > >
> > > > After changing bufs arrays to a dynamic allocated array
> > > > VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> > >
> > > I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> > > more reasonable.
> > >
> > > It would be hard to iterate the whole array, it would go worse with a
> > > filter. Such iterate may need to go twice because you mix
> > > post-processing buffer and decoding buffer(with MV) in the same array.
> >
> > Here I don't want to change drivers behavior so I keep the same value.
> > If it happens that they need more buffers, like for dynamic resolution
> > change
> > feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >
> I just don't like the idea that using a variant length array here.
>
"I don't like" is not an argument. We had a number of arguments for
using a generic helper (originally idr, but later decided to go with
XArray, because the former is now deprecated) that we pointed out in
our review comments for previous revisions. It wasn't really about the
size being variable, but rather avoiding open coding things in vb2 and
duplicating what's already implemented in generic code.
> And I could explain why you won't need so many buffers for the performance
> of decoding.
>
> VP9 could support 10 reference frames in dpb.
>
> Even for those frequent resolution changing test set, it would only happen
> to two resolutions,
>
> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> also leaves enough slots for re-order latency.
>
> If your case had more two resolutions, likes low->medium->high.
>
> I would suggest just skip the medium resolutions, just allocate the lower
> one first for fast playback then the highest for all the possible
>
> medium cases. Reallocation happens frequently would only cause memory
> fragment, nothing benefits your performance.
>
We have mechanisms in the kernel to deal with memory fragmentation
(migration/compaction) and it would still only matters for the
pathologic cases of hardware that require physically contiguous memory.
Modern hardware with proper DMA capabilities can either scatter-gather
or are equipped with an IOMMU, so the allocation always happens in page
granularity and fragmentation is avoided.
Best regards,
Tomasz
> >
> > >
> > > > Remove it from the core definitions but keep it for drivers internal
> > > > needs.
> > > >
> > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > > ---
> > > > drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> > > > drivers/media/platform/amphion/vdec.c | 1 +
> > > > .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> > > > drivers/media/platform/qcom/venus/hfi.h | 2 ++
> > > > drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> > > > drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> > > > include/media/videobuf2-core.h | 1 -
> > > > include/media/videobuf2-v4l2.h | 4 ----
> > > > 8 files changed, 11 insertions(+), 5 deletions(-)
> > > >
> > > > diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > index 86e1e926fa45..899783f67580 100644
> > > > --- a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > @@ -31,6 +31,8 @@
> > > >
> > > > #include <trace/events/vb2.h>
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > static int debug;
> > > > module_param(debug, int, 0644);
> > > >
> > > > diff --git a/drivers/media/platform/amphion/vdec.c
> > > > b/drivers/media/platform/amphion/vdec.c
> > > > index 3fa1a74a2e20..b3219f6d17fa 100644
> > > > --- a/drivers/media/platform/amphion/vdec.c
> > > > +++ b/drivers/media/platform/amphion/vdec.c
> > > > @@ -28,6 +28,7 @@
> > > >
> > > > #define VDEC_MIN_BUFFER_CAP 8
> > > > #define VDEC_MIN_BUFFER_OUT 8
> > > > +#define VB2_MAX_FRAME 32
> > > >
> > > > struct vdec_fs_info {
> > > > char name[8];
> > > > diff --git
> > > > a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > index 6532a69f1fa8..a1e0f24bb91c 100644
> > > > --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > @@ -16,6 +16,8 @@
> > > > #include "../vdec_drv_if.h"
> > > > #include "../vdec_vpu_if.h"
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > /* reset_frame_context defined in VP9 spec */
> > > > #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> > > > #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> > > > diff --git a/drivers/media/platform/qcom/venus/hfi.h
> > > > b/drivers/media/platform/qcom/venus/hfi.h
> > > > index f25d412d6553..bd5ca5a8b945 100644
> > > > --- a/drivers/media/platform/qcom/venus/hfi.h
> > > > +++ b/drivers/media/platform/qcom/venus/hfi.h
> > > > @@ -10,6 +10,8 @@
> > > >
> > > > #include "hfi_helper.h"
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > #define VIDC_SESSION_TYPE_VPE 0
> > > > #define VIDC_SESSION_TYPE_ENC 1
> > > > #define VIDC_SESSION_TYPE_DEC 2
> > > > diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> > > > b/drivers/media/platform/verisilicon/hantro_hw.h
> > > > index e83f0c523a30..9e8faf7ba6fb 100644
> > > > --- a/drivers/media/platform/verisilicon/hantro_hw.h
> > > > +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> > > > @@ -15,6 +15,8 @@
> > > > #include <media/v4l2-vp9.h>
> > > > #include <media/videobuf2-core.h>
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > #define DEC_8190_ALIGN_MASK 0x07U
> > > >
> > > > #define MB_DIM 16
> > > > diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > b/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > index e530767e80a5..6627b5c2d4d6 100644
> > > > --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > @@ -10,6 +10,8 @@
> > > > #include "ipu3.h"
> > > > #include "ipu3-dmamap.h"
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > /******************** v4l2_subdev_ops ********************/
> > > >
> > > > #define IPU3_RUNNING_MODE_VIDEO 0
> > > > diff --git a/include/media/videobuf2-core.h
> > > > b/include/media/videobuf2-core.h
> > > > index 77921cf894ef..080b783d608d 100644
> > > > --- a/include/media/videobuf2-core.h
> > > > +++ b/include/media/videobuf2-core.h
> > > > @@ -20,7 +20,6 @@
> > > > #include <media/media-request.h>
> > > > #include <media/frame_vector.h>
> > > >
> > > > -#define VB2_MAX_FRAME (32)
> > > > #define VB2_MAX_PLANES (8)
> > > >
> > > > /**
> > > > diff --git a/include/media/videobuf2-v4l2.h
> > > > b/include/media/videobuf2-v4l2.h
> > > > index 5a845887850b..88a7a565170e 100644
> > > > --- a/include/media/videobuf2-v4l2.h
> > > > +++ b/include/media/videobuf2-v4l2.h
> > > > @@ -15,10 +15,6 @@
> > > > #include <linux/videodev2.h>
> > > > #include <media/videobuf2-core.h>
> > > >
> > > > -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> > > > -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> > > > -#endif
> > > > -
> > > > #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> > > > #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> > > > #endif
> > > > --
> > > > 2.39.2
> > > >
> --
> Hsia-Jun(Randy) Li
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-12 10:48 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-12 10:48 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>
> On 7/3/23 16:09, Benjamin Gaignard wrote:
> > CAUTION: Email originated externally, do not click links or open
> > attachments unless you recognize the sender and know the content is
> > safe.
> >
> >
> > Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> > >
> > > On 6/22/23 21:13, Benjamin Gaignard wrote:
> > > > CAUTION: Email originated externally, do not click links or open
> > > > attachments unless you recognize the sender and know the content is
> > > > safe.
> > > >
> > > >
> > > > After changing bufs arrays to a dynamic allocated array
> > > > VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> > >
> > > I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> > > more reasonable.
> > >
> > > It would be hard to iterate the whole array, it would go worse with a
> > > filter. Such iterate may need to go twice because you mix
> > > post-processing buffer and decoding buffer(with MV) in the same array.
> >
> > Here I don't want to change drivers behavior so I keep the same value.
> > If it happens that they need more buffers, like for dynamic resolution
> > change
> > feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >
> I just don't like the idea that using a variant length array here.
>
"I don't like" is not an argument. We had a number of arguments for
using a generic helper (originally idr, but later decided to go with
XArray, because the former is now deprecated) that we pointed out in
our review comments for previous revisions. It wasn't really about the
size being variable, but rather avoiding open coding things in vb2 and
duplicating what's already implemented in generic code.
> And I could explain why you won't need so many buffers for the performance
> of decoding.
>
> VP9 could support 10 reference frames in dpb.
>
> Even for those frequent resolution changing test set, it would only happen
> to two resolutions,
>
> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> also leaves enough slots for re-order latency.
>
> If your case had more two resolutions, likes low->medium->high.
>
> I would suggest just skip the medium resolutions, just allocate the lower
> one first for fast playback then the highest for all the possible
>
> medium cases. Reallocation happens frequently would only cause memory
> fragment, nothing benefits your performance.
>
We have mechanisms in the kernel to deal with memory fragmentation
(migration/compaction) and it would still only matters for the
pathologic cases of hardware that require physically contiguous memory.
Modern hardware with proper DMA capabilities can either scatter-gather
or are equipped with an IOMMU, so the allocation always happens in page
granularity and fragmentation is avoided.
Best regards,
Tomasz
> >
> > >
> > > > Remove it from the core definitions but keep it for drivers internal
> > > > needs.
> > > >
> > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > > ---
> > > > drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> > > > drivers/media/platform/amphion/vdec.c | 1 +
> > > > .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> > > > drivers/media/platform/qcom/venus/hfi.h | 2 ++
> > > > drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> > > > drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> > > > include/media/videobuf2-core.h | 1 -
> > > > include/media/videobuf2-v4l2.h | 4 ----
> > > > 8 files changed, 11 insertions(+), 5 deletions(-)
> > > >
> > > > diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > index 86e1e926fa45..899783f67580 100644
> > > > --- a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > @@ -31,6 +31,8 @@
> > > >
> > > > #include <trace/events/vb2.h>
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > static int debug;
> > > > module_param(debug, int, 0644);
> > > >
> > > > diff --git a/drivers/media/platform/amphion/vdec.c
> > > > b/drivers/media/platform/amphion/vdec.c
> > > > index 3fa1a74a2e20..b3219f6d17fa 100644
> > > > --- a/drivers/media/platform/amphion/vdec.c
> > > > +++ b/drivers/media/platform/amphion/vdec.c
> > > > @@ -28,6 +28,7 @@
> > > >
> > > > #define VDEC_MIN_BUFFER_CAP 8
> > > > #define VDEC_MIN_BUFFER_OUT 8
> > > > +#define VB2_MAX_FRAME 32
> > > >
> > > > struct vdec_fs_info {
> > > > char name[8];
> > > > diff --git
> > > > a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > index 6532a69f1fa8..a1e0f24bb91c 100644
> > > > --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> > > > @@ -16,6 +16,8 @@
> > > > #include "../vdec_drv_if.h"
> > > > #include "../vdec_vpu_if.h"
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > /* reset_frame_context defined in VP9 spec */
> > > > #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> > > > #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> > > > diff --git a/drivers/media/platform/qcom/venus/hfi.h
> > > > b/drivers/media/platform/qcom/venus/hfi.h
> > > > index f25d412d6553..bd5ca5a8b945 100644
> > > > --- a/drivers/media/platform/qcom/venus/hfi.h
> > > > +++ b/drivers/media/platform/qcom/venus/hfi.h
> > > > @@ -10,6 +10,8 @@
> > > >
> > > > #include "hfi_helper.h"
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > #define VIDC_SESSION_TYPE_VPE 0
> > > > #define VIDC_SESSION_TYPE_ENC 1
> > > > #define VIDC_SESSION_TYPE_DEC 2
> > > > diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> > > > b/drivers/media/platform/verisilicon/hantro_hw.h
> > > > index e83f0c523a30..9e8faf7ba6fb 100644
> > > > --- a/drivers/media/platform/verisilicon/hantro_hw.h
> > > > +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> > > > @@ -15,6 +15,8 @@
> > > > #include <media/v4l2-vp9.h>
> > > > #include <media/videobuf2-core.h>
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > #define DEC_8190_ALIGN_MASK 0x07U
> > > >
> > > > #define MB_DIM 16
> > > > diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > b/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > index e530767e80a5..6627b5c2d4d6 100644
> > > > --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> > > > @@ -10,6 +10,8 @@
> > > > #include "ipu3.h"
> > > > #include "ipu3-dmamap.h"
> > > >
> > > > +#define VB2_MAX_FRAME 32
> > > > +
> > > > /******************** v4l2_subdev_ops ********************/
> > > >
> > > > #define IPU3_RUNNING_MODE_VIDEO 0
> > > > diff --git a/include/media/videobuf2-core.h
> > > > b/include/media/videobuf2-core.h
> > > > index 77921cf894ef..080b783d608d 100644
> > > > --- a/include/media/videobuf2-core.h
> > > > +++ b/include/media/videobuf2-core.h
> > > > @@ -20,7 +20,6 @@
> > > > #include <media/media-request.h>
> > > > #include <media/frame_vector.h>
> > > >
> > > > -#define VB2_MAX_FRAME (32)
> > > > #define VB2_MAX_PLANES (8)
> > > >
> > > > /**
> > > > diff --git a/include/media/videobuf2-v4l2.h
> > > > b/include/media/videobuf2-v4l2.h
> > > > index 5a845887850b..88a7a565170e 100644
> > > > --- a/include/media/videobuf2-v4l2.h
> > > > +++ b/include/media/videobuf2-v4l2.h
> > > > @@ -15,10 +15,6 @@
> > > > #include <linux/videodev2.h>
> > > > #include <media/videobuf2-core.h>
> > > >
> > > > -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> > > > -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> > > > -#endif
> > > > -
> > > > #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> > > > #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> > > > #endif
> > > > --
> > > > 2.39.2
> > > >
> --
> Hsia-Jun(Randy) Li
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-07-12 10:48 ` Tomasz Figa
(?)
@ 2023-07-17 7:44 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-17 7:44 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On 7/12/23 18:48, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> After changing bufs arrays to a dynamic allocated array
>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>> more reasonable.
>>>>
>>>> It would be hard to iterate the whole array, it would go worse with a
>>>> filter. Such iterate may need to go twice because you mix
>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>> Here I don't want to change drivers behavior so I keep the same value.
>>> If it happens that they need more buffers, like for dynamic resolution
>>> change
>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>
>> I just don't like the idea that using a variant length array here.
>>
> "I don't like" is not an argument. We had a number of arguments for
> using a generic helper (originally idr, but later decided to go with
> XArray, because the former is now deprecated) that we pointed out in
> our review comments for previous revisions. It wasn't really about the
> size being variable, but rather avoiding open coding things in vb2 and
> duplicating what's already implemented in generic code.
I just want to say I don't think we need a variable length array to
store the buffer here.
And the below is the reason that such a case could be avoided in the
first place.
>
>> And I could explain why you won't need so many buffers for the performance
>> of decoding.
>>
>> VP9 could support 10 reference frames in dpb.
>>
>> Even for those frequent resolution changing test set, it would only happen
>> to two resolutions,
>>
>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>> also leaves enough slots for re-order latency.
>>
>> If your case had more two resolutions, likes low->medium->high.
>>
>> I would suggest just skip the medium resolutions, just allocate the lower
>> one first for fast playback then the highest for all the possible
>>
>> medium cases. Reallocation happens frequently would only cause memory
>> fragment, nothing benefits your performance.
>>
> We have mechanisms in the kernel to deal with memory fragmentation
> (migration/compaction) and it would still only matters for the
> pathologic cases of hardware that require physically contiguous memory.
> Modern hardware with proper DMA capabilities can either scatter-gather
> or are equipped with an IOMMU, so the allocation always happens in page
> granularity and fragmentation is avoided.
Unfortunately, there are more devices that didn't have a IOMMU attached
to it, supporting scatter gather is more odd.
It would be more likely that IOMMU would be disabled for the performance
reason.
>
> Best regards,
> Tomasz
>
>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>> needs.
>>>>>
>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>> ---
>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>> include/media/videobuf2-core.h | 1 -
>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> index 86e1e926fa45..899783f67580 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> @@ -31,6 +31,8 @@
>>>>>
>>>>> #include <trace/events/vb2.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> static int debug;
>>>>> module_param(debug, int, 0644);
>>>>>
>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>> @@ -28,6 +28,7 @@
>>>>>
>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>> +#define VB2_MAX_FRAME 32
>>>>>
>>>>> struct vdec_fs_info {
>>>>> char name[8];
>>>>> diff --git
>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> @@ -16,6 +16,8 @@
>>>>> #include "../vdec_drv_if.h"
>>>>> #include "../vdec_vpu_if.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /* reset_frame_context defined in VP9 spec */
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>> @@ -10,6 +10,8 @@
>>>>>
>>>>> #include "hfi_helper.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> @@ -15,6 +15,8 @@
>>>>> #include <media/v4l2-vp9.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>
>>>>> #define MB_DIM 16
>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> @@ -10,6 +10,8 @@
>>>>> #include "ipu3.h"
>>>>> #include "ipu3-dmamap.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>
>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>> diff --git a/include/media/videobuf2-core.h
>>>>> b/include/media/videobuf2-core.h
>>>>> index 77921cf894ef..080b783d608d 100644
>>>>> --- a/include/media/videobuf2-core.h
>>>>> +++ b/include/media/videobuf2-core.h
>>>>> @@ -20,7 +20,6 @@
>>>>> #include <media/media-request.h>
>>>>> #include <media/frame_vector.h>
>>>>>
>>>>> -#define VB2_MAX_FRAME (32)
>>>>> #define VB2_MAX_PLANES (8)
>>>>>
>>>>> /**
>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>> b/include/media/videobuf2-v4l2.h
>>>>> index 5a845887850b..88a7a565170e 100644
>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>> @@ -15,10 +15,6 @@
>>>>> #include <linux/videodev2.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#endif
>>>>> -
>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #endif
>>>>> --
>>>>> 2.39.2
>>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-17 7:44 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-17 7:44 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On 7/12/23 18:48, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> After changing bufs arrays to a dynamic allocated array
>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>> more reasonable.
>>>>
>>>> It would be hard to iterate the whole array, it would go worse with a
>>>> filter. Such iterate may need to go twice because you mix
>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>> Here I don't want to change drivers behavior so I keep the same value.
>>> If it happens that they need more buffers, like for dynamic resolution
>>> change
>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>
>> I just don't like the idea that using a variant length array here.
>>
> "I don't like" is not an argument. We had a number of arguments for
> using a generic helper (originally idr, but later decided to go with
> XArray, because the former is now deprecated) that we pointed out in
> our review comments for previous revisions. It wasn't really about the
> size being variable, but rather avoiding open coding things in vb2 and
> duplicating what's already implemented in generic code.
I just want to say I don't think we need a variable length array to
store the buffer here.
And the below is the reason that such a case could be avoided in the
first place.
>
>> And I could explain why you won't need so many buffers for the performance
>> of decoding.
>>
>> VP9 could support 10 reference frames in dpb.
>>
>> Even for those frequent resolution changing test set, it would only happen
>> to two resolutions,
>>
>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>> also leaves enough slots for re-order latency.
>>
>> If your case had more two resolutions, likes low->medium->high.
>>
>> I would suggest just skip the medium resolutions, just allocate the lower
>> one first for fast playback then the highest for all the possible
>>
>> medium cases. Reallocation happens frequently would only cause memory
>> fragment, nothing benefits your performance.
>>
> We have mechanisms in the kernel to deal with memory fragmentation
> (migration/compaction) and it would still only matters for the
> pathologic cases of hardware that require physically contiguous memory.
> Modern hardware with proper DMA capabilities can either scatter-gather
> or are equipped with an IOMMU, so the allocation always happens in page
> granularity and fragmentation is avoided.
Unfortunately, there are more devices that didn't have a IOMMU attached
to it, supporting scatter gather is more odd.
It would be more likely that IOMMU would be disabled for the performance
reason.
>
> Best regards,
> Tomasz
>
>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>> needs.
>>>>>
>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>> ---
>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>> include/media/videobuf2-core.h | 1 -
>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> index 86e1e926fa45..899783f67580 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> @@ -31,6 +31,8 @@
>>>>>
>>>>> #include <trace/events/vb2.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> static int debug;
>>>>> module_param(debug, int, 0644);
>>>>>
>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>> @@ -28,6 +28,7 @@
>>>>>
>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>> +#define VB2_MAX_FRAME 32
>>>>>
>>>>> struct vdec_fs_info {
>>>>> char name[8];
>>>>> diff --git
>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> @@ -16,6 +16,8 @@
>>>>> #include "../vdec_drv_if.h"
>>>>> #include "../vdec_vpu_if.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /* reset_frame_context defined in VP9 spec */
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>> @@ -10,6 +10,8 @@
>>>>>
>>>>> #include "hfi_helper.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> @@ -15,6 +15,8 @@
>>>>> #include <media/v4l2-vp9.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>
>>>>> #define MB_DIM 16
>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> @@ -10,6 +10,8 @@
>>>>> #include "ipu3.h"
>>>>> #include "ipu3-dmamap.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>
>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>> diff --git a/include/media/videobuf2-core.h
>>>>> b/include/media/videobuf2-core.h
>>>>> index 77921cf894ef..080b783d608d 100644
>>>>> --- a/include/media/videobuf2-core.h
>>>>> +++ b/include/media/videobuf2-core.h
>>>>> @@ -20,7 +20,6 @@
>>>>> #include <media/media-request.h>
>>>>> #include <media/frame_vector.h>
>>>>>
>>>>> -#define VB2_MAX_FRAME (32)
>>>>> #define VB2_MAX_PLANES (8)
>>>>>
>>>>> /**
>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>> b/include/media/videobuf2-v4l2.h
>>>>> index 5a845887850b..88a7a565170e 100644
>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>> @@ -15,10 +15,6 @@
>>>>> #include <linux/videodev2.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#endif
>>>>> -
>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #endif
>>>>> --
>>>>> 2.39.2
>>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-17 7:44 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-17 7:44 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On 7/12/23 18:48, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> After changing bufs arrays to a dynamic allocated array
>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>> more reasonable.
>>>>
>>>> It would be hard to iterate the whole array, it would go worse with a
>>>> filter. Such iterate may need to go twice because you mix
>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>> Here I don't want to change drivers behavior so I keep the same value.
>>> If it happens that they need more buffers, like for dynamic resolution
>>> change
>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>
>> I just don't like the idea that using a variant length array here.
>>
> "I don't like" is not an argument. We had a number of arguments for
> using a generic helper (originally idr, but later decided to go with
> XArray, because the former is now deprecated) that we pointed out in
> our review comments for previous revisions. It wasn't really about the
> size being variable, but rather avoiding open coding things in vb2 and
> duplicating what's already implemented in generic code.
I just want to say I don't think we need a variable length array to
store the buffer here.
And the below is the reason that such a case could be avoided in the
first place.
>
>> And I could explain why you won't need so many buffers for the performance
>> of decoding.
>>
>> VP9 could support 10 reference frames in dpb.
>>
>> Even for those frequent resolution changing test set, it would only happen
>> to two resolutions,
>>
>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>> also leaves enough slots for re-order latency.
>>
>> If your case had more two resolutions, likes low->medium->high.
>>
>> I would suggest just skip the medium resolutions, just allocate the lower
>> one first for fast playback then the highest for all the possible
>>
>> medium cases. Reallocation happens frequently would only cause memory
>> fragment, nothing benefits your performance.
>>
> We have mechanisms in the kernel to deal with memory fragmentation
> (migration/compaction) and it would still only matters for the
> pathologic cases of hardware that require physically contiguous memory.
> Modern hardware with proper DMA capabilities can either scatter-gather
> or are equipped with an IOMMU, so the allocation always happens in page
> granularity and fragmentation is avoided.
Unfortunately, there are more devices that didn't have a IOMMU attached
to it, supporting scatter gather is more odd.
It would be more likely that IOMMU would be disabled for the performance
reason.
>
> Best regards,
> Tomasz
>
>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>> needs.
>>>>>
>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>> ---
>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>> include/media/videobuf2-core.h | 1 -
>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> index 86e1e926fa45..899783f67580 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> @@ -31,6 +31,8 @@
>>>>>
>>>>> #include <trace/events/vb2.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> static int debug;
>>>>> module_param(debug, int, 0644);
>>>>>
>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>> @@ -28,6 +28,7 @@
>>>>>
>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>> +#define VB2_MAX_FRAME 32
>>>>>
>>>>> struct vdec_fs_info {
>>>>> char name[8];
>>>>> diff --git
>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>> @@ -16,6 +16,8 @@
>>>>> #include "../vdec_drv_if.h"
>>>>> #include "../vdec_vpu_if.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /* reset_frame_context defined in VP9 spec */
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>> @@ -10,6 +10,8 @@
>>>>>
>>>>> #include "hfi_helper.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>> @@ -15,6 +15,8 @@
>>>>> #include <media/v4l2-vp9.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>
>>>>> #define MB_DIM 16
>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>> @@ -10,6 +10,8 @@
>>>>> #include "ipu3.h"
>>>>> #include "ipu3-dmamap.h"
>>>>>
>>>>> +#define VB2_MAX_FRAME 32
>>>>> +
>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>
>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>> diff --git a/include/media/videobuf2-core.h
>>>>> b/include/media/videobuf2-core.h
>>>>> index 77921cf894ef..080b783d608d 100644
>>>>> --- a/include/media/videobuf2-core.h
>>>>> +++ b/include/media/videobuf2-core.h
>>>>> @@ -20,7 +20,6 @@
>>>>> #include <media/media-request.h>
>>>>> #include <media/frame_vector.h>
>>>>>
>>>>> -#define VB2_MAX_FRAME (32)
>>>>> #define VB2_MAX_PLANES (8)
>>>>>
>>>>> /**
>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>> b/include/media/videobuf2-v4l2.h
>>>>> index 5a845887850b..88a7a565170e 100644
>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>> @@ -15,10 +15,6 @@
>>>>> #include <linux/videodev2.h>
>>>>> #include <media/videobuf2-core.h>
>>>>>
>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>> -#endif
>>>>> -
>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>> #endif
>>>>> --
>>>>> 2.39.2
>>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-07-17 7:44 ` Hsia-Jun Li
(?)
@ 2023-07-28 6:46 ` Tomasz Figa
-1 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-28 6:46 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
> On 7/12/23 18:48, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
> >> On 7/3/23 16:09, Benjamin Gaignard wrote:
> >>> CAUTION: Email originated externally, do not click links or open
> >>> attachments unless you recognize the sender and know the content is
> >>> safe.
> >>>
> >>>
> >>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> >>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>> CAUTION: Email originated externally, do not click links or open
> >>>>> attachments unless you recognize the sender and know the content is
> >>>>> safe.
> >>>>>
> >>>>>
> >>>>> After changing bufs arrays to a dynamic allocated array
> >>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> >>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> >>>> more reasonable.
> >>>>
> >>>> It would be hard to iterate the whole array, it would go worse with a
> >>>> filter. Such iterate may need to go twice because you mix
> >>>> post-processing buffer and decoding buffer(with MV) in the same array.
> >>> Here I don't want to change drivers behavior so I keep the same value.
> >>> If it happens that they need more buffers, like for dynamic resolution
> >>> change
> >>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >>>
> >> I just don't like the idea that using a variant length array here.
> >>
> > "I don't like" is not an argument. We had a number of arguments for
> > using a generic helper (originally idr, but later decided to go with
> > XArray, because the former is now deprecated) that we pointed out in
> > our review comments for previous revisions. It wasn't really about the
> > size being variable, but rather avoiding open coding things in vb2 and
> > duplicating what's already implemented in generic code.
>
> I just want to say I don't think we need a variable length array to
> store the buffer here.
>
> And the below is the reason that such a case could be avoided in the
> first place.
>
> >
> >> And I could explain why you won't need so many buffers for the performance
> >> of decoding.
> >>
> >> VP9 could support 10 reference frames in dpb.
> >>
> >> Even for those frequent resolution changing test set, it would only happen
> >> to two resolutions,
> >>
> >> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> >> also leaves enough slots for re-order latency.
> >>
> >> If your case had more two resolutions, likes low->medium->high.
> >>
> >> I would suggest just skip the medium resolutions, just allocate the lower
> >> one first for fast playback then the highest for all the possible
> >>
> >> medium cases. Reallocation happens frequently would only cause memory
> >> fragment, nothing benefits your performance.
> >>
> > We have mechanisms in the kernel to deal with memory fragmentation
> > (migration/compaction) and it would still only matters for the
> > pathologic cases of hardware that require physically contiguous memory.
> > Modern hardware with proper DMA capabilities can either scatter-gather
> > or are equipped with an IOMMU, so the allocation always happens in page
> > granularity and fragmentation is avoided.
>
> Unfortunately, there are more devices that didn't have a IOMMU attached
> to it, supporting scatter gather is more odd.
>
> It would be more likely that IOMMU would be disabled for the performance
> reason.
These days IOMMU is totally mandatory if you want to think about
having any level of security in your system. Sure, there could be some
systems that are completely isolated from any external environment,
like some offline industry automation machines, but then arguably
their running conditions would also be quite static and require very
little memory re-allocation.
I also don't buy the performance reason. CPUs have been behind MMUs
for ages and nobody is running them with paging disabled for
performance reasons. Similarly, most of the modern consumer systems
(mobile phones, PCs) run with IOMMUs enabled for pretty much anything
because of the security reason and they don't seem to be having any
performance issues. In fact, it can improve the performance, because
memory allocation is much easier and without contiguous careouts (as
we used to have long ago on Android devices) the extra memory can be
used for buffers and caches to improve system performance.
Best regards,
Tomasz
>
> >
> > Best regards,
> > Tomasz
> >
> >>>>> Remove it from the core definitions but keep it for drivers internal
> >>>>> needs.
> >>>>>
> >>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>> ---
> >>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> >>>>> drivers/media/platform/amphion/vdec.c | 1 +
> >>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> >>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> >>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> >>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> >>>>> include/media/videobuf2-core.h | 1 -
> >>>>> include/media/videobuf2-v4l2.h | 4 ----
> >>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
> >>>>>
> >>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> index 86e1e926fa45..899783f67580 100644
> >>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> @@ -31,6 +31,8 @@
> >>>>>
> >>>>> #include <trace/events/vb2.h>
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> static int debug;
> >>>>> module_param(debug, int, 0644);
> >>>>>
> >>>>> diff --git a/drivers/media/platform/amphion/vdec.c
> >>>>> b/drivers/media/platform/amphion/vdec.c
> >>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
> >>>>> --- a/drivers/media/platform/amphion/vdec.c
> >>>>> +++ b/drivers/media/platform/amphion/vdec.c
> >>>>> @@ -28,6 +28,7 @@
> >>>>>
> >>>>> #define VDEC_MIN_BUFFER_CAP 8
> >>>>> #define VDEC_MIN_BUFFER_OUT 8
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>>
> >>>>> struct vdec_fs_info {
> >>>>> char name[8];
> >>>>> diff --git
> >>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
> >>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> @@ -16,6 +16,8 @@
> >>>>> #include "../vdec_drv_if.h"
> >>>>> #include "../vdec_vpu_if.h"
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> /* reset_frame_context defined in VP9 spec */
> >>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> >>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> >>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
> >>>>> b/drivers/media/platform/qcom/venus/hfi.h
> >>>>> index f25d412d6553..bd5ca5a8b945 100644
> >>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
> >>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
> >>>>> @@ -10,6 +10,8 @@
> >>>>>
> >>>>> #include "hfi_helper.h"
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> #define VIDC_SESSION_TYPE_VPE 0
> >>>>> #define VIDC_SESSION_TYPE_ENC 1
> >>>>> #define VIDC_SESSION_TYPE_DEC 2
> >>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> index e83f0c523a30..9e8faf7ba6fb 100644
> >>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> @@ -15,6 +15,8 @@
> >>>>> #include <media/v4l2-vp9.h>
> >>>>> #include <media/videobuf2-core.h>
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> #define DEC_8190_ALIGN_MASK 0x07U
> >>>>>
> >>>>> #define MB_DIM 16
> >>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> index e530767e80a5..6627b5c2d4d6 100644
> >>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> @@ -10,6 +10,8 @@
> >>>>> #include "ipu3.h"
> >>>>> #include "ipu3-dmamap.h"
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> /******************** v4l2_subdev_ops ********************/
> >>>>>
> >>>>> #define IPU3_RUNNING_MODE_VIDEO 0
> >>>>> diff --git a/include/media/videobuf2-core.h
> >>>>> b/include/media/videobuf2-core.h
> >>>>> index 77921cf894ef..080b783d608d 100644
> >>>>> --- a/include/media/videobuf2-core.h
> >>>>> +++ b/include/media/videobuf2-core.h
> >>>>> @@ -20,7 +20,6 @@
> >>>>> #include <media/media-request.h>
> >>>>> #include <media/frame_vector.h>
> >>>>>
> >>>>> -#define VB2_MAX_FRAME (32)
> >>>>> #define VB2_MAX_PLANES (8)
> >>>>>
> >>>>> /**
> >>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>> b/include/media/videobuf2-v4l2.h
> >>>>> index 5a845887850b..88a7a565170e 100644
> >>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>> @@ -15,10 +15,6 @@
> >>>>> #include <linux/videodev2.h>
> >>>>> #include <media/videobuf2-core.h>
> >>>>>
> >>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>> -#endif
> >>>>> -
> >>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>> #endif
> >>>>> --
> >>>>> 2.39.2
> >>>>>
> >> --
> >> Hsia-Jun(Randy) Li
> >>
> --
> Hsia-Jun(Randy) Li
>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-28 6:46 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-28 6:46 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
> On 7/12/23 18:48, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
> >> On 7/3/23 16:09, Benjamin Gaignard wrote:
> >>> CAUTION: Email originated externally, do not click links or open
> >>> attachments unless you recognize the sender and know the content is
> >>> safe.
> >>>
> >>>
> >>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> >>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>> CAUTION: Email originated externally, do not click links or open
> >>>>> attachments unless you recognize the sender and know the content is
> >>>>> safe.
> >>>>>
> >>>>>
> >>>>> After changing bufs arrays to a dynamic allocated array
> >>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> >>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> >>>> more reasonable.
> >>>>
> >>>> It would be hard to iterate the whole array, it would go worse with a
> >>>> filter. Such iterate may need to go twice because you mix
> >>>> post-processing buffer and decoding buffer(with MV) in the same array.
> >>> Here I don't want to change drivers behavior so I keep the same value.
> >>> If it happens that they need more buffers, like for dynamic resolution
> >>> change
> >>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >>>
> >> I just don't like the idea that using a variant length array here.
> >>
> > "I don't like" is not an argument. We had a number of arguments for
> > using a generic helper (originally idr, but later decided to go with
> > XArray, because the former is now deprecated) that we pointed out in
> > our review comments for previous revisions. It wasn't really about the
> > size being variable, but rather avoiding open coding things in vb2 and
> > duplicating what's already implemented in generic code.
>
> I just want to say I don't think we need a variable length array to
> store the buffer here.
>
> And the below is the reason that such a case could be avoided in the
> first place.
>
> >
> >> And I could explain why you won't need so many buffers for the performance
> >> of decoding.
> >>
> >> VP9 could support 10 reference frames in dpb.
> >>
> >> Even for those frequent resolution changing test set, it would only happen
> >> to two resolutions,
> >>
> >> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> >> also leaves enough slots for re-order latency.
> >>
> >> If your case had more two resolutions, likes low->medium->high.
> >>
> >> I would suggest just skip the medium resolutions, just allocate the lower
> >> one first for fast playback then the highest for all the possible
> >>
> >> medium cases. Reallocation happens frequently would only cause memory
> >> fragment, nothing benefits your performance.
> >>
> > We have mechanisms in the kernel to deal with memory fragmentation
> > (migration/compaction) and it would still only matters for the
> > pathologic cases of hardware that require physically contiguous memory.
> > Modern hardware with proper DMA capabilities can either scatter-gather
> > or are equipped with an IOMMU, so the allocation always happens in page
> > granularity and fragmentation is avoided.
>
> Unfortunately, there are more devices that didn't have a IOMMU attached
> to it, supporting scatter gather is more odd.
>
> It would be more likely that IOMMU would be disabled for the performance
> reason.
These days IOMMU is totally mandatory if you want to think about
having any level of security in your system. Sure, there could be some
systems that are completely isolated from any external environment,
like some offline industry automation machines, but then arguably
their running conditions would also be quite static and require very
little memory re-allocation.
I also don't buy the performance reason. CPUs have been behind MMUs
for ages and nobody is running them with paging disabled for
performance reasons. Similarly, most of the modern consumer systems
(mobile phones, PCs) run with IOMMUs enabled for pretty much anything
because of the security reason and they don't seem to be having any
performance issues. In fact, it can improve the performance, because
memory allocation is much easier and without contiguous careouts (as
we used to have long ago on Android devices) the extra memory can be
used for buffers and caches to improve system performance.
Best regards,
Tomasz
>
> >
> > Best regards,
> > Tomasz
> >
> >>>>> Remove it from the core definitions but keep it for drivers internal
> >>>>> needs.
> >>>>>
> >>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>> ---
> >>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> >>>>> drivers/media/platform/amphion/vdec.c | 1 +
> >>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> >>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> >>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> >>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> >>>>> include/media/videobuf2-core.h | 1 -
> >>>>> include/media/videobuf2-v4l2.h | 4 ----
> >>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
> >>>>>
> >>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> index 86e1e926fa45..899783f67580 100644
> >>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> @@ -31,6 +31,8 @@
> >>>>>
> >>>>> #include <trace/events/vb2.h>
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> static int debug;
> >>>>> module_param(debug, int, 0644);
> >>>>>
> >>>>> diff --git a/drivers/media/platform/amphion/vdec.c
> >>>>> b/drivers/media/platform/amphion/vdec.c
> >>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
> >>>>> --- a/drivers/media/platform/amphion/vdec.c
> >>>>> +++ b/drivers/media/platform/amphion/vdec.c
> >>>>> @@ -28,6 +28,7 @@
> >>>>>
> >>>>> #define VDEC_MIN_BUFFER_CAP 8
> >>>>> #define VDEC_MIN_BUFFER_OUT 8
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>>
> >>>>> struct vdec_fs_info {
> >>>>> char name[8];
> >>>>> diff --git
> >>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
> >>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> @@ -16,6 +16,8 @@
> >>>>> #include "../vdec_drv_if.h"
> >>>>> #include "../vdec_vpu_if.h"
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> /* reset_frame_context defined in VP9 spec */
> >>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> >>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> >>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
> >>>>> b/drivers/media/platform/qcom/venus/hfi.h
> >>>>> index f25d412d6553..bd5ca5a8b945 100644
> >>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
> >>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
> >>>>> @@ -10,6 +10,8 @@
> >>>>>
> >>>>> #include "hfi_helper.h"
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> #define VIDC_SESSION_TYPE_VPE 0
> >>>>> #define VIDC_SESSION_TYPE_ENC 1
> >>>>> #define VIDC_SESSION_TYPE_DEC 2
> >>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> index e83f0c523a30..9e8faf7ba6fb 100644
> >>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> @@ -15,6 +15,8 @@
> >>>>> #include <media/v4l2-vp9.h>
> >>>>> #include <media/videobuf2-core.h>
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> #define DEC_8190_ALIGN_MASK 0x07U
> >>>>>
> >>>>> #define MB_DIM 16
> >>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> index e530767e80a5..6627b5c2d4d6 100644
> >>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> @@ -10,6 +10,8 @@
> >>>>> #include "ipu3.h"
> >>>>> #include "ipu3-dmamap.h"
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> /******************** v4l2_subdev_ops ********************/
> >>>>>
> >>>>> #define IPU3_RUNNING_MODE_VIDEO 0
> >>>>> diff --git a/include/media/videobuf2-core.h
> >>>>> b/include/media/videobuf2-core.h
> >>>>> index 77921cf894ef..080b783d608d 100644
> >>>>> --- a/include/media/videobuf2-core.h
> >>>>> +++ b/include/media/videobuf2-core.h
> >>>>> @@ -20,7 +20,6 @@
> >>>>> #include <media/media-request.h>
> >>>>> #include <media/frame_vector.h>
> >>>>>
> >>>>> -#define VB2_MAX_FRAME (32)
> >>>>> #define VB2_MAX_PLANES (8)
> >>>>>
> >>>>> /**
> >>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>> b/include/media/videobuf2-v4l2.h
> >>>>> index 5a845887850b..88a7a565170e 100644
> >>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>> @@ -15,10 +15,6 @@
> >>>>> #include <linux/videodev2.h>
> >>>>> #include <media/videobuf2-core.h>
> >>>>>
> >>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>> -#endif
> >>>>> -
> >>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>> #endif
> >>>>> --
> >>>>> 2.39.2
> >>>>>
> >> --
> >> Hsia-Jun(Randy) Li
> >>
> --
> Hsia-Jun(Randy) Li
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-28 6:46 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-28 6:46 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
> On 7/12/23 18:48, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
> >> On 7/3/23 16:09, Benjamin Gaignard wrote:
> >>> CAUTION: Email originated externally, do not click links or open
> >>> attachments unless you recognize the sender and know the content is
> >>> safe.
> >>>
> >>>
> >>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> >>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>> CAUTION: Email originated externally, do not click links or open
> >>>>> attachments unless you recognize the sender and know the content is
> >>>>> safe.
> >>>>>
> >>>>>
> >>>>> After changing bufs arrays to a dynamic allocated array
> >>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> >>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> >>>> more reasonable.
> >>>>
> >>>> It would be hard to iterate the whole array, it would go worse with a
> >>>> filter. Such iterate may need to go twice because you mix
> >>>> post-processing buffer and decoding buffer(with MV) in the same array.
> >>> Here I don't want to change drivers behavior so I keep the same value.
> >>> If it happens that they need more buffers, like for dynamic resolution
> >>> change
> >>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >>>
> >> I just don't like the idea that using a variant length array here.
> >>
> > "I don't like" is not an argument. We had a number of arguments for
> > using a generic helper (originally idr, but later decided to go with
> > XArray, because the former is now deprecated) that we pointed out in
> > our review comments for previous revisions. It wasn't really about the
> > size being variable, but rather avoiding open coding things in vb2 and
> > duplicating what's already implemented in generic code.
>
> I just want to say I don't think we need a variable length array to
> store the buffer here.
>
> And the below is the reason that such a case could be avoided in the
> first place.
>
> >
> >> And I could explain why you won't need so many buffers for the performance
> >> of decoding.
> >>
> >> VP9 could support 10 reference frames in dpb.
> >>
> >> Even for those frequent resolution changing test set, it would only happen
> >> to two resolutions,
> >>
> >> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> >> also leaves enough slots for re-order latency.
> >>
> >> If your case had more two resolutions, likes low->medium->high.
> >>
> >> I would suggest just skip the medium resolutions, just allocate the lower
> >> one first for fast playback then the highest for all the possible
> >>
> >> medium cases. Reallocation happens frequently would only cause memory
> >> fragment, nothing benefits your performance.
> >>
> > We have mechanisms in the kernel to deal with memory fragmentation
> > (migration/compaction) and it would still only matters for the
> > pathologic cases of hardware that require physically contiguous memory.
> > Modern hardware with proper DMA capabilities can either scatter-gather
> > or are equipped with an IOMMU, so the allocation always happens in page
> > granularity and fragmentation is avoided.
>
> Unfortunately, there are more devices that didn't have a IOMMU attached
> to it, supporting scatter gather is more odd.
>
> It would be more likely that IOMMU would be disabled for the performance
> reason.
These days IOMMU is totally mandatory if you want to think about
having any level of security in your system. Sure, there could be some
systems that are completely isolated from any external environment,
like some offline industry automation machines, but then arguably
their running conditions would also be quite static and require very
little memory re-allocation.
I also don't buy the performance reason. CPUs have been behind MMUs
for ages and nobody is running them with paging disabled for
performance reasons. Similarly, most of the modern consumer systems
(mobile phones, PCs) run with IOMMUs enabled for pretty much anything
because of the security reason and they don't seem to be having any
performance issues. In fact, it can improve the performance, because
memory allocation is much easier and without contiguous careouts (as
we used to have long ago on Android devices) the extra memory can be
used for buffers and caches to improve system performance.
Best regards,
Tomasz
>
> >
> > Best regards,
> > Tomasz
> >
> >>>>> Remove it from the core definitions but keep it for drivers internal
> >>>>> needs.
> >>>>>
> >>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>> ---
> >>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> >>>>> drivers/media/platform/amphion/vdec.c | 1 +
> >>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> >>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> >>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> >>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> >>>>> include/media/videobuf2-core.h | 1 -
> >>>>> include/media/videobuf2-v4l2.h | 4 ----
> >>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
> >>>>>
> >>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> index 86e1e926fa45..899783f67580 100644
> >>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>> @@ -31,6 +31,8 @@
> >>>>>
> >>>>> #include <trace/events/vb2.h>
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> static int debug;
> >>>>> module_param(debug, int, 0644);
> >>>>>
> >>>>> diff --git a/drivers/media/platform/amphion/vdec.c
> >>>>> b/drivers/media/platform/amphion/vdec.c
> >>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
> >>>>> --- a/drivers/media/platform/amphion/vdec.c
> >>>>> +++ b/drivers/media/platform/amphion/vdec.c
> >>>>> @@ -28,6 +28,7 @@
> >>>>>
> >>>>> #define VDEC_MIN_BUFFER_CAP 8
> >>>>> #define VDEC_MIN_BUFFER_OUT 8
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>>
> >>>>> struct vdec_fs_info {
> >>>>> char name[8];
> >>>>> diff --git
> >>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
> >>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>> @@ -16,6 +16,8 @@
> >>>>> #include "../vdec_drv_if.h"
> >>>>> #include "../vdec_vpu_if.h"
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> /* reset_frame_context defined in VP9 spec */
> >>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> >>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> >>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
> >>>>> b/drivers/media/platform/qcom/venus/hfi.h
> >>>>> index f25d412d6553..bd5ca5a8b945 100644
> >>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
> >>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
> >>>>> @@ -10,6 +10,8 @@
> >>>>>
> >>>>> #include "hfi_helper.h"
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> #define VIDC_SESSION_TYPE_VPE 0
> >>>>> #define VIDC_SESSION_TYPE_ENC 1
> >>>>> #define VIDC_SESSION_TYPE_DEC 2
> >>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> index e83f0c523a30..9e8faf7ba6fb 100644
> >>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>> @@ -15,6 +15,8 @@
> >>>>> #include <media/v4l2-vp9.h>
> >>>>> #include <media/videobuf2-core.h>
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> #define DEC_8190_ALIGN_MASK 0x07U
> >>>>>
> >>>>> #define MB_DIM 16
> >>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> index e530767e80a5..6627b5c2d4d6 100644
> >>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>> @@ -10,6 +10,8 @@
> >>>>> #include "ipu3.h"
> >>>>> #include "ipu3-dmamap.h"
> >>>>>
> >>>>> +#define VB2_MAX_FRAME 32
> >>>>> +
> >>>>> /******************** v4l2_subdev_ops ********************/
> >>>>>
> >>>>> #define IPU3_RUNNING_MODE_VIDEO 0
> >>>>> diff --git a/include/media/videobuf2-core.h
> >>>>> b/include/media/videobuf2-core.h
> >>>>> index 77921cf894ef..080b783d608d 100644
> >>>>> --- a/include/media/videobuf2-core.h
> >>>>> +++ b/include/media/videobuf2-core.h
> >>>>> @@ -20,7 +20,6 @@
> >>>>> #include <media/media-request.h>
> >>>>> #include <media/frame_vector.h>
> >>>>>
> >>>>> -#define VB2_MAX_FRAME (32)
> >>>>> #define VB2_MAX_PLANES (8)
> >>>>>
> >>>>> /**
> >>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>> b/include/media/videobuf2-v4l2.h
> >>>>> index 5a845887850b..88a7a565170e 100644
> >>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>> @@ -15,10 +15,6 @@
> >>>>> #include <linux/videodev2.h>
> >>>>> #include <media/videobuf2-core.h>
> >>>>>
> >>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>> -#endif
> >>>>> -
> >>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>> #endif
> >>>>> --
> >>>>> 2.39.2
> >>>>>
> >> --
> >> Hsia-Jun(Randy) Li
> >>
> --
> Hsia-Jun(Randy) Li
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-07-28 6:46 ` Tomasz Figa
(?)
@ 2023-07-28 6:55 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-28 6:55 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On 7/28/23 14:46, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>> On 7/12/23 18:48, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> After changing bufs arrays to a dynamic allocated array
>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>>>> more reasonable.
>>>>>>
>>>>>> It would be hard to iterate the whole array, it would go worse with a
>>>>>> filter. Such iterate may need to go twice because you mix
>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>>> Here I don't want to change drivers behavior so I keep the same value.
>>>>> If it happens that they need more buffers, like for dynamic resolution
>>>>> change
>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>>>
>>>> I just don't like the idea that using a variant length array here.
>>>>
>>> "I don't like" is not an argument. We had a number of arguments for
>>> using a generic helper (originally idr, but later decided to go with
>>> XArray, because the former is now deprecated) that we pointed out in
>>> our review comments for previous revisions. It wasn't really about the
>>> size being variable, but rather avoiding open coding things in vb2 and
>>> duplicating what's already implemented in generic code.
>>
>> I just want to say I don't think we need a variable length array to
>> store the buffer here.
>>
>> And the below is the reason that such a case could be avoided in the
>> first place.
>>
>>>
>>>> And I could explain why you won't need so many buffers for the performance
>>>> of decoding.
>>>>
>>>> VP9 could support 10 reference frames in dpb.
>>>>
>>>> Even for those frequent resolution changing test set, it would only happen
>>>> to two resolutions,
>>>>
>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>>>> also leaves enough slots for re-order latency.
>>>>
>>>> If your case had more two resolutions, likes low->medium->high.
>>>>
>>>> I would suggest just skip the medium resolutions, just allocate the lower
>>>> one first for fast playback then the highest for all the possible
>>>>
>>>> medium cases. Reallocation happens frequently would only cause memory
>>>> fragment, nothing benefits your performance.
>>>>
>>> We have mechanisms in the kernel to deal with memory fragmentation
>>> (migration/compaction) and it would still only matters for the
>>> pathologic cases of hardware that require physically contiguous memory.
>>> Modern hardware with proper DMA capabilities can either scatter-gather
>>> or are equipped with an IOMMU, so the allocation always happens in page
>>> granularity and fragmentation is avoided.
>>
>> Unfortunately, there are more devices that didn't have a IOMMU attached
>> to it, supporting scatter gather is more odd.
>>
>> It would be more likely that IOMMU would be disabled for the performance
>> reason.
>
> These days IOMMU is totally mandatory if you want to think about
> having any level of security in your system. Sure, there could be some
> systems that are completely isolated from any external environment,
> like some offline industry automation machines, but then arguably
> their running conditions would also be quite static and require very
> little memory re-allocation.
Vendor just decided not to included such hardware.
That is why From ION to DMA-heap, people like to allocate from a cavout
out memory.
>
> I also don't buy the performance reason. CPUs have been behind MMUs
> for ages and nobody is running them with paging disabled for
> performance reasons. Similarly, most of the modern consumer systems
Page lookup would increase the delay. Besides a few upstream devices
prove them only use a level 1 page table without TBL.
> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
> because of the security reason and they don't seem to be having any
If the page is secure, you can't operate it in a insecure IOMMU or MMU.
The most security way here, we should use a dedicated memory(or a zone
in unified memory).
I believe there are more users in mobile for DMA-heap than kernel's dma
allocation API.
> performance issues. In fact, it can improve the performance, because
> memory allocation is much easier and without contiguous careouts (as
> we used to have long ago on Android devices) the extra memory can be
> used for buffers and caches to improve system performance.
>
> Best regards,
> Tomasz
>
>>
>>>
>>> Best regards,
>>> Tomasz
>>>
>>>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>>>> needs.
>>>>>>>
>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>> ---
>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>>>> include/media/videobuf2-core.h | 1 -
>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> index 86e1e926fa45..899783f67580 100644
>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> @@ -31,6 +31,8 @@
>>>>>>>
>>>>>>> #include <trace/events/vb2.h>
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> static int debug;
>>>>>>> module_param(debug, int, 0644);
>>>>>>>
>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>>>> @@ -28,6 +28,7 @@
>>>>>>>
>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>
>>>>>>> struct vdec_fs_info {
>>>>>>> char name[8];
>>>>>>> diff --git
>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> @@ -16,6 +16,8 @@
>>>>>>> #include "../vdec_drv_if.h"
>>>>>>> #include "../vdec_vpu_if.h"
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> /* reset_frame_context defined in VP9 spec */
>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>
>>>>>>> #include "hfi_helper.h"
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> @@ -15,6 +15,8 @@
>>>>>>> #include <media/v4l2-vp9.h>
>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>>>
>>>>>>> #define MB_DIM 16
>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> @@ -10,6 +10,8 @@
>>>>>>> #include "ipu3.h"
>>>>>>> #include "ipu3-dmamap.h"
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>>>
>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>> b/include/media/videobuf2-core.h
>>>>>>> index 77921cf894ef..080b783d608d 100644
>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>> @@ -20,7 +20,6 @@
>>>>>>> #include <media/media-request.h>
>>>>>>> #include <media/frame_vector.h>
>>>>>>>
>>>>>>> -#define VB2_MAX_FRAME (32)
>>>>>>> #define VB2_MAX_PLANES (8)
>>>>>>>
>>>>>>> /**
>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>> index 5a845887850b..88a7a565170e 100644
>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>> @@ -15,10 +15,6 @@
>>>>>>> #include <linux/videodev2.h>
>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>
>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>> -#endif
>>>>>>> -
>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>> #endif
>>>>>>> --
>>>>>>> 2.39.2
>>>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-28 6:55 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-28 6:55 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On 7/28/23 14:46, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>> On 7/12/23 18:48, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> After changing bufs arrays to a dynamic allocated array
>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>>>> more reasonable.
>>>>>>
>>>>>> It would be hard to iterate the whole array, it would go worse with a
>>>>>> filter. Such iterate may need to go twice because you mix
>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>>> Here I don't want to change drivers behavior so I keep the same value.
>>>>> If it happens that they need more buffers, like for dynamic resolution
>>>>> change
>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>>>
>>>> I just don't like the idea that using a variant length array here.
>>>>
>>> "I don't like" is not an argument. We had a number of arguments for
>>> using a generic helper (originally idr, but later decided to go with
>>> XArray, because the former is now deprecated) that we pointed out in
>>> our review comments for previous revisions. It wasn't really about the
>>> size being variable, but rather avoiding open coding things in vb2 and
>>> duplicating what's already implemented in generic code.
>>
>> I just want to say I don't think we need a variable length array to
>> store the buffer here.
>>
>> And the below is the reason that such a case could be avoided in the
>> first place.
>>
>>>
>>>> And I could explain why you won't need so many buffers for the performance
>>>> of decoding.
>>>>
>>>> VP9 could support 10 reference frames in dpb.
>>>>
>>>> Even for those frequent resolution changing test set, it would only happen
>>>> to two resolutions,
>>>>
>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>>>> also leaves enough slots for re-order latency.
>>>>
>>>> If your case had more two resolutions, likes low->medium->high.
>>>>
>>>> I would suggest just skip the medium resolutions, just allocate the lower
>>>> one first for fast playback then the highest for all the possible
>>>>
>>>> medium cases. Reallocation happens frequently would only cause memory
>>>> fragment, nothing benefits your performance.
>>>>
>>> We have mechanisms in the kernel to deal with memory fragmentation
>>> (migration/compaction) and it would still only matters for the
>>> pathologic cases of hardware that require physically contiguous memory.
>>> Modern hardware with proper DMA capabilities can either scatter-gather
>>> or are equipped with an IOMMU, so the allocation always happens in page
>>> granularity and fragmentation is avoided.
>>
>> Unfortunately, there are more devices that didn't have a IOMMU attached
>> to it, supporting scatter gather is more odd.
>>
>> It would be more likely that IOMMU would be disabled for the performance
>> reason.
>
> These days IOMMU is totally mandatory if you want to think about
> having any level of security in your system. Sure, there could be some
> systems that are completely isolated from any external environment,
> like some offline industry automation machines, but then arguably
> their running conditions would also be quite static and require very
> little memory re-allocation.
Vendor just decided not to included such hardware.
That is why From ION to DMA-heap, people like to allocate from a cavout
out memory.
>
> I also don't buy the performance reason. CPUs have been behind MMUs
> for ages and nobody is running them with paging disabled for
> performance reasons. Similarly, most of the modern consumer systems
Page lookup would increase the delay. Besides a few upstream devices
prove them only use a level 1 page table without TBL.
> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
> because of the security reason and they don't seem to be having any
If the page is secure, you can't operate it in a insecure IOMMU or MMU.
The most security way here, we should use a dedicated memory(or a zone
in unified memory).
I believe there are more users in mobile for DMA-heap than kernel's dma
allocation API.
> performance issues. In fact, it can improve the performance, because
> memory allocation is much easier and without contiguous careouts (as
> we used to have long ago on Android devices) the extra memory can be
> used for buffers and caches to improve system performance.
>
> Best regards,
> Tomasz
>
>>
>>>
>>> Best regards,
>>> Tomasz
>>>
>>>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>>>> needs.
>>>>>>>
>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>> ---
>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>>>> include/media/videobuf2-core.h | 1 -
>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> index 86e1e926fa45..899783f67580 100644
>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> @@ -31,6 +31,8 @@
>>>>>>>
>>>>>>> #include <trace/events/vb2.h>
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> static int debug;
>>>>>>> module_param(debug, int, 0644);
>>>>>>>
>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>>>> @@ -28,6 +28,7 @@
>>>>>>>
>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>
>>>>>>> struct vdec_fs_info {
>>>>>>> char name[8];
>>>>>>> diff --git
>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> @@ -16,6 +16,8 @@
>>>>>>> #include "../vdec_drv_if.h"
>>>>>>> #include "../vdec_vpu_if.h"
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> /* reset_frame_context defined in VP9 spec */
>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>
>>>>>>> #include "hfi_helper.h"
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> @@ -15,6 +15,8 @@
>>>>>>> #include <media/v4l2-vp9.h>
>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>>>
>>>>>>> #define MB_DIM 16
>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> @@ -10,6 +10,8 @@
>>>>>>> #include "ipu3.h"
>>>>>>> #include "ipu3-dmamap.h"
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>>>
>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>> b/include/media/videobuf2-core.h
>>>>>>> index 77921cf894ef..080b783d608d 100644
>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>> @@ -20,7 +20,6 @@
>>>>>>> #include <media/media-request.h>
>>>>>>> #include <media/frame_vector.h>
>>>>>>>
>>>>>>> -#define VB2_MAX_FRAME (32)
>>>>>>> #define VB2_MAX_PLANES (8)
>>>>>>>
>>>>>>> /**
>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>> index 5a845887850b..88a7a565170e 100644
>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>> @@ -15,10 +15,6 @@
>>>>>>> #include <linux/videodev2.h>
>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>
>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>> -#endif
>>>>>>> -
>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>> #endif
>>>>>>> --
>>>>>>> 2.39.2
>>>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-28 6:55 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-28 6:55 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On 7/28/23 14:46, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>> On 7/12/23 18:48, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> After changing bufs arrays to a dynamic allocated array
>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>>>> more reasonable.
>>>>>>
>>>>>> It would be hard to iterate the whole array, it would go worse with a
>>>>>> filter. Such iterate may need to go twice because you mix
>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>>> Here I don't want to change drivers behavior so I keep the same value.
>>>>> If it happens that they need more buffers, like for dynamic resolution
>>>>> change
>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>>>
>>>> I just don't like the idea that using a variant length array here.
>>>>
>>> "I don't like" is not an argument. We had a number of arguments for
>>> using a generic helper (originally idr, but later decided to go with
>>> XArray, because the former is now deprecated) that we pointed out in
>>> our review comments for previous revisions. It wasn't really about the
>>> size being variable, but rather avoiding open coding things in vb2 and
>>> duplicating what's already implemented in generic code.
>>
>> I just want to say I don't think we need a variable length array to
>> store the buffer here.
>>
>> And the below is the reason that such a case could be avoided in the
>> first place.
>>
>>>
>>>> And I could explain why you won't need so many buffers for the performance
>>>> of decoding.
>>>>
>>>> VP9 could support 10 reference frames in dpb.
>>>>
>>>> Even for those frequent resolution changing test set, it would only happen
>>>> to two resolutions,
>>>>
>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>>>> also leaves enough slots for re-order latency.
>>>>
>>>> If your case had more two resolutions, likes low->medium->high.
>>>>
>>>> I would suggest just skip the medium resolutions, just allocate the lower
>>>> one first for fast playback then the highest for all the possible
>>>>
>>>> medium cases. Reallocation happens frequently would only cause memory
>>>> fragment, nothing benefits your performance.
>>>>
>>> We have mechanisms in the kernel to deal with memory fragmentation
>>> (migration/compaction) and it would still only matters for the
>>> pathologic cases of hardware that require physically contiguous memory.
>>> Modern hardware with proper DMA capabilities can either scatter-gather
>>> or are equipped with an IOMMU, so the allocation always happens in page
>>> granularity and fragmentation is avoided.
>>
>> Unfortunately, there are more devices that didn't have a IOMMU attached
>> to it, supporting scatter gather is more odd.
>>
>> It would be more likely that IOMMU would be disabled for the performance
>> reason.
>
> These days IOMMU is totally mandatory if you want to think about
> having any level of security in your system. Sure, there could be some
> systems that are completely isolated from any external environment,
> like some offline industry automation machines, but then arguably
> their running conditions would also be quite static and require very
> little memory re-allocation.
Vendor just decided not to included such hardware.
That is why From ION to DMA-heap, people like to allocate from a cavout
out memory.
>
> I also don't buy the performance reason. CPUs have been behind MMUs
> for ages and nobody is running them with paging disabled for
> performance reasons. Similarly, most of the modern consumer systems
Page lookup would increase the delay. Besides a few upstream devices
prove them only use a level 1 page table without TBL.
> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
> because of the security reason and they don't seem to be having any
If the page is secure, you can't operate it in a insecure IOMMU or MMU.
The most security way here, we should use a dedicated memory(or a zone
in unified memory).
I believe there are more users in mobile for DMA-heap than kernel's dma
allocation API.
> performance issues. In fact, it can improve the performance, because
> memory allocation is much easier and without contiguous careouts (as
> we used to have long ago on Android devices) the extra memory can be
> used for buffers and caches to improve system performance.
>
> Best regards,
> Tomasz
>
>>
>>>
>>> Best regards,
>>> Tomasz
>>>
>>>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>>>> needs.
>>>>>>>
>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>> ---
>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>>>> include/media/videobuf2-core.h | 1 -
>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> index 86e1e926fa45..899783f67580 100644
>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> @@ -31,6 +31,8 @@
>>>>>>>
>>>>>>> #include <trace/events/vb2.h>
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> static int debug;
>>>>>>> module_param(debug, int, 0644);
>>>>>>>
>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>>>> @@ -28,6 +28,7 @@
>>>>>>>
>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>
>>>>>>> struct vdec_fs_info {
>>>>>>> char name[8];
>>>>>>> diff --git
>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>> @@ -16,6 +16,8 @@
>>>>>>> #include "../vdec_drv_if.h"
>>>>>>> #include "../vdec_vpu_if.h"
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> /* reset_frame_context defined in VP9 spec */
>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>
>>>>>>> #include "hfi_helper.h"
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>> @@ -15,6 +15,8 @@
>>>>>>> #include <media/v4l2-vp9.h>
>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>>>
>>>>>>> #define MB_DIM 16
>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>> @@ -10,6 +10,8 @@
>>>>>>> #include "ipu3.h"
>>>>>>> #include "ipu3-dmamap.h"
>>>>>>>
>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>> +
>>>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>>>
>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>> b/include/media/videobuf2-core.h
>>>>>>> index 77921cf894ef..080b783d608d 100644
>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>> @@ -20,7 +20,6 @@
>>>>>>> #include <media/media-request.h>
>>>>>>> #include <media/frame_vector.h>
>>>>>>>
>>>>>>> -#define VB2_MAX_FRAME (32)
>>>>>>> #define VB2_MAX_PLANES (8)
>>>>>>>
>>>>>>> /**
>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>> index 5a845887850b..88a7a565170e 100644
>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>> @@ -15,10 +15,6 @@
>>>>>>> #include <linux/videodev2.h>
>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>
>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>> -#endif
>>>>>>> -
>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>> #endif
>>>>>>> --
>>>>>>> 2.39.2
>>>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-07-28 6:55 ` Hsia-Jun Li
(?)
@ 2023-07-28 7:03 ` Tomasz Figa
-1 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-28 7:03 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
>
> On 7/28/23 14:46, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
> >>
> >>
> >> On 7/12/23 18:48, Tomasz Figa wrote:
> >>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>
> >>>
> >>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
> >>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
> >>>>> CAUTION: Email originated externally, do not click links or open
> >>>>> attachments unless you recognize the sender and know the content is
> >>>>> safe.
> >>>>>
> >>>>>
> >>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> >>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>>> attachments unless you recognize the sender and know the content is
> >>>>>>> safe.
> >>>>>>>
> >>>>>>>
> >>>>>>> After changing bufs arrays to a dynamic allocated array
> >>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> >>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> >>>>>> more reasonable.
> >>>>>>
> >>>>>> It would be hard to iterate the whole array, it would go worse with a
> >>>>>> filter. Such iterate may need to go twice because you mix
> >>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
> >>>>> Here I don't want to change drivers behavior so I keep the same value.
> >>>>> If it happens that they need more buffers, like for dynamic resolution
> >>>>> change
> >>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >>>>>
> >>>> I just don't like the idea that using a variant length array here.
> >>>>
> >>> "I don't like" is not an argument. We had a number of arguments for
> >>> using a generic helper (originally idr, but later decided to go with
> >>> XArray, because the former is now deprecated) that we pointed out in
> >>> our review comments for previous revisions. It wasn't really about the
> >>> size being variable, but rather avoiding open coding things in vb2 and
> >>> duplicating what's already implemented in generic code.
> >>
> >> I just want to say I don't think we need a variable length array to
> >> store the buffer here.
> >>
> >> And the below is the reason that such a case could be avoided in the
> >> first place.
> >>
> >>>
> >>>> And I could explain why you won't need so many buffers for the performance
> >>>> of decoding.
> >>>>
> >>>> VP9 could support 10 reference frames in dpb.
> >>>>
> >>>> Even for those frequent resolution changing test set, it would only happen
> >>>> to two resolutions,
> >>>>
> >>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> >>>> also leaves enough slots for re-order latency.
> >>>>
> >>>> If your case had more two resolutions, likes low->medium->high.
> >>>>
> >>>> I would suggest just skip the medium resolutions, just allocate the lower
> >>>> one first for fast playback then the highest for all the possible
> >>>>
> >>>> medium cases. Reallocation happens frequently would only cause memory
> >>>> fragment, nothing benefits your performance.
> >>>>
> >>> We have mechanisms in the kernel to deal with memory fragmentation
> >>> (migration/compaction) and it would still only matters for the
> >>> pathologic cases of hardware that require physically contiguous memory.
> >>> Modern hardware with proper DMA capabilities can either scatter-gather
> >>> or are equipped with an IOMMU, so the allocation always happens in page
> >>> granularity and fragmentation is avoided.
> >>
> >> Unfortunately, there are more devices that didn't have a IOMMU attached
> >> to it, supporting scatter gather is more odd.
> >>
> >> It would be more likely that IOMMU would be disabled for the performance
> >> reason.
> >
> > These days IOMMU is totally mandatory if you want to think about
> > having any level of security in your system. Sure, there could be some
> > systems that are completely isolated from any external environment,
> > like some offline industry automation machines, but then arguably
> > their running conditions would also be quite static and require very
> > little memory re-allocation.
> Vendor just decided not to included such hardware.
> That is why From ION to DMA-heap, people like to allocate from a cavout
> out memory.
> >
> > I also don't buy the performance reason. CPUs have been behind MMUs
> > for ages and nobody is running them with paging disabled for
> > performance reasons. Similarly, most of the modern consumer systems
> Page lookup would increase the delay. Besides a few upstream devices
> prove them only use a level 1 page table without TBL.
That's just an excuse for a bad hardware design/implementation. As I
said, there are good IOMMU implementations out there that don't suffer
from performance issues.
> > (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
> > because of the security reason and they don't seem to be having any
> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
> The most security way here, we should use a dedicated memory(or a zone
> in unified memory).
You still need something to enforce that the hardware is not accessing
memory that it's not supposed to access. How do you do that without an
IOMMU?
> I believe there are more users in mobile for DMA-heap than kernel's dma
> allocation API.
Yes, but that's completely separate from whether there is an IOMMU or
not. It's just a different allocation API.
> > performance issues. In fact, it can improve the performance, because
> > memory allocation is much easier and without contiguous careouts (as
> > we used to have long ago on Android devices) the extra memory can be
> > used for buffers and caches to improve system performance.
> >
> > Best regards,
> > Tomasz
> >
> >>
> >>>
> >>> Best regards,
> >>> Tomasz
> >>>
> >>>>>>> Remove it from the core definitions but keep it for drivers internal
> >>>>>>> needs.
> >>>>>>>
> >>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>>>> ---
> >>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> >>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
> >>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> >>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> >>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> >>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> >>>>>>> include/media/videobuf2-core.h | 1 -
> >>>>>>> include/media/videobuf2-v4l2.h | 4 ----
> >>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
> >>>>>>>
> >>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> index 86e1e926fa45..899783f67580 100644
> >>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> @@ -31,6 +31,8 @@
> >>>>>>>
> >>>>>>> #include <trace/events/vb2.h>
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> static int debug;
> >>>>>>> module_param(debug, int, 0644);
> >>>>>>>
> >>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
> >>>>>>> b/drivers/media/platform/amphion/vdec.c
> >>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
> >>>>>>> --- a/drivers/media/platform/amphion/vdec.c
> >>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
> >>>>>>> @@ -28,6 +28,7 @@
> >>>>>>>
> >>>>>>> #define VDEC_MIN_BUFFER_CAP 8
> >>>>>>> #define VDEC_MIN_BUFFER_OUT 8
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>
> >>>>>>> struct vdec_fs_info {
> >>>>>>> char name[8];
> >>>>>>> diff --git
> >>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
> >>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> @@ -16,6 +16,8 @@
> >>>>>>> #include "../vdec_drv_if.h"
> >>>>>>> #include "../vdec_vpu_if.h"
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> /* reset_frame_context defined in VP9 spec */
> >>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> >>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> >>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> index f25d412d6553..bd5ca5a8b945 100644
> >>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> @@ -10,6 +10,8 @@
> >>>>>>>
> >>>>>>> #include "hfi_helper.h"
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> #define VIDC_SESSION_TYPE_VPE 0
> >>>>>>> #define VIDC_SESSION_TYPE_ENC 1
> >>>>>>> #define VIDC_SESSION_TYPE_DEC 2
> >>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
> >>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> @@ -15,6 +15,8 @@
> >>>>>>> #include <media/v4l2-vp9.h>
> >>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
> >>>>>>>
> >>>>>>> #define MB_DIM 16
> >>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> index e530767e80a5..6627b5c2d4d6 100644
> >>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> @@ -10,6 +10,8 @@
> >>>>>>> #include "ipu3.h"
> >>>>>>> #include "ipu3-dmamap.h"
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> /******************** v4l2_subdev_ops ********************/
> >>>>>>>
> >>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
> >>>>>>> diff --git a/include/media/videobuf2-core.h
> >>>>>>> b/include/media/videobuf2-core.h
> >>>>>>> index 77921cf894ef..080b783d608d 100644
> >>>>>>> --- a/include/media/videobuf2-core.h
> >>>>>>> +++ b/include/media/videobuf2-core.h
> >>>>>>> @@ -20,7 +20,6 @@
> >>>>>>> #include <media/media-request.h>
> >>>>>>> #include <media/frame_vector.h>
> >>>>>>>
> >>>>>>> -#define VB2_MAX_FRAME (32)
> >>>>>>> #define VB2_MAX_PLANES (8)
> >>>>>>>
> >>>>>>> /**
> >>>>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>>>> b/include/media/videobuf2-v4l2.h
> >>>>>>> index 5a845887850b..88a7a565170e 100644
> >>>>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>>>> @@ -15,10 +15,6 @@
> >>>>>>> #include <linux/videodev2.h>
> >>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>
> >>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>> -#endif
> >>>>>>> -
> >>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>> #endif
> >>>>>>> --
> >>>>>>> 2.39.2
> >>>>>>>
> >>>> --
> >>>> Hsia-Jun(Randy) Li
> >>>>
> >> --
> >> Hsia-Jun(Randy) Li
> >>
>
> --
> Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-28 7:03 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-28 7:03 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
>
> On 7/28/23 14:46, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
> >>
> >>
> >> On 7/12/23 18:48, Tomasz Figa wrote:
> >>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>
> >>>
> >>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
> >>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
> >>>>> CAUTION: Email originated externally, do not click links or open
> >>>>> attachments unless you recognize the sender and know the content is
> >>>>> safe.
> >>>>>
> >>>>>
> >>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> >>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>>> attachments unless you recognize the sender and know the content is
> >>>>>>> safe.
> >>>>>>>
> >>>>>>>
> >>>>>>> After changing bufs arrays to a dynamic allocated array
> >>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> >>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> >>>>>> more reasonable.
> >>>>>>
> >>>>>> It would be hard to iterate the whole array, it would go worse with a
> >>>>>> filter. Such iterate may need to go twice because you mix
> >>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
> >>>>> Here I don't want to change drivers behavior so I keep the same value.
> >>>>> If it happens that they need more buffers, like for dynamic resolution
> >>>>> change
> >>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >>>>>
> >>>> I just don't like the idea that using a variant length array here.
> >>>>
> >>> "I don't like" is not an argument. We had a number of arguments for
> >>> using a generic helper (originally idr, but later decided to go with
> >>> XArray, because the former is now deprecated) that we pointed out in
> >>> our review comments for previous revisions. It wasn't really about the
> >>> size being variable, but rather avoiding open coding things in vb2 and
> >>> duplicating what's already implemented in generic code.
> >>
> >> I just want to say I don't think we need a variable length array to
> >> store the buffer here.
> >>
> >> And the below is the reason that such a case could be avoided in the
> >> first place.
> >>
> >>>
> >>>> And I could explain why you won't need so many buffers for the performance
> >>>> of decoding.
> >>>>
> >>>> VP9 could support 10 reference frames in dpb.
> >>>>
> >>>> Even for those frequent resolution changing test set, it would only happen
> >>>> to two resolutions,
> >>>>
> >>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> >>>> also leaves enough slots for re-order latency.
> >>>>
> >>>> If your case had more two resolutions, likes low->medium->high.
> >>>>
> >>>> I would suggest just skip the medium resolutions, just allocate the lower
> >>>> one first for fast playback then the highest for all the possible
> >>>>
> >>>> medium cases. Reallocation happens frequently would only cause memory
> >>>> fragment, nothing benefits your performance.
> >>>>
> >>> We have mechanisms in the kernel to deal with memory fragmentation
> >>> (migration/compaction) and it would still only matters for the
> >>> pathologic cases of hardware that require physically contiguous memory.
> >>> Modern hardware with proper DMA capabilities can either scatter-gather
> >>> or are equipped with an IOMMU, so the allocation always happens in page
> >>> granularity and fragmentation is avoided.
> >>
> >> Unfortunately, there are more devices that didn't have a IOMMU attached
> >> to it, supporting scatter gather is more odd.
> >>
> >> It would be more likely that IOMMU would be disabled for the performance
> >> reason.
> >
> > These days IOMMU is totally mandatory if you want to think about
> > having any level of security in your system. Sure, there could be some
> > systems that are completely isolated from any external environment,
> > like some offline industry automation machines, but then arguably
> > their running conditions would also be quite static and require very
> > little memory re-allocation.
> Vendor just decided not to included such hardware.
> That is why From ION to DMA-heap, people like to allocate from a cavout
> out memory.
> >
> > I also don't buy the performance reason. CPUs have been behind MMUs
> > for ages and nobody is running them with paging disabled for
> > performance reasons. Similarly, most of the modern consumer systems
> Page lookup would increase the delay. Besides a few upstream devices
> prove them only use a level 1 page table without TBL.
That's just an excuse for a bad hardware design/implementation. As I
said, there are good IOMMU implementations out there that don't suffer
from performance issues.
> > (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
> > because of the security reason and they don't seem to be having any
> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
> The most security way here, we should use a dedicated memory(or a zone
> in unified memory).
You still need something to enforce that the hardware is not accessing
memory that it's not supposed to access. How do you do that without an
IOMMU?
> I believe there are more users in mobile for DMA-heap than kernel's dma
> allocation API.
Yes, but that's completely separate from whether there is an IOMMU or
not. It's just a different allocation API.
> > performance issues. In fact, it can improve the performance, because
> > memory allocation is much easier and without contiguous careouts (as
> > we used to have long ago on Android devices) the extra memory can be
> > used for buffers and caches to improve system performance.
> >
> > Best regards,
> > Tomasz
> >
> >>
> >>>
> >>> Best regards,
> >>> Tomasz
> >>>
> >>>>>>> Remove it from the core definitions but keep it for drivers internal
> >>>>>>> needs.
> >>>>>>>
> >>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>>>> ---
> >>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> >>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
> >>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> >>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> >>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> >>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> >>>>>>> include/media/videobuf2-core.h | 1 -
> >>>>>>> include/media/videobuf2-v4l2.h | 4 ----
> >>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
> >>>>>>>
> >>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> index 86e1e926fa45..899783f67580 100644
> >>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> @@ -31,6 +31,8 @@
> >>>>>>>
> >>>>>>> #include <trace/events/vb2.h>
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> static int debug;
> >>>>>>> module_param(debug, int, 0644);
> >>>>>>>
> >>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
> >>>>>>> b/drivers/media/platform/amphion/vdec.c
> >>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
> >>>>>>> --- a/drivers/media/platform/amphion/vdec.c
> >>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
> >>>>>>> @@ -28,6 +28,7 @@
> >>>>>>>
> >>>>>>> #define VDEC_MIN_BUFFER_CAP 8
> >>>>>>> #define VDEC_MIN_BUFFER_OUT 8
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>
> >>>>>>> struct vdec_fs_info {
> >>>>>>> char name[8];
> >>>>>>> diff --git
> >>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
> >>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> @@ -16,6 +16,8 @@
> >>>>>>> #include "../vdec_drv_if.h"
> >>>>>>> #include "../vdec_vpu_if.h"
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> /* reset_frame_context defined in VP9 spec */
> >>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> >>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> >>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> index f25d412d6553..bd5ca5a8b945 100644
> >>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> @@ -10,6 +10,8 @@
> >>>>>>>
> >>>>>>> #include "hfi_helper.h"
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> #define VIDC_SESSION_TYPE_VPE 0
> >>>>>>> #define VIDC_SESSION_TYPE_ENC 1
> >>>>>>> #define VIDC_SESSION_TYPE_DEC 2
> >>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
> >>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> @@ -15,6 +15,8 @@
> >>>>>>> #include <media/v4l2-vp9.h>
> >>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
> >>>>>>>
> >>>>>>> #define MB_DIM 16
> >>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> index e530767e80a5..6627b5c2d4d6 100644
> >>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> @@ -10,6 +10,8 @@
> >>>>>>> #include "ipu3.h"
> >>>>>>> #include "ipu3-dmamap.h"
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> /******************** v4l2_subdev_ops ********************/
> >>>>>>>
> >>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
> >>>>>>> diff --git a/include/media/videobuf2-core.h
> >>>>>>> b/include/media/videobuf2-core.h
> >>>>>>> index 77921cf894ef..080b783d608d 100644
> >>>>>>> --- a/include/media/videobuf2-core.h
> >>>>>>> +++ b/include/media/videobuf2-core.h
> >>>>>>> @@ -20,7 +20,6 @@
> >>>>>>> #include <media/media-request.h>
> >>>>>>> #include <media/frame_vector.h>
> >>>>>>>
> >>>>>>> -#define VB2_MAX_FRAME (32)
> >>>>>>> #define VB2_MAX_PLANES (8)
> >>>>>>>
> >>>>>>> /**
> >>>>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>>>> b/include/media/videobuf2-v4l2.h
> >>>>>>> index 5a845887850b..88a7a565170e 100644
> >>>>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>>>> @@ -15,10 +15,6 @@
> >>>>>>> #include <linux/videodev2.h>
> >>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>
> >>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>> -#endif
> >>>>>>> -
> >>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>> #endif
> >>>>>>> --
> >>>>>>> 2.39.2
> >>>>>>>
> >>>> --
> >>>> Hsia-Jun(Randy) Li
> >>>>
> >> --
> >> Hsia-Jun(Randy) Li
> >>
>
> --
> Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-28 7:03 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-28 7:03 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
>
> On 7/28/23 14:46, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
> >>
> >>
> >> On 7/12/23 18:48, Tomasz Figa wrote:
> >>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>
> >>>
> >>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
> >>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
> >>>>> CAUTION: Email originated externally, do not click links or open
> >>>>> attachments unless you recognize the sender and know the content is
> >>>>> safe.
> >>>>>
> >>>>>
> >>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> >>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>>> attachments unless you recognize the sender and know the content is
> >>>>>>> safe.
> >>>>>>>
> >>>>>>>
> >>>>>>> After changing bufs arrays to a dynamic allocated array
> >>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> >>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> >>>>>> more reasonable.
> >>>>>>
> >>>>>> It would be hard to iterate the whole array, it would go worse with a
> >>>>>> filter. Such iterate may need to go twice because you mix
> >>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
> >>>>> Here I don't want to change drivers behavior so I keep the same value.
> >>>>> If it happens that they need more buffers, like for dynamic resolution
> >>>>> change
> >>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >>>>>
> >>>> I just don't like the idea that using a variant length array here.
> >>>>
> >>> "I don't like" is not an argument. We had a number of arguments for
> >>> using a generic helper (originally idr, but later decided to go with
> >>> XArray, because the former is now deprecated) that we pointed out in
> >>> our review comments for previous revisions. It wasn't really about the
> >>> size being variable, but rather avoiding open coding things in vb2 and
> >>> duplicating what's already implemented in generic code.
> >>
> >> I just want to say I don't think we need a variable length array to
> >> store the buffer here.
> >>
> >> And the below is the reason that such a case could be avoided in the
> >> first place.
> >>
> >>>
> >>>> And I could explain why you won't need so many buffers for the performance
> >>>> of decoding.
> >>>>
> >>>> VP9 could support 10 reference frames in dpb.
> >>>>
> >>>> Even for those frequent resolution changing test set, it would only happen
> >>>> to two resolutions,
> >>>>
> >>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> >>>> also leaves enough slots for re-order latency.
> >>>>
> >>>> If your case had more two resolutions, likes low->medium->high.
> >>>>
> >>>> I would suggest just skip the medium resolutions, just allocate the lower
> >>>> one first for fast playback then the highest for all the possible
> >>>>
> >>>> medium cases. Reallocation happens frequently would only cause memory
> >>>> fragment, nothing benefits your performance.
> >>>>
> >>> We have mechanisms in the kernel to deal with memory fragmentation
> >>> (migration/compaction) and it would still only matters for the
> >>> pathologic cases of hardware that require physically contiguous memory.
> >>> Modern hardware with proper DMA capabilities can either scatter-gather
> >>> or are equipped with an IOMMU, so the allocation always happens in page
> >>> granularity and fragmentation is avoided.
> >>
> >> Unfortunately, there are more devices that didn't have a IOMMU attached
> >> to it, supporting scatter gather is more odd.
> >>
> >> It would be more likely that IOMMU would be disabled for the performance
> >> reason.
> >
> > These days IOMMU is totally mandatory if you want to think about
> > having any level of security in your system. Sure, there could be some
> > systems that are completely isolated from any external environment,
> > like some offline industry automation machines, but then arguably
> > their running conditions would also be quite static and require very
> > little memory re-allocation.
> Vendor just decided not to included such hardware.
> That is why From ION to DMA-heap, people like to allocate from a cavout
> out memory.
> >
> > I also don't buy the performance reason. CPUs have been behind MMUs
> > for ages and nobody is running them with paging disabled for
> > performance reasons. Similarly, most of the modern consumer systems
> Page lookup would increase the delay. Besides a few upstream devices
> prove them only use a level 1 page table without TBL.
That's just an excuse for a bad hardware design/implementation. As I
said, there are good IOMMU implementations out there that don't suffer
from performance issues.
> > (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
> > because of the security reason and they don't seem to be having any
> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
> The most security way here, we should use a dedicated memory(or a zone
> in unified memory).
You still need something to enforce that the hardware is not accessing
memory that it's not supposed to access. How do you do that without an
IOMMU?
> I believe there are more users in mobile for DMA-heap than kernel's dma
> allocation API.
Yes, but that's completely separate from whether there is an IOMMU or
not. It's just a different allocation API.
> > performance issues. In fact, it can improve the performance, because
> > memory allocation is much easier and without contiguous careouts (as
> > we used to have long ago on Android devices) the extra memory can be
> > used for buffers and caches to improve system performance.
> >
> > Best regards,
> > Tomasz
> >
> >>
> >>>
> >>> Best regards,
> >>> Tomasz
> >>>
> >>>>>>> Remove it from the core definitions but keep it for drivers internal
> >>>>>>> needs.
> >>>>>>>
> >>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>>>> ---
> >>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> >>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
> >>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> >>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> >>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> >>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> >>>>>>> include/media/videobuf2-core.h | 1 -
> >>>>>>> include/media/videobuf2-v4l2.h | 4 ----
> >>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
> >>>>>>>
> >>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> index 86e1e926fa45..899783f67580 100644
> >>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>> @@ -31,6 +31,8 @@
> >>>>>>>
> >>>>>>> #include <trace/events/vb2.h>
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> static int debug;
> >>>>>>> module_param(debug, int, 0644);
> >>>>>>>
> >>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
> >>>>>>> b/drivers/media/platform/amphion/vdec.c
> >>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
> >>>>>>> --- a/drivers/media/platform/amphion/vdec.c
> >>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
> >>>>>>> @@ -28,6 +28,7 @@
> >>>>>>>
> >>>>>>> #define VDEC_MIN_BUFFER_CAP 8
> >>>>>>> #define VDEC_MIN_BUFFER_OUT 8
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>
> >>>>>>> struct vdec_fs_info {
> >>>>>>> char name[8];
> >>>>>>> diff --git
> >>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
> >>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>> @@ -16,6 +16,8 @@
> >>>>>>> #include "../vdec_drv_if.h"
> >>>>>>> #include "../vdec_vpu_if.h"
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> /* reset_frame_context defined in VP9 spec */
> >>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> >>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> >>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> index f25d412d6553..bd5ca5a8b945 100644
> >>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>> @@ -10,6 +10,8 @@
> >>>>>>>
> >>>>>>> #include "hfi_helper.h"
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> #define VIDC_SESSION_TYPE_VPE 0
> >>>>>>> #define VIDC_SESSION_TYPE_ENC 1
> >>>>>>> #define VIDC_SESSION_TYPE_DEC 2
> >>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
> >>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>> @@ -15,6 +15,8 @@
> >>>>>>> #include <media/v4l2-vp9.h>
> >>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
> >>>>>>>
> >>>>>>> #define MB_DIM 16
> >>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> index e530767e80a5..6627b5c2d4d6 100644
> >>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>> @@ -10,6 +10,8 @@
> >>>>>>> #include "ipu3.h"
> >>>>>>> #include "ipu3-dmamap.h"
> >>>>>>>
> >>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>> +
> >>>>>>> /******************** v4l2_subdev_ops ********************/
> >>>>>>>
> >>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
> >>>>>>> diff --git a/include/media/videobuf2-core.h
> >>>>>>> b/include/media/videobuf2-core.h
> >>>>>>> index 77921cf894ef..080b783d608d 100644
> >>>>>>> --- a/include/media/videobuf2-core.h
> >>>>>>> +++ b/include/media/videobuf2-core.h
> >>>>>>> @@ -20,7 +20,6 @@
> >>>>>>> #include <media/media-request.h>
> >>>>>>> #include <media/frame_vector.h>
> >>>>>>>
> >>>>>>> -#define VB2_MAX_FRAME (32)
> >>>>>>> #define VB2_MAX_PLANES (8)
> >>>>>>>
> >>>>>>> /**
> >>>>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>>>> b/include/media/videobuf2-v4l2.h
> >>>>>>> index 5a845887850b..88a7a565170e 100644
> >>>>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>>>> @@ -15,10 +15,6 @@
> >>>>>>> #include <linux/videodev2.h>
> >>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>
> >>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>> -#endif
> >>>>>>> -
> >>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>> #endif
> >>>>>>> --
> >>>>>>> 2.39.2
> >>>>>>>
> >>>> --
> >>>> Hsia-Jun(Randy) Li
> >>>>
> >> --
> >> Hsia-Jun(Randy) Li
> >>
>
> --
> Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-07-28 7:03 ` Tomasz Figa
(?)
@ 2023-07-28 7:24 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-28 7:24 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On 7/28/23 15:03, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>>
>> On 7/28/23 14:46, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>>>
>>>>
>>>> On 7/12/23 18:48, Tomasz Figa wrote:
>>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>
>>>>>
>>>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>>>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> After changing bufs arrays to a dynamic allocated array
>>>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>>>>>> more reasonable.
>>>>>>>>
>>>>>>>> It would be hard to iterate the whole array, it would go worse with a
>>>>>>>> filter. Such iterate may need to go twice because you mix
>>>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>>>>> Here I don't want to change drivers behavior so I keep the same value.
>>>>>>> If it happens that they need more buffers, like for dynamic resolution
>>>>>>> change
>>>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>>>>>
>>>>>> I just don't like the idea that using a variant length array here.
>>>>>>
>>>>> "I don't like" is not an argument. We had a number of arguments for
>>>>> using a generic helper (originally idr, but later decided to go with
>>>>> XArray, because the former is now deprecated) that we pointed out in
>>>>> our review comments for previous revisions. It wasn't really about the
>>>>> size being variable, but rather avoiding open coding things in vb2 and
>>>>> duplicating what's already implemented in generic code.
>>>>
>>>> I just want to say I don't think we need a variable length array to
>>>> store the buffer here.
>>>>
>>>> And the below is the reason that such a case could be avoided in the
>>>> first place.
>>>>
>>>>>
>>>>>> And I could explain why you won't need so many buffers for the performance
>>>>>> of decoding.
>>>>>>
>>>>>> VP9 could support 10 reference frames in dpb.
>>>>>>
>>>>>> Even for those frequent resolution changing test set, it would only happen
>>>>>> to two resolutions,
>>>>>>
>>>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>>>>>> also leaves enough slots for re-order latency.
>>>>>>
>>>>>> If your case had more two resolutions, likes low->medium->high.
>>>>>>
>>>>>> I would suggest just skip the medium resolutions, just allocate the lower
>>>>>> one first for fast playback then the highest for all the possible
>>>>>>
>>>>>> medium cases. Reallocation happens frequently would only cause memory
>>>>>> fragment, nothing benefits your performance.
>>>>>>
>>>>> We have mechanisms in the kernel to deal with memory fragmentation
>>>>> (migration/compaction) and it would still only matters for the
>>>>> pathologic cases of hardware that require physically contiguous memory.
>>>>> Modern hardware with proper DMA capabilities can either scatter-gather
>>>>> or are equipped with an IOMMU, so the allocation always happens in page
>>>>> granularity and fragmentation is avoided.
>>>>
>>>> Unfortunately, there are more devices that didn't have a IOMMU attached
>>>> to it, supporting scatter gather is more odd.
>>>>
>>>> It would be more likely that IOMMU would be disabled for the performance
>>>> reason.
>>>
>>> These days IOMMU is totally mandatory if you want to think about
>>> having any level of security in your system. Sure, there could be some
>>> systems that are completely isolated from any external environment,
>>> like some offline industry automation machines, but then arguably
>>> their running conditions would also be quite static and require very
>>> little memory re-allocation.
>> Vendor just decided not to included such hardware.
>> That is why From ION to DMA-heap, people like to allocate from a cavout
>> out memory.
>>>
>>> I also don't buy the performance reason. CPUs have been behind MMUs
>>> for ages and nobody is running them with paging disabled for
>>> performance reasons. Similarly, most of the modern consumer systems
>> Page lookup would increase the delay. Besides a few upstream devices
>> prove them only use a level 1 page table without TBL.
>
> That's just an excuse for a bad hardware design/implementation. As I
> said, there are good IOMMU implementations out there that don't suffer
> from performance issues.
>
I could do nothing about that.
Besides, even with TLB, cache missing could happen frequently,
especially we need to access many (5~16, 10 usually) buffers and more
11MBytes each in a hardware processing.
You can't have a very large TLB.
>>> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
>>> because of the security reason and they don't seem to be having any
>> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
>> The most security way here, we should use a dedicated memory(or a zone
>> in unified memory).
>
> You still need something to enforce that the hardware is not accessing
> memory that it's not supposed to access. How do you do that without an
> IOMMU?
>
If you know the arm security pipeline and security controller, you could
found we could reserved a range of memory for a security id(devices in
secure world may be a different security domain).
Besides, a MMU or security MPU could mark some pages for the secure
world access only, it doesn't mean the device need an IOMMU to access
them. The MPU could filter the access through the AXI id.
>> I believe there are more users in mobile for DMA-heap than kernel's dma
>> allocation API.
>
> Yes, but that's completely separate from whether there is an IOMMU or
> not. It's just a different allocation API.
>
The memory heap would mean a dedicated memory usually(we don't talk
about system heap or why there are many vendor heaps). Dedicated memory
means contiguous memory in the most of cases.
>>> performance issues. In fact, it can improve the performance, because
>>> memory allocation is much easier and without contiguous careouts (as
>>> we used to have long ago on Android devices) the extra memory can be
>>> used for buffers and caches to improve system performance.
>>>
>>> Best regards,
>>> Tomasz
>>>
>>>>
>>>>>
>>>>> Best regards,
>>>>> Tomasz
>>>>>
>>>>>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>>>>>> needs.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>>> ---
>>>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>>>>>> include/media/videobuf2-core.h | 1 -
>>>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>>>>>
>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> index 86e1e926fa45..899783f67580 100644
>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> @@ -31,6 +31,8 @@
>>>>>>>>>
>>>>>>>>> #include <trace/events/vb2.h>
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> static int debug;
>>>>>>>>> module_param(debug, int, 0644);
>>>>>>>>>
>>>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>>>>>> @@ -28,6 +28,7 @@
>>>>>>>>>
>>>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>
>>>>>>>>> struct vdec_fs_info {
>>>>>>>>> char name[8];
>>>>>>>>> diff --git
>>>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> @@ -16,6 +16,8 @@
>>>>>>>>> #include "../vdec_drv_if.h"
>>>>>>>>> #include "../vdec_vpu_if.h"
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> /* reset_frame_context defined in VP9 spec */
>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>>
>>>>>>>>> #include "hfi_helper.h"
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> @@ -15,6 +15,8 @@
>>>>>>>>> #include <media/v4l2-vp9.h>
>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>>>>>
>>>>>>>>> #define MB_DIM 16
>>>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>> #include "ipu3.h"
>>>>>>>>> #include "ipu3-dmamap.h"
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>>>>>
>>>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>> index 77921cf894ef..080b783d608d 100644
>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>> @@ -20,7 +20,6 @@
>>>>>>>>> #include <media/media-request.h>
>>>>>>>>> #include <media/frame_vector.h>
>>>>>>>>>
>>>>>>>>> -#define VB2_MAX_FRAME (32)
>>>>>>>>> #define VB2_MAX_PLANES (8)
>>>>>>>>>
>>>>>>>>> /**
>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>> index 5a845887850b..88a7a565170e 100644
>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>> @@ -15,10 +15,6 @@
>>>>>>>>> #include <linux/videodev2.h>
>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>
>>>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>> -#endif
>>>>>>>>> -
>>>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>> #endif
>>>>>>>>> --
>>>>>>>>> 2.39.2
>>>>>>>>>
>>>>>> --
>>>>>> Hsia-Jun(Randy) Li
>>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
>>
>> --
>> Hsia-Jun(Randy) Li
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-28 7:24 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-28 7:24 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On 7/28/23 15:03, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>>
>> On 7/28/23 14:46, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>>>
>>>>
>>>> On 7/12/23 18:48, Tomasz Figa wrote:
>>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>
>>>>>
>>>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>>>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> After changing bufs arrays to a dynamic allocated array
>>>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>>>>>> more reasonable.
>>>>>>>>
>>>>>>>> It would be hard to iterate the whole array, it would go worse with a
>>>>>>>> filter. Such iterate may need to go twice because you mix
>>>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>>>>> Here I don't want to change drivers behavior so I keep the same value.
>>>>>>> If it happens that they need more buffers, like for dynamic resolution
>>>>>>> change
>>>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>>>>>
>>>>>> I just don't like the idea that using a variant length array here.
>>>>>>
>>>>> "I don't like" is not an argument. We had a number of arguments for
>>>>> using a generic helper (originally idr, but later decided to go with
>>>>> XArray, because the former is now deprecated) that we pointed out in
>>>>> our review comments for previous revisions. It wasn't really about the
>>>>> size being variable, but rather avoiding open coding things in vb2 and
>>>>> duplicating what's already implemented in generic code.
>>>>
>>>> I just want to say I don't think we need a variable length array to
>>>> store the buffer here.
>>>>
>>>> And the below is the reason that such a case could be avoided in the
>>>> first place.
>>>>
>>>>>
>>>>>> And I could explain why you won't need so many buffers for the performance
>>>>>> of decoding.
>>>>>>
>>>>>> VP9 could support 10 reference frames in dpb.
>>>>>>
>>>>>> Even for those frequent resolution changing test set, it would only happen
>>>>>> to two resolutions,
>>>>>>
>>>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>>>>>> also leaves enough slots for re-order latency.
>>>>>>
>>>>>> If your case had more two resolutions, likes low->medium->high.
>>>>>>
>>>>>> I would suggest just skip the medium resolutions, just allocate the lower
>>>>>> one first for fast playback then the highest for all the possible
>>>>>>
>>>>>> medium cases. Reallocation happens frequently would only cause memory
>>>>>> fragment, nothing benefits your performance.
>>>>>>
>>>>> We have mechanisms in the kernel to deal with memory fragmentation
>>>>> (migration/compaction) and it would still only matters for the
>>>>> pathologic cases of hardware that require physically contiguous memory.
>>>>> Modern hardware with proper DMA capabilities can either scatter-gather
>>>>> or are equipped with an IOMMU, so the allocation always happens in page
>>>>> granularity and fragmentation is avoided.
>>>>
>>>> Unfortunately, there are more devices that didn't have a IOMMU attached
>>>> to it, supporting scatter gather is more odd.
>>>>
>>>> It would be more likely that IOMMU would be disabled for the performance
>>>> reason.
>>>
>>> These days IOMMU is totally mandatory if you want to think about
>>> having any level of security in your system. Sure, there could be some
>>> systems that are completely isolated from any external environment,
>>> like some offline industry automation machines, but then arguably
>>> their running conditions would also be quite static and require very
>>> little memory re-allocation.
>> Vendor just decided not to included such hardware.
>> That is why From ION to DMA-heap, people like to allocate from a cavout
>> out memory.
>>>
>>> I also don't buy the performance reason. CPUs have been behind MMUs
>>> for ages and nobody is running them with paging disabled for
>>> performance reasons. Similarly, most of the modern consumer systems
>> Page lookup would increase the delay. Besides a few upstream devices
>> prove them only use a level 1 page table without TBL.
>
> That's just an excuse for a bad hardware design/implementation. As I
> said, there are good IOMMU implementations out there that don't suffer
> from performance issues.
>
I could do nothing about that.
Besides, even with TLB, cache missing could happen frequently,
especially we need to access many (5~16, 10 usually) buffers and more
11MBytes each in a hardware processing.
You can't have a very large TLB.
>>> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
>>> because of the security reason and they don't seem to be having any
>> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
>> The most security way here, we should use a dedicated memory(or a zone
>> in unified memory).
>
> You still need something to enforce that the hardware is not accessing
> memory that it's not supposed to access. How do you do that without an
> IOMMU?
>
If you know the arm security pipeline and security controller, you could
found we could reserved a range of memory for a security id(devices in
secure world may be a different security domain).
Besides, a MMU or security MPU could mark some pages for the secure
world access only, it doesn't mean the device need an IOMMU to access
them. The MPU could filter the access through the AXI id.
>> I believe there are more users in mobile for DMA-heap than kernel's dma
>> allocation API.
>
> Yes, but that's completely separate from whether there is an IOMMU or
> not. It's just a different allocation API.
>
The memory heap would mean a dedicated memory usually(we don't talk
about system heap or why there are many vendor heaps). Dedicated memory
means contiguous memory in the most of cases.
>>> performance issues. In fact, it can improve the performance, because
>>> memory allocation is much easier and without contiguous careouts (as
>>> we used to have long ago on Android devices) the extra memory can be
>>> used for buffers and caches to improve system performance.
>>>
>>> Best regards,
>>> Tomasz
>>>
>>>>
>>>>>
>>>>> Best regards,
>>>>> Tomasz
>>>>>
>>>>>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>>>>>> needs.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>>> ---
>>>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>>>>>> include/media/videobuf2-core.h | 1 -
>>>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>>>>>
>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> index 86e1e926fa45..899783f67580 100644
>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> @@ -31,6 +31,8 @@
>>>>>>>>>
>>>>>>>>> #include <trace/events/vb2.h>
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> static int debug;
>>>>>>>>> module_param(debug, int, 0644);
>>>>>>>>>
>>>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>>>>>> @@ -28,6 +28,7 @@
>>>>>>>>>
>>>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>
>>>>>>>>> struct vdec_fs_info {
>>>>>>>>> char name[8];
>>>>>>>>> diff --git
>>>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> @@ -16,6 +16,8 @@
>>>>>>>>> #include "../vdec_drv_if.h"
>>>>>>>>> #include "../vdec_vpu_if.h"
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> /* reset_frame_context defined in VP9 spec */
>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>>
>>>>>>>>> #include "hfi_helper.h"
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> @@ -15,6 +15,8 @@
>>>>>>>>> #include <media/v4l2-vp9.h>
>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>>>>>
>>>>>>>>> #define MB_DIM 16
>>>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>> #include "ipu3.h"
>>>>>>>>> #include "ipu3-dmamap.h"
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>>>>>
>>>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>> index 77921cf894ef..080b783d608d 100644
>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>> @@ -20,7 +20,6 @@
>>>>>>>>> #include <media/media-request.h>
>>>>>>>>> #include <media/frame_vector.h>
>>>>>>>>>
>>>>>>>>> -#define VB2_MAX_FRAME (32)
>>>>>>>>> #define VB2_MAX_PLANES (8)
>>>>>>>>>
>>>>>>>>> /**
>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>> index 5a845887850b..88a7a565170e 100644
>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>> @@ -15,10 +15,6 @@
>>>>>>>>> #include <linux/videodev2.h>
>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>
>>>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>> -#endif
>>>>>>>>> -
>>>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>> #endif
>>>>>>>>> --
>>>>>>>>> 2.39.2
>>>>>>>>>
>>>>>> --
>>>>>> Hsia-Jun(Randy) Li
>>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
>>
>> --
>> Hsia-Jun(Randy) Li
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-07-28 7:24 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-28 7:24 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On 7/28/23 15:03, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>>
>> On 7/28/23 14:46, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>>>
>>>>
>>>> On 7/12/23 18:48, Tomasz Figa wrote:
>>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>
>>>>>
>>>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>>>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> After changing bufs arrays to a dynamic allocated array
>>>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>>>>>> more reasonable.
>>>>>>>>
>>>>>>>> It would be hard to iterate the whole array, it would go worse with a
>>>>>>>> filter. Such iterate may need to go twice because you mix
>>>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>>>>> Here I don't want to change drivers behavior so I keep the same value.
>>>>>>> If it happens that they need more buffers, like for dynamic resolution
>>>>>>> change
>>>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>>>>>
>>>>>> I just don't like the idea that using a variant length array here.
>>>>>>
>>>>> "I don't like" is not an argument. We had a number of arguments for
>>>>> using a generic helper (originally idr, but later decided to go with
>>>>> XArray, because the former is now deprecated) that we pointed out in
>>>>> our review comments for previous revisions. It wasn't really about the
>>>>> size being variable, but rather avoiding open coding things in vb2 and
>>>>> duplicating what's already implemented in generic code.
>>>>
>>>> I just want to say I don't think we need a variable length array to
>>>> store the buffer here.
>>>>
>>>> And the below is the reason that such a case could be avoided in the
>>>> first place.
>>>>
>>>>>
>>>>>> And I could explain why you won't need so many buffers for the performance
>>>>>> of decoding.
>>>>>>
>>>>>> VP9 could support 10 reference frames in dpb.
>>>>>>
>>>>>> Even for those frequent resolution changing test set, it would only happen
>>>>>> to two resolutions,
>>>>>>
>>>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>>>>>> also leaves enough slots for re-order latency.
>>>>>>
>>>>>> If your case had more two resolutions, likes low->medium->high.
>>>>>>
>>>>>> I would suggest just skip the medium resolutions, just allocate the lower
>>>>>> one first for fast playback then the highest for all the possible
>>>>>>
>>>>>> medium cases. Reallocation happens frequently would only cause memory
>>>>>> fragment, nothing benefits your performance.
>>>>>>
>>>>> We have mechanisms in the kernel to deal with memory fragmentation
>>>>> (migration/compaction) and it would still only matters for the
>>>>> pathologic cases of hardware that require physically contiguous memory.
>>>>> Modern hardware with proper DMA capabilities can either scatter-gather
>>>>> or are equipped with an IOMMU, so the allocation always happens in page
>>>>> granularity and fragmentation is avoided.
>>>>
>>>> Unfortunately, there are more devices that didn't have a IOMMU attached
>>>> to it, supporting scatter gather is more odd.
>>>>
>>>> It would be more likely that IOMMU would be disabled for the performance
>>>> reason.
>>>
>>> These days IOMMU is totally mandatory if you want to think about
>>> having any level of security in your system. Sure, there could be some
>>> systems that are completely isolated from any external environment,
>>> like some offline industry automation machines, but then arguably
>>> their running conditions would also be quite static and require very
>>> little memory re-allocation.
>> Vendor just decided not to included such hardware.
>> That is why From ION to DMA-heap, people like to allocate from a cavout
>> out memory.
>>>
>>> I also don't buy the performance reason. CPUs have been behind MMUs
>>> for ages and nobody is running them with paging disabled for
>>> performance reasons. Similarly, most of the modern consumer systems
>> Page lookup would increase the delay. Besides a few upstream devices
>> prove them only use a level 1 page table without TBL.
>
> That's just an excuse for a bad hardware design/implementation. As I
> said, there are good IOMMU implementations out there that don't suffer
> from performance issues.
>
I could do nothing about that.
Besides, even with TLB, cache missing could happen frequently,
especially we need to access many (5~16, 10 usually) buffers and more
11MBytes each in a hardware processing.
You can't have a very large TLB.
>>> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
>>> because of the security reason and they don't seem to be having any
>> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
>> The most security way here, we should use a dedicated memory(or a zone
>> in unified memory).
>
> You still need something to enforce that the hardware is not accessing
> memory that it's not supposed to access. How do you do that without an
> IOMMU?
>
If you know the arm security pipeline and security controller, you could
found we could reserved a range of memory for a security id(devices in
secure world may be a different security domain).
Besides, a MMU or security MPU could mark some pages for the secure
world access only, it doesn't mean the device need an IOMMU to access
them. The MPU could filter the access through the AXI id.
>> I believe there are more users in mobile for DMA-heap than kernel's dma
>> allocation API.
>
> Yes, but that's completely separate from whether there is an IOMMU or
> not. It's just a different allocation API.
>
The memory heap would mean a dedicated memory usually(we don't talk
about system heap or why there are many vendor heaps). Dedicated memory
means contiguous memory in the most of cases.
>>> performance issues. In fact, it can improve the performance, because
>>> memory allocation is much easier and without contiguous careouts (as
>>> we used to have long ago on Android devices) the extra memory can be
>>> used for buffers and caches to improve system performance.
>>>
>>> Best regards,
>>> Tomasz
>>>
>>>>
>>>>>
>>>>> Best regards,
>>>>> Tomasz
>>>>>
>>>>>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>>>>>> needs.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>>> ---
>>>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>>>>>> include/media/videobuf2-core.h | 1 -
>>>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>>>>>
>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> index 86e1e926fa45..899783f67580 100644
>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> @@ -31,6 +31,8 @@
>>>>>>>>>
>>>>>>>>> #include <trace/events/vb2.h>
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> static int debug;
>>>>>>>>> module_param(debug, int, 0644);
>>>>>>>>>
>>>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>>>>>> @@ -28,6 +28,7 @@
>>>>>>>>>
>>>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>
>>>>>>>>> struct vdec_fs_info {
>>>>>>>>> char name[8];
>>>>>>>>> diff --git
>>>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>> @@ -16,6 +16,8 @@
>>>>>>>>> #include "../vdec_drv_if.h"
>>>>>>>>> #include "../vdec_vpu_if.h"
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> /* reset_frame_context defined in VP9 spec */
>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>>
>>>>>>>>> #include "hfi_helper.h"
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>> @@ -15,6 +15,8 @@
>>>>>>>>> #include <media/v4l2-vp9.h>
>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>>>>>
>>>>>>>>> #define MB_DIM 16
>>>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>> #include "ipu3.h"
>>>>>>>>> #include "ipu3-dmamap.h"
>>>>>>>>>
>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>> +
>>>>>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>>>>>
>>>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>> index 77921cf894ef..080b783d608d 100644
>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>> @@ -20,7 +20,6 @@
>>>>>>>>> #include <media/media-request.h>
>>>>>>>>> #include <media/frame_vector.h>
>>>>>>>>>
>>>>>>>>> -#define VB2_MAX_FRAME (32)
>>>>>>>>> #define VB2_MAX_PLANES (8)
>>>>>>>>>
>>>>>>>>> /**
>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>> index 5a845887850b..88a7a565170e 100644
>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>> @@ -15,10 +15,6 @@
>>>>>>>>> #include <linux/videodev2.h>
>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>
>>>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>> -#endif
>>>>>>>>> -
>>>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>> #endif
>>>>>>>>> --
>>>>>>>>> 2.39.2
>>>>>>>>>
>>>>>> --
>>>>>> Hsia-Jun(Randy) Li
>>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
>>
>> --
>> Hsia-Jun(Randy) Li
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-07-28 7:24 ` Hsia-Jun Li
(?)
@ 2023-09-07 4:15 ` Tomasz Figa
-1 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-09-07 4:15 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Fri, Jul 28, 2023 at 4:25 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
>
> On 7/28/23 15:03, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
> >>
> >>
> >>
> >> On 7/28/23 14:46, Tomasz Figa wrote:
> >>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>
> >>>
> >>> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
> >>>>
> >>>>
> >>>> On 7/12/23 18:48, Tomasz Figa wrote:
> >>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>>>
> >>>>>
> >>>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
> >>>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
> >>>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>>> attachments unless you recognize the sender and know the content is
> >>>>>>> safe.
> >>>>>>>
> >>>>>>>
> >>>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> >>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>>>>> attachments unless you recognize the sender and know the content is
> >>>>>>>>> safe.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> After changing bufs arrays to a dynamic allocated array
> >>>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> >>>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> >>>>>>>> more reasonable.
> >>>>>>>>
> >>>>>>>> It would be hard to iterate the whole array, it would go worse with a
> >>>>>>>> filter. Such iterate may need to go twice because you mix
> >>>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
> >>>>>>> Here I don't want to change drivers behavior so I keep the same value.
> >>>>>>> If it happens that they need more buffers, like for dynamic resolution
> >>>>>>> change
> >>>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >>>>>>>
> >>>>>> I just don't like the idea that using a variant length array here.
> >>>>>>
> >>>>> "I don't like" is not an argument. We had a number of arguments for
> >>>>> using a generic helper (originally idr, but later decided to go with
> >>>>> XArray, because the former is now deprecated) that we pointed out in
> >>>>> our review comments for previous revisions. It wasn't really about the
> >>>>> size being variable, but rather avoiding open coding things in vb2 and
> >>>>> duplicating what's already implemented in generic code.
> >>>>
> >>>> I just want to say I don't think we need a variable length array to
> >>>> store the buffer here.
> >>>>
> >>>> And the below is the reason that such a case could be avoided in the
> >>>> first place.
> >>>>
> >>>>>
> >>>>>> And I could explain why you won't need so many buffers for the performance
> >>>>>> of decoding.
> >>>>>>
> >>>>>> VP9 could support 10 reference frames in dpb.
> >>>>>>
> >>>>>> Even for those frequent resolution changing test set, it would only happen
> >>>>>> to two resolutions,
> >>>>>>
> >>>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> >>>>>> also leaves enough slots for re-order latency.
> >>>>>>
> >>>>>> If your case had more two resolutions, likes low->medium->high.
> >>>>>>
> >>>>>> I would suggest just skip the medium resolutions, just allocate the lower
> >>>>>> one first for fast playback then the highest for all the possible
> >>>>>>
> >>>>>> medium cases. Reallocation happens frequently would only cause memory
> >>>>>> fragment, nothing benefits your performance.
> >>>>>>
> >>>>> We have mechanisms in the kernel to deal with memory fragmentation
> >>>>> (migration/compaction) and it would still only matters for the
> >>>>> pathologic cases of hardware that require physically contiguous memory.
> >>>>> Modern hardware with proper DMA capabilities can either scatter-gather
> >>>>> or are equipped with an IOMMU, so the allocation always happens in page
> >>>>> granularity and fragmentation is avoided.
> >>>>
> >>>> Unfortunately, there are more devices that didn't have a IOMMU attached
> >>>> to it, supporting scatter gather is more odd.
> >>>>
> >>>> It would be more likely that IOMMU would be disabled for the performance
> >>>> reason.
> >>>
> >>> These days IOMMU is totally mandatory if you want to think about
> >>> having any level of security in your system. Sure, there could be some
> >>> systems that are completely isolated from any external environment,
> >>> like some offline industry automation machines, but then arguably
> >>> their running conditions would also be quite static and require very
> >>> little memory re-allocation.
> >> Vendor just decided not to included such hardware.
> >> That is why From ION to DMA-heap, people like to allocate from a cavout
> >> out memory.
> >>>
> >>> I also don't buy the performance reason. CPUs have been behind MMUs
> >>> for ages and nobody is running them with paging disabled for
> >>> performance reasons. Similarly, most of the modern consumer systems
> >> Page lookup would increase the delay. Besides a few upstream devices
> >> prove them only use a level 1 page table without TBL.
> >
> > That's just an excuse for a bad hardware design/implementation. As I
> > said, there are good IOMMU implementations out there that don't suffer
> > from performance issues.
> >
> I could do nothing about that.
> Besides, even with TLB, cache missing could happen frequently,
> especially we need to access many (5~16, 10 usually) buffers and more
> 11MBytes each in a hardware processing.
> You can't have a very large TLB.
Right, but as I wrote in my previous emails, we have the right methods
in the kernel for providing drivers with contiguous memory and those
can be used for those special cases.
> >>> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
> >>> because of the security reason and they don't seem to be having any
> >> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
> >> The most security way here, we should use a dedicated memory(or a zone
> >> in unified memory).
> >
> > You still need something to enforce that the hardware is not accessing
> > memory that it's not supposed to access. How do you do that without an
> > IOMMU?
> >
> If you know the arm security pipeline and security controller, you could
> found we could reserved a range of memory for a security id(devices in
> secure world may be a different security domain).
> Besides, a MMU or security MPU could mark some pages for the secure
> world access only, it doesn't mean the device need an IOMMU to access
> them. The MPU could filter the access through the AXI id.
> >> I believe there are more users in mobile for DMA-heap than kernel's dma
> >> allocation API.
> >
> > Yes, but that's completely separate from whether there is an IOMMU or
> > not. It's just a different allocation API.
> >
> The memory heap would mean a dedicated memory usually(we don't talk
> about system heap or why there are many vendor heaps). Dedicated memory
> means contiguous memory in the most of cases.
No, and no.
First no - DMA-buf heap doesn't imply dedicated memory and usually one
wants to completely avoid carving out memory, because it becomes
useless if specific use case is not active.
Second no - there are ways to provide dedicated memory regions to the
DMA mapping API, such as shared or restricted DMA pool [1].
[1] https://elixir.bootlin.com/linux/latest/source/Documentation/devicetree/bindings/reserved-memory/shared-dma-pool.yaml
Best regards,
Tomasz
> >>> performance issues. In fact, it can improve the performance, because
> >>> memory allocation is much easier and without contiguous careouts (as
> >>> we used to have long ago on Android devices) the extra memory can be
> >>> used for buffers and caches to improve system performance.
> >>>
> >>> Best regards,
> >>> Tomasz
> >>>
> >>>>
> >>>>>
> >>>>> Best regards,
> >>>>> Tomasz
> >>>>>
> >>>>>>>>> Remove it from the core definitions but keep it for drivers internal
> >>>>>>>>> needs.
> >>>>>>>>>
> >>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>>>>>> ---
> >>>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> >>>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
> >>>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> >>>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> >>>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> >>>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> >>>>>>>>> include/media/videobuf2-core.h | 1 -
> >>>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
> >>>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
> >>>>>>>>>
> >>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> index 86e1e926fa45..899783f67580 100644
> >>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> @@ -31,6 +31,8 @@
> >>>>>>>>>
> >>>>>>>>> #include <trace/events/vb2.h>
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> static int debug;
> >>>>>>>>> module_param(debug, int, 0644);
> >>>>>>>>>
> >>>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> b/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
> >>>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> @@ -28,6 +28,7 @@
> >>>>>>>>>
> >>>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
> >>>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>>
> >>>>>>>>> struct vdec_fs_info {
> >>>>>>>>> char name[8];
> >>>>>>>>> diff --git
> >>>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
> >>>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> @@ -16,6 +16,8 @@
> >>>>>>>>> #include "../vdec_drv_if.h"
> >>>>>>>>> #include "../vdec_vpu_if.h"
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> /* reset_frame_context defined in VP9 spec */
> >>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> >>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> >>>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
> >>>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> @@ -10,6 +10,8 @@
> >>>>>>>>>
> >>>>>>>>> #include "hfi_helper.h"
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
> >>>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
> >>>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
> >>>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
> >>>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> @@ -15,6 +15,8 @@
> >>>>>>>>> #include <media/v4l2-vp9.h>
> >>>>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
> >>>>>>>>>
> >>>>>>>>> #define MB_DIM 16
> >>>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
> >>>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> @@ -10,6 +10,8 @@
> >>>>>>>>> #include "ipu3.h"
> >>>>>>>>> #include "ipu3-dmamap.h"
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> /******************** v4l2_subdev_ops ********************/
> >>>>>>>>>
> >>>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
> >>>>>>>>> diff --git a/include/media/videobuf2-core.h
> >>>>>>>>> b/include/media/videobuf2-core.h
> >>>>>>>>> index 77921cf894ef..080b783d608d 100644
> >>>>>>>>> --- a/include/media/videobuf2-core.h
> >>>>>>>>> +++ b/include/media/videobuf2-core.h
> >>>>>>>>> @@ -20,7 +20,6 @@
> >>>>>>>>> #include <media/media-request.h>
> >>>>>>>>> #include <media/frame_vector.h>
> >>>>>>>>>
> >>>>>>>>> -#define VB2_MAX_FRAME (32)
> >>>>>>>>> #define VB2_MAX_PLANES (8)
> >>>>>>>>>
> >>>>>>>>> /**
> >>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>>>>>> b/include/media/videobuf2-v4l2.h
> >>>>>>>>> index 5a845887850b..88a7a565170e 100644
> >>>>>>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>>>>>> @@ -15,10 +15,6 @@
> >>>>>>>>> #include <linux/videodev2.h>
> >>>>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>>>
> >>>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>>>> -#endif
> >>>>>>>>> -
> >>>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>>>> #endif
> >>>>>>>>> --
> >>>>>>>>> 2.39.2
> >>>>>>>>>
> >>>>>> --
> >>>>>> Hsia-Jun(Randy) Li
> >>>>>>
> >>>> --
> >>>> Hsia-Jun(Randy) Li
> >>>>
> >>
> >> --
> >> Hsia-Jun(Randy) Li
>
> --
> Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-09-07 4:15 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-09-07 4:15 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Fri, Jul 28, 2023 at 4:25 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
>
> On 7/28/23 15:03, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
> >>
> >>
> >>
> >> On 7/28/23 14:46, Tomasz Figa wrote:
> >>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>
> >>>
> >>> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
> >>>>
> >>>>
> >>>> On 7/12/23 18:48, Tomasz Figa wrote:
> >>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>>>
> >>>>>
> >>>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
> >>>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
> >>>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>>> attachments unless you recognize the sender and know the content is
> >>>>>>> safe.
> >>>>>>>
> >>>>>>>
> >>>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> >>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>>>>> attachments unless you recognize the sender and know the content is
> >>>>>>>>> safe.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> After changing bufs arrays to a dynamic allocated array
> >>>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> >>>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> >>>>>>>> more reasonable.
> >>>>>>>>
> >>>>>>>> It would be hard to iterate the whole array, it would go worse with a
> >>>>>>>> filter. Such iterate may need to go twice because you mix
> >>>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
> >>>>>>> Here I don't want to change drivers behavior so I keep the same value.
> >>>>>>> If it happens that they need more buffers, like for dynamic resolution
> >>>>>>> change
> >>>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >>>>>>>
> >>>>>> I just don't like the idea that using a variant length array here.
> >>>>>>
> >>>>> "I don't like" is not an argument. We had a number of arguments for
> >>>>> using a generic helper (originally idr, but later decided to go with
> >>>>> XArray, because the former is now deprecated) that we pointed out in
> >>>>> our review comments for previous revisions. It wasn't really about the
> >>>>> size being variable, but rather avoiding open coding things in vb2 and
> >>>>> duplicating what's already implemented in generic code.
> >>>>
> >>>> I just want to say I don't think we need a variable length array to
> >>>> store the buffer here.
> >>>>
> >>>> And the below is the reason that such a case could be avoided in the
> >>>> first place.
> >>>>
> >>>>>
> >>>>>> And I could explain why you won't need so many buffers for the performance
> >>>>>> of decoding.
> >>>>>>
> >>>>>> VP9 could support 10 reference frames in dpb.
> >>>>>>
> >>>>>> Even for those frequent resolution changing test set, it would only happen
> >>>>>> to two resolutions,
> >>>>>>
> >>>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> >>>>>> also leaves enough slots for re-order latency.
> >>>>>>
> >>>>>> If your case had more two resolutions, likes low->medium->high.
> >>>>>>
> >>>>>> I would suggest just skip the medium resolutions, just allocate the lower
> >>>>>> one first for fast playback then the highest for all the possible
> >>>>>>
> >>>>>> medium cases. Reallocation happens frequently would only cause memory
> >>>>>> fragment, nothing benefits your performance.
> >>>>>>
> >>>>> We have mechanisms in the kernel to deal with memory fragmentation
> >>>>> (migration/compaction) and it would still only matters for the
> >>>>> pathologic cases of hardware that require physically contiguous memory.
> >>>>> Modern hardware with proper DMA capabilities can either scatter-gather
> >>>>> or are equipped with an IOMMU, so the allocation always happens in page
> >>>>> granularity and fragmentation is avoided.
> >>>>
> >>>> Unfortunately, there are more devices that didn't have a IOMMU attached
> >>>> to it, supporting scatter gather is more odd.
> >>>>
> >>>> It would be more likely that IOMMU would be disabled for the performance
> >>>> reason.
> >>>
> >>> These days IOMMU is totally mandatory if you want to think about
> >>> having any level of security in your system. Sure, there could be some
> >>> systems that are completely isolated from any external environment,
> >>> like some offline industry automation machines, but then arguably
> >>> their running conditions would also be quite static and require very
> >>> little memory re-allocation.
> >> Vendor just decided not to included such hardware.
> >> That is why From ION to DMA-heap, people like to allocate from a cavout
> >> out memory.
> >>>
> >>> I also don't buy the performance reason. CPUs have been behind MMUs
> >>> for ages and nobody is running them with paging disabled for
> >>> performance reasons. Similarly, most of the modern consumer systems
> >> Page lookup would increase the delay. Besides a few upstream devices
> >> prove them only use a level 1 page table without TBL.
> >
> > That's just an excuse for a bad hardware design/implementation. As I
> > said, there are good IOMMU implementations out there that don't suffer
> > from performance issues.
> >
> I could do nothing about that.
> Besides, even with TLB, cache missing could happen frequently,
> especially we need to access many (5~16, 10 usually) buffers and more
> 11MBytes each in a hardware processing.
> You can't have a very large TLB.
Right, but as I wrote in my previous emails, we have the right methods
in the kernel for providing drivers with contiguous memory and those
can be used for those special cases.
> >>> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
> >>> because of the security reason and they don't seem to be having any
> >> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
> >> The most security way here, we should use a dedicated memory(or a zone
> >> in unified memory).
> >
> > You still need something to enforce that the hardware is not accessing
> > memory that it's not supposed to access. How do you do that without an
> > IOMMU?
> >
> If you know the arm security pipeline and security controller, you could
> found we could reserved a range of memory for a security id(devices in
> secure world may be a different security domain).
> Besides, a MMU or security MPU could mark some pages for the secure
> world access only, it doesn't mean the device need an IOMMU to access
> them. The MPU could filter the access through the AXI id.
> >> I believe there are more users in mobile for DMA-heap than kernel's dma
> >> allocation API.
> >
> > Yes, but that's completely separate from whether there is an IOMMU or
> > not. It's just a different allocation API.
> >
> The memory heap would mean a dedicated memory usually(we don't talk
> about system heap or why there are many vendor heaps). Dedicated memory
> means contiguous memory in the most of cases.
No, and no.
First no - DMA-buf heap doesn't imply dedicated memory and usually one
wants to completely avoid carving out memory, because it becomes
useless if specific use case is not active.
Second no - there are ways to provide dedicated memory regions to the
DMA mapping API, such as shared or restricted DMA pool [1].
[1] https://elixir.bootlin.com/linux/latest/source/Documentation/devicetree/bindings/reserved-memory/shared-dma-pool.yaml
Best regards,
Tomasz
> >>> performance issues. In fact, it can improve the performance, because
> >>> memory allocation is much easier and without contiguous careouts (as
> >>> we used to have long ago on Android devices) the extra memory can be
> >>> used for buffers and caches to improve system performance.
> >>>
> >>> Best regards,
> >>> Tomasz
> >>>
> >>>>
> >>>>>
> >>>>> Best regards,
> >>>>> Tomasz
> >>>>>
> >>>>>>>>> Remove it from the core definitions but keep it for drivers internal
> >>>>>>>>> needs.
> >>>>>>>>>
> >>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>>>>>> ---
> >>>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> >>>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
> >>>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> >>>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> >>>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> >>>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> >>>>>>>>> include/media/videobuf2-core.h | 1 -
> >>>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
> >>>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
> >>>>>>>>>
> >>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> index 86e1e926fa45..899783f67580 100644
> >>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> @@ -31,6 +31,8 @@
> >>>>>>>>>
> >>>>>>>>> #include <trace/events/vb2.h>
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> static int debug;
> >>>>>>>>> module_param(debug, int, 0644);
> >>>>>>>>>
> >>>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> b/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
> >>>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> @@ -28,6 +28,7 @@
> >>>>>>>>>
> >>>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
> >>>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>>
> >>>>>>>>> struct vdec_fs_info {
> >>>>>>>>> char name[8];
> >>>>>>>>> diff --git
> >>>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
> >>>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> @@ -16,6 +16,8 @@
> >>>>>>>>> #include "../vdec_drv_if.h"
> >>>>>>>>> #include "../vdec_vpu_if.h"
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> /* reset_frame_context defined in VP9 spec */
> >>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> >>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> >>>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
> >>>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> @@ -10,6 +10,8 @@
> >>>>>>>>>
> >>>>>>>>> #include "hfi_helper.h"
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
> >>>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
> >>>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
> >>>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
> >>>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> @@ -15,6 +15,8 @@
> >>>>>>>>> #include <media/v4l2-vp9.h>
> >>>>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
> >>>>>>>>>
> >>>>>>>>> #define MB_DIM 16
> >>>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
> >>>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> @@ -10,6 +10,8 @@
> >>>>>>>>> #include "ipu3.h"
> >>>>>>>>> #include "ipu3-dmamap.h"
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> /******************** v4l2_subdev_ops ********************/
> >>>>>>>>>
> >>>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
> >>>>>>>>> diff --git a/include/media/videobuf2-core.h
> >>>>>>>>> b/include/media/videobuf2-core.h
> >>>>>>>>> index 77921cf894ef..080b783d608d 100644
> >>>>>>>>> --- a/include/media/videobuf2-core.h
> >>>>>>>>> +++ b/include/media/videobuf2-core.h
> >>>>>>>>> @@ -20,7 +20,6 @@
> >>>>>>>>> #include <media/media-request.h>
> >>>>>>>>> #include <media/frame_vector.h>
> >>>>>>>>>
> >>>>>>>>> -#define VB2_MAX_FRAME (32)
> >>>>>>>>> #define VB2_MAX_PLANES (8)
> >>>>>>>>>
> >>>>>>>>> /**
> >>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>>>>>> b/include/media/videobuf2-v4l2.h
> >>>>>>>>> index 5a845887850b..88a7a565170e 100644
> >>>>>>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>>>>>> @@ -15,10 +15,6 @@
> >>>>>>>>> #include <linux/videodev2.h>
> >>>>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>>>
> >>>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>>>> -#endif
> >>>>>>>>> -
> >>>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>>>> #endif
> >>>>>>>>> --
> >>>>>>>>> 2.39.2
> >>>>>>>>>
> >>>>>> --
> >>>>>> Hsia-Jun(Randy) Li
> >>>>>>
> >>>> --
> >>>> Hsia-Jun(Randy) Li
> >>>>
> >>
> >> --
> >> Hsia-Jun(Randy) Li
>
> --
> Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-09-07 4:15 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-09-07 4:15 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
On Fri, Jul 28, 2023 at 4:25 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
>
> On 7/28/23 15:03, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
> >>
> >>
> >>
> >> On 7/28/23 14:46, Tomasz Figa wrote:
> >>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>
> >>>
> >>> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
> >>>>
> >>>>
> >>>> On 7/12/23 18:48, Tomasz Figa wrote:
> >>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >>>>>
> >>>>>
> >>>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
> >>>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
> >>>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>>> attachments unless you recognize the sender and know the content is
> >>>>>>> safe.
> >>>>>>>
> >>>>>>>
> >>>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
> >>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>>>>> attachments unless you recognize the sender and know the content is
> >>>>>>>>> safe.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> After changing bufs arrays to a dynamic allocated array
> >>>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
> >>>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
> >>>>>>>> more reasonable.
> >>>>>>>>
> >>>>>>>> It would be hard to iterate the whole array, it would go worse with a
> >>>>>>>> filter. Such iterate may need to go twice because you mix
> >>>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
> >>>>>>> Here I don't want to change drivers behavior so I keep the same value.
> >>>>>>> If it happens that they need more buffers, like for dynamic resolution
> >>>>>>> change
> >>>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
> >>>>>>>
> >>>>>> I just don't like the idea that using a variant length array here.
> >>>>>>
> >>>>> "I don't like" is not an argument. We had a number of arguments for
> >>>>> using a generic helper (originally idr, but later decided to go with
> >>>>> XArray, because the former is now deprecated) that we pointed out in
> >>>>> our review comments for previous revisions. It wasn't really about the
> >>>>> size being variable, but rather avoiding open coding things in vb2 and
> >>>>> duplicating what's already implemented in generic code.
> >>>>
> >>>> I just want to say I don't think we need a variable length array to
> >>>> store the buffer here.
> >>>>
> >>>> And the below is the reason that such a case could be avoided in the
> >>>> first place.
> >>>>
> >>>>>
> >>>>>> And I could explain why you won't need so many buffers for the performance
> >>>>>> of decoding.
> >>>>>>
> >>>>>> VP9 could support 10 reference frames in dpb.
> >>>>>>
> >>>>>> Even for those frequent resolution changing test set, it would only happen
> >>>>>> to two resolutions,
> >>>>>>
> >>>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
> >>>>>> also leaves enough slots for re-order latency.
> >>>>>>
> >>>>>> If your case had more two resolutions, likes low->medium->high.
> >>>>>>
> >>>>>> I would suggest just skip the medium resolutions, just allocate the lower
> >>>>>> one first for fast playback then the highest for all the possible
> >>>>>>
> >>>>>> medium cases. Reallocation happens frequently would only cause memory
> >>>>>> fragment, nothing benefits your performance.
> >>>>>>
> >>>>> We have mechanisms in the kernel to deal with memory fragmentation
> >>>>> (migration/compaction) and it would still only matters for the
> >>>>> pathologic cases of hardware that require physically contiguous memory.
> >>>>> Modern hardware with proper DMA capabilities can either scatter-gather
> >>>>> or are equipped with an IOMMU, so the allocation always happens in page
> >>>>> granularity and fragmentation is avoided.
> >>>>
> >>>> Unfortunately, there are more devices that didn't have a IOMMU attached
> >>>> to it, supporting scatter gather is more odd.
> >>>>
> >>>> It would be more likely that IOMMU would be disabled for the performance
> >>>> reason.
> >>>
> >>> These days IOMMU is totally mandatory if you want to think about
> >>> having any level of security in your system. Sure, there could be some
> >>> systems that are completely isolated from any external environment,
> >>> like some offline industry automation machines, but then arguably
> >>> their running conditions would also be quite static and require very
> >>> little memory re-allocation.
> >> Vendor just decided not to included such hardware.
> >> That is why From ION to DMA-heap, people like to allocate from a cavout
> >> out memory.
> >>>
> >>> I also don't buy the performance reason. CPUs have been behind MMUs
> >>> for ages and nobody is running them with paging disabled for
> >>> performance reasons. Similarly, most of the modern consumer systems
> >> Page lookup would increase the delay. Besides a few upstream devices
> >> prove them only use a level 1 page table without TBL.
> >
> > That's just an excuse for a bad hardware design/implementation. As I
> > said, there are good IOMMU implementations out there that don't suffer
> > from performance issues.
> >
> I could do nothing about that.
> Besides, even with TLB, cache missing could happen frequently,
> especially we need to access many (5~16, 10 usually) buffers and more
> 11MBytes each in a hardware processing.
> You can't have a very large TLB.
Right, but as I wrote in my previous emails, we have the right methods
in the kernel for providing drivers with contiguous memory and those
can be used for those special cases.
> >>> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
> >>> because of the security reason and they don't seem to be having any
> >> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
> >> The most security way here, we should use a dedicated memory(or a zone
> >> in unified memory).
> >
> > You still need something to enforce that the hardware is not accessing
> > memory that it's not supposed to access. How do you do that without an
> > IOMMU?
> >
> If you know the arm security pipeline and security controller, you could
> found we could reserved a range of memory for a security id(devices in
> secure world may be a different security domain).
> Besides, a MMU or security MPU could mark some pages for the secure
> world access only, it doesn't mean the device need an IOMMU to access
> them. The MPU could filter the access through the AXI id.
> >> I believe there are more users in mobile for DMA-heap than kernel's dma
> >> allocation API.
> >
> > Yes, but that's completely separate from whether there is an IOMMU or
> > not. It's just a different allocation API.
> >
> The memory heap would mean a dedicated memory usually(we don't talk
> about system heap or why there are many vendor heaps). Dedicated memory
> means contiguous memory in the most of cases.
No, and no.
First no - DMA-buf heap doesn't imply dedicated memory and usually one
wants to completely avoid carving out memory, because it becomes
useless if specific use case is not active.
Second no - there are ways to provide dedicated memory regions to the
DMA mapping API, such as shared or restricted DMA pool [1].
[1] https://elixir.bootlin.com/linux/latest/source/Documentation/devicetree/bindings/reserved-memory/shared-dma-pool.yaml
Best regards,
Tomasz
> >>> performance issues. In fact, it can improve the performance, because
> >>> memory allocation is much easier and without contiguous careouts (as
> >>> we used to have long ago on Android devices) the extra memory can be
> >>> used for buffers and caches to improve system performance.
> >>>
> >>> Best regards,
> >>> Tomasz
> >>>
> >>>>
> >>>>>
> >>>>> Best regards,
> >>>>> Tomasz
> >>>>>
> >>>>>>>>> Remove it from the core definitions but keep it for drivers internal
> >>>>>>>>> needs.
> >>>>>>>>>
> >>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>>>>>> ---
> >>>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
> >>>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
> >>>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
> >>>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
> >>>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
> >>>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
> >>>>>>>>> include/media/videobuf2-core.h | 1 -
> >>>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
> >>>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
> >>>>>>>>>
> >>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> index 86e1e926fa45..899783f67580 100644
> >>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>>>>> @@ -31,6 +31,8 @@
> >>>>>>>>>
> >>>>>>>>> #include <trace/events/vb2.h>
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> static int debug;
> >>>>>>>>> module_param(debug, int, 0644);
> >>>>>>>>>
> >>>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> b/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
> >>>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
> >>>>>>>>> @@ -28,6 +28,7 @@
> >>>>>>>>>
> >>>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
> >>>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>>
> >>>>>>>>> struct vdec_fs_info {
> >>>>>>>>> char name[8];
> >>>>>>>>> diff --git
> >>>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
> >>>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
> >>>>>>>>> @@ -16,6 +16,8 @@
> >>>>>>>>> #include "../vdec_drv_if.h"
> >>>>>>>>> #include "../vdec_vpu_if.h"
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> /* reset_frame_context defined in VP9 spec */
> >>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
> >>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
> >>>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
> >>>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
> >>>>>>>>> @@ -10,6 +10,8 @@
> >>>>>>>>>
> >>>>>>>>> #include "hfi_helper.h"
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
> >>>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
> >>>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
> >>>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
> >>>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
> >>>>>>>>> @@ -15,6 +15,8 @@
> >>>>>>>>> #include <media/v4l2-vp9.h>
> >>>>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
> >>>>>>>>>
> >>>>>>>>> #define MB_DIM 16
> >>>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
> >>>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
> >>>>>>>>> @@ -10,6 +10,8 @@
> >>>>>>>>> #include "ipu3.h"
> >>>>>>>>> #include "ipu3-dmamap.h"
> >>>>>>>>>
> >>>>>>>>> +#define VB2_MAX_FRAME 32
> >>>>>>>>> +
> >>>>>>>>> /******************** v4l2_subdev_ops ********************/
> >>>>>>>>>
> >>>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
> >>>>>>>>> diff --git a/include/media/videobuf2-core.h
> >>>>>>>>> b/include/media/videobuf2-core.h
> >>>>>>>>> index 77921cf894ef..080b783d608d 100644
> >>>>>>>>> --- a/include/media/videobuf2-core.h
> >>>>>>>>> +++ b/include/media/videobuf2-core.h
> >>>>>>>>> @@ -20,7 +20,6 @@
> >>>>>>>>> #include <media/media-request.h>
> >>>>>>>>> #include <media/frame_vector.h>
> >>>>>>>>>
> >>>>>>>>> -#define VB2_MAX_FRAME (32)
> >>>>>>>>> #define VB2_MAX_PLANES (8)
> >>>>>>>>>
> >>>>>>>>> /**
> >>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>>>>>> b/include/media/videobuf2-v4l2.h
> >>>>>>>>> index 5a845887850b..88a7a565170e 100644
> >>>>>>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>>>>>> @@ -15,10 +15,6 @@
> >>>>>>>>> #include <linux/videodev2.h>
> >>>>>>>>> #include <media/videobuf2-core.h>
> >>>>>>>>>
> >>>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
> >>>>>>>>> -#endif
> >>>>>>>>> -
> >>>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
> >>>>>>>>> #endif
> >>>>>>>>> --
> >>>>>>>>> 2.39.2
> >>>>>>>>>
> >>>>>> --
> >>>>>> Hsia-Jun(Randy) Li
> >>>>>>
> >>>> --
> >>>> Hsia-Jun(Randy) Li
> >>>>
> >>
> >> --
> >> Hsia-Jun(Randy) Li
>
> --
> Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
2023-09-07 4:15 ` Tomasz Figa
(?)
@ 2023-09-07 6:54 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-09-07 6:54 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
We are little off topic here.
On 9/7/23 12:15, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Fri, Jul 28, 2023 at 4:25 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>>
>> On 7/28/23 15:03, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>>>
>>>>
>>>>
>>>> On 7/28/23 14:46, Tomasz Figa wrote:
>>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>
>>>>>
>>>>> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>>>>>
>>>>>>
>>>>>> On 7/12/23 18:48, Tomasz Figa wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>>>>>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>>>>> safe.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> After changing bufs arrays to a dynamic allocated array
>>>>>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>>>>>>>> more reasonable.
>>>>>>>>>>
>>>>>>>>>> It would be hard to iterate the whole array, it would go worse with a
>>>>>>>>>> filter. Such iterate may need to go twice because you mix
>>>>>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>>>>>>> Here I don't want to change drivers behavior so I keep the same value.
>>>>>>>>> If it happens that they need more buffers, like for dynamic resolution
>>>>>>>>> change
>>>>>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>>>>>>>
>>>>>>>> I just don't like the idea that using a variant length array here.
>>>>>>>>
>>>>>>> "I don't like" is not an argument. We had a number of arguments for
>>>>>>> using a generic helper (originally idr, but later decided to go with
>>>>>>> XArray, because the former is now deprecated) that we pointed out in
>>>>>>> our review comments for previous revisions. It wasn't really about the
>>>>>>> size being variable, but rather avoiding open coding things in vb2 and
>>>>>>> duplicating what's already implemented in generic code.
>>>>>>
>>>>>> I just want to say I don't think we need a variable length array to
>>>>>> store the buffer here.
>>>>>>
>>>>>> And the below is the reason that such a case could be avoided in the
>>>>>> first place.
>>>>>>
>>>>>>>
>>>>>>>> And I could explain why you won't need so many buffers for the performance
>>>>>>>> of decoding.
>>>>>>>>
>>>>>>>> VP9 could support 10 reference frames in dpb.
>>>>>>>>
>>>>>>>> Even for those frequent resolution changing test set, it would only happen
>>>>>>>> to two resolutions,
>>>>>>>>
>>>>>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>>>>>>>> also leaves enough slots for re-order latency.
>>>>>>>>
>>>>>>>> If your case had more two resolutions, likes low->medium->high.
>>>>>>>>
>>>>>>>> I would suggest just skip the medium resolutions, just allocate the lower
>>>>>>>> one first for fast playback then the highest for all the possible
>>>>>>>>
>>>>>>>> medium cases. Reallocation happens frequently would only cause memory
>>>>>>>> fragment, nothing benefits your performance.
>>>>>>>>
>>>>>>> We have mechanisms in the kernel to deal with memory fragmentation
>>>>>>> (migration/compaction) and it would still only matters for the
>>>>>>> pathologic cases of hardware that require physically contiguous memory.
>>>>>>> Modern hardware with proper DMA capabilities can either scatter-gather
>>>>>>> or are equipped with an IOMMU, so the allocation always happens in page
>>>>>>> granularity and fragmentation is avoided.
>>>>>>
>>>>>> Unfortunately, there are more devices that didn't have a IOMMU attached
>>>>>> to it, supporting scatter gather is more odd.
>>>>>>
>>>>>> It would be more likely that IOMMU would be disabled for the performance
>>>>>> reason.
>>>>>
>>>>> These days IOMMU is totally mandatory if you want to think about
>>>>> having any level of security in your system. Sure, there could be some
>>>>> systems that are completely isolated from any external environment,
>>>>> like some offline industry automation machines, but then arguably
>>>>> their running conditions would also be quite static and require very
>>>>> little memory re-allocation.
>>>> Vendor just decided not to included such hardware.
>>>> That is why From ION to DMA-heap, people like to allocate from a cavout
>>>> out memory.
>>>>>
>>>>> I also don't buy the performance reason. CPUs have been behind MMUs
>>>>> for ages and nobody is running them with paging disabled for
>>>>> performance reasons. Similarly, most of the modern consumer systems
>>>> Page lookup would increase the delay. Besides a few upstream devices
>>>> prove them only use a level 1 page table without TBL.
>>>
>>> That's just an excuse for a bad hardware design/implementation. As I
>>> said, there are good IOMMU implementations out there that don't suffer
>>> from performance issues.
>>>
>> I could do nothing about that.
>> Besides, even with TLB, cache missing could happen frequently,
>> especially we need to access many (5~16, 10 usually) buffers and more
>> 11MBytes each in a hardware processing.
>> You can't have a very large TLB.
>
> Right, but as I wrote in my previous emails, we have the right methods
> in the kernel for providing drivers with contiguous memory and those
> can be used for those special cases.
>
I think we were talking about whether the IOMMU should be considered
mandatory for the future possible hardware.
About the right methods, I don't think we have a protocol for negotiate
the allocation hints between drivers or anybody.
I sent an email to dri-devel to start(continue) the discussion of that.
>>>>> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
>>>>> because of the security reason and they don't seem to be having any
>>>> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
>>>> The most security way here, we should use a dedicated memory(or a zone
>>>> in unified memory).
>>>
>>> You still need something to enforce that the hardware is not accessing
>>> memory that it's not supposed to access. How do you do that without an
>>> IOMMU?
>>>
>> If you know the arm security pipeline and security controller, you could
>> found we could reserved a range of memory for a security id(devices in
>> secure world may be a different security domain).
>> Besides, a MMU or security MPU could mark some pages for the secure
>> world access only, it doesn't mean the device need an IOMMU to access
>> them. The MPU could filter the access through the AXI id.
>>>> I believe there are more users in mobile for DMA-heap than kernel's dma
>>>> allocation API.
>>>
>>> Yes, but that's completely separate from whether there is an IOMMU or
>>> not. It's just a different allocation API.
>>>
>> The memory heap would mean a dedicated memory usually(we don't talk
>> about system heap or why there are many vendor heaps). Dedicated memory
>> means contiguous memory in the most of cases.
>
> No, and no.
> First no - DMA-buf heap doesn't imply dedicated memory and usually one
> wants to completely avoid carving out memory, because it becomes
> useless if specific use case is not active.
> Second no - there are ways to provide dedicated memory regions to the
> DMA mapping API, such as shared or restricted DMA pool [1].
>
> [1] https://elixir.bootlin.com/linux/latest/source/Documentation/devicetree/bindings/reserved-memory/shared-dma-pool.yaml
>
I think the reserved memory exits even before the ION was in kernel.
Why Android prefer to use ION and now DMA-heap, I thinik it is a
google's problem.
Youl can say what reserved memory didn't do is allowing allocate from
the userspace. That is how gralloc in Android works.
Allocation decided by the users could be a important feature, for
example, a video codec could decode either secure or non secure buffer.
We could make the driver only allocate the non secure memory while the
usersapce could feed the secure memory, that could satisfy both generic
application and DRM video playback.
> Best regards,
> Tomasz
>
>>>>> performance issues. In fact, it can improve the performance, because
>>>>> memory allocation is much easier and without contiguous careouts (as
>>>>> we used to have long ago on Android devices) the extra memory can be
>>>>> used for buffers and caches to improve system performance.
>>>>>
>>>>> Best regards,
>>>>> Tomasz
>>>>>
>>>>>>
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Tomasz
>>>>>>>
>>>>>>>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>>>>>>>> needs.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>>>>> ---
>>>>>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>>>>>>>> include/media/videobuf2-core.h | 1 -
>>>>>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> index 86e1e926fa45..899783f67580 100644
>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> @@ -31,6 +31,8 @@
>>>>>>>>>>>
>>>>>>>>>>> #include <trace/events/vb2.h>
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> static int debug;
>>>>>>>>>>> module_param(debug, int, 0644);
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> @@ -28,6 +28,7 @@
>>>>>>>>>>>
>>>>>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>>
>>>>>>>>>>> struct vdec_fs_info {
>>>>>>>>>>> char name[8];
>>>>>>>>>>> diff --git
>>>>>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> @@ -16,6 +16,8 @@
>>>>>>>>>>> #include "../vdec_drv_if.h"
>>>>>>>>>>> #include "../vdec_vpu_if.h"
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> /* reset_frame_context defined in VP9 spec */
>>>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>>>>
>>>>>>>>>>> #include "hfi_helper.h"
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> @@ -15,6 +15,8 @@
>>>>>>>>>>> #include <media/v4l2-vp9.h>
>>>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>>>>>>>
>>>>>>>>>>> #define MB_DIM 16
>>>>>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>>>> #include "ipu3.h"
>>>>>>>>>>> #include "ipu3-dmamap.h"
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>>>>>>>
>>>>>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>>> index 77921cf894ef..080b783d608d 100644
>>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>>> @@ -20,7 +20,6 @@
>>>>>>>>>>> #include <media/media-request.h>
>>>>>>>>>>> #include <media/frame_vector.h>
>>>>>>>>>>>
>>>>>>>>>>> -#define VB2_MAX_FRAME (32)
>>>>>>>>>>> #define VB2_MAX_PLANES (8)
>>>>>>>>>>>
>>>>>>>>>>> /**
>>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> index 5a845887850b..88a7a565170e 100644
>>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> @@ -15,10 +15,6 @@
>>>>>>>>>>> #include <linux/videodev2.h>
>>>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>>>
>>>>>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>>>> -#endif
>>>>>>>>>>> -
>>>>>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>>>> #endif
>>>>>>>>>>> --
>>>>>>>>>>> 2.39.2
>>>>>>>>>>>
>>>>>>>> --
>>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>>
>>>>>> --
>>>>>> Hsia-Jun(Randy) Li
>>>>>>
>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>
>> --
>> Hsia-Jun(Randy) Li
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-09-07 6:54 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-09-07 6:54 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
We are little off topic here.
On 9/7/23 12:15, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Fri, Jul 28, 2023 at 4:25 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>>
>> On 7/28/23 15:03, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>>>
>>>>
>>>>
>>>> On 7/28/23 14:46, Tomasz Figa wrote:
>>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>
>>>>>
>>>>> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>>>>>
>>>>>>
>>>>>> On 7/12/23 18:48, Tomasz Figa wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>>>>>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>>>>> safe.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> After changing bufs arrays to a dynamic allocated array
>>>>>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>>>>>>>> more reasonable.
>>>>>>>>>>
>>>>>>>>>> It would be hard to iterate the whole array, it would go worse with a
>>>>>>>>>> filter. Such iterate may need to go twice because you mix
>>>>>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>>>>>>> Here I don't want to change drivers behavior so I keep the same value.
>>>>>>>>> If it happens that they need more buffers, like for dynamic resolution
>>>>>>>>> change
>>>>>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>>>>>>>
>>>>>>>> I just don't like the idea that using a variant length array here.
>>>>>>>>
>>>>>>> "I don't like" is not an argument. We had a number of arguments for
>>>>>>> using a generic helper (originally idr, but later decided to go with
>>>>>>> XArray, because the former is now deprecated) that we pointed out in
>>>>>>> our review comments for previous revisions. It wasn't really about the
>>>>>>> size being variable, but rather avoiding open coding things in vb2 and
>>>>>>> duplicating what's already implemented in generic code.
>>>>>>
>>>>>> I just want to say I don't think we need a variable length array to
>>>>>> store the buffer here.
>>>>>>
>>>>>> And the below is the reason that such a case could be avoided in the
>>>>>> first place.
>>>>>>
>>>>>>>
>>>>>>>> And I could explain why you won't need so many buffers for the performance
>>>>>>>> of decoding.
>>>>>>>>
>>>>>>>> VP9 could support 10 reference frames in dpb.
>>>>>>>>
>>>>>>>> Even for those frequent resolution changing test set, it would only happen
>>>>>>>> to two resolutions,
>>>>>>>>
>>>>>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>>>>>>>> also leaves enough slots for re-order latency.
>>>>>>>>
>>>>>>>> If your case had more two resolutions, likes low->medium->high.
>>>>>>>>
>>>>>>>> I would suggest just skip the medium resolutions, just allocate the lower
>>>>>>>> one first for fast playback then the highest for all the possible
>>>>>>>>
>>>>>>>> medium cases. Reallocation happens frequently would only cause memory
>>>>>>>> fragment, nothing benefits your performance.
>>>>>>>>
>>>>>>> We have mechanisms in the kernel to deal with memory fragmentation
>>>>>>> (migration/compaction) and it would still only matters for the
>>>>>>> pathologic cases of hardware that require physically contiguous memory.
>>>>>>> Modern hardware with proper DMA capabilities can either scatter-gather
>>>>>>> or are equipped with an IOMMU, so the allocation always happens in page
>>>>>>> granularity and fragmentation is avoided.
>>>>>>
>>>>>> Unfortunately, there are more devices that didn't have a IOMMU attached
>>>>>> to it, supporting scatter gather is more odd.
>>>>>>
>>>>>> It would be more likely that IOMMU would be disabled for the performance
>>>>>> reason.
>>>>>
>>>>> These days IOMMU is totally mandatory if you want to think about
>>>>> having any level of security in your system. Sure, there could be some
>>>>> systems that are completely isolated from any external environment,
>>>>> like some offline industry automation machines, but then arguably
>>>>> their running conditions would also be quite static and require very
>>>>> little memory re-allocation.
>>>> Vendor just decided not to included such hardware.
>>>> That is why From ION to DMA-heap, people like to allocate from a cavout
>>>> out memory.
>>>>>
>>>>> I also don't buy the performance reason. CPUs have been behind MMUs
>>>>> for ages and nobody is running them with paging disabled for
>>>>> performance reasons. Similarly, most of the modern consumer systems
>>>> Page lookup would increase the delay. Besides a few upstream devices
>>>> prove them only use a level 1 page table without TBL.
>>>
>>> That's just an excuse for a bad hardware design/implementation. As I
>>> said, there are good IOMMU implementations out there that don't suffer
>>> from performance issues.
>>>
>> I could do nothing about that.
>> Besides, even with TLB, cache missing could happen frequently,
>> especially we need to access many (5~16, 10 usually) buffers and more
>> 11MBytes each in a hardware processing.
>> You can't have a very large TLB.
>
> Right, but as I wrote in my previous emails, we have the right methods
> in the kernel for providing drivers with contiguous memory and those
> can be used for those special cases.
>
I think we were talking about whether the IOMMU should be considered
mandatory for the future possible hardware.
About the right methods, I don't think we have a protocol for negotiate
the allocation hints between drivers or anybody.
I sent an email to dri-devel to start(continue) the discussion of that.
>>>>> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
>>>>> because of the security reason and they don't seem to be having any
>>>> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
>>>> The most security way here, we should use a dedicated memory(or a zone
>>>> in unified memory).
>>>
>>> You still need something to enforce that the hardware is not accessing
>>> memory that it's not supposed to access. How do you do that without an
>>> IOMMU?
>>>
>> If you know the arm security pipeline and security controller, you could
>> found we could reserved a range of memory for a security id(devices in
>> secure world may be a different security domain).
>> Besides, a MMU or security MPU could mark some pages for the secure
>> world access only, it doesn't mean the device need an IOMMU to access
>> them. The MPU could filter the access through the AXI id.
>>>> I believe there are more users in mobile for DMA-heap than kernel's dma
>>>> allocation API.
>>>
>>> Yes, but that's completely separate from whether there is an IOMMU or
>>> not. It's just a different allocation API.
>>>
>> The memory heap would mean a dedicated memory usually(we don't talk
>> about system heap or why there are many vendor heaps). Dedicated memory
>> means contiguous memory in the most of cases.
>
> No, and no.
> First no - DMA-buf heap doesn't imply dedicated memory and usually one
> wants to completely avoid carving out memory, because it becomes
> useless if specific use case is not active.
> Second no - there are ways to provide dedicated memory regions to the
> DMA mapping API, such as shared or restricted DMA pool [1].
>
> [1] https://elixir.bootlin.com/linux/latest/source/Documentation/devicetree/bindings/reserved-memory/shared-dma-pool.yaml
>
I think the reserved memory exits even before the ION was in kernel.
Why Android prefer to use ION and now DMA-heap, I thinik it is a
google's problem.
Youl can say what reserved memory didn't do is allowing allocate from
the userspace. That is how gralloc in Android works.
Allocation decided by the users could be a important feature, for
example, a video codec could decode either secure or non secure buffer.
We could make the driver only allocate the non secure memory while the
usersapce could feed the secure memory, that could satisfy both generic
application and DRM video playback.
> Best regards,
> Tomasz
>
>>>>> performance issues. In fact, it can improve the performance, because
>>>>> memory allocation is much easier and without contiguous careouts (as
>>>>> we used to have long ago on Android devices) the extra memory can be
>>>>> used for buffers and caches to improve system performance.
>>>>>
>>>>> Best regards,
>>>>> Tomasz
>>>>>
>>>>>>
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Tomasz
>>>>>>>
>>>>>>>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>>>>>>>> needs.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>>>>> ---
>>>>>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>>>>>>>> include/media/videobuf2-core.h | 1 -
>>>>>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> index 86e1e926fa45..899783f67580 100644
>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> @@ -31,6 +31,8 @@
>>>>>>>>>>>
>>>>>>>>>>> #include <trace/events/vb2.h>
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> static int debug;
>>>>>>>>>>> module_param(debug, int, 0644);
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> @@ -28,6 +28,7 @@
>>>>>>>>>>>
>>>>>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>>
>>>>>>>>>>> struct vdec_fs_info {
>>>>>>>>>>> char name[8];
>>>>>>>>>>> diff --git
>>>>>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> @@ -16,6 +16,8 @@
>>>>>>>>>>> #include "../vdec_drv_if.h"
>>>>>>>>>>> #include "../vdec_vpu_if.h"
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> /* reset_frame_context defined in VP9 spec */
>>>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>>>>
>>>>>>>>>>> #include "hfi_helper.h"
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> @@ -15,6 +15,8 @@
>>>>>>>>>>> #include <media/v4l2-vp9.h>
>>>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>>>>>>>
>>>>>>>>>>> #define MB_DIM 16
>>>>>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>>>> #include "ipu3.h"
>>>>>>>>>>> #include "ipu3-dmamap.h"
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>>>>>>>
>>>>>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>>> index 77921cf894ef..080b783d608d 100644
>>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>>> @@ -20,7 +20,6 @@
>>>>>>>>>>> #include <media/media-request.h>
>>>>>>>>>>> #include <media/frame_vector.h>
>>>>>>>>>>>
>>>>>>>>>>> -#define VB2_MAX_FRAME (32)
>>>>>>>>>>> #define VB2_MAX_PLANES (8)
>>>>>>>>>>>
>>>>>>>>>>> /**
>>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> index 5a845887850b..88a7a565170e 100644
>>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> @@ -15,10 +15,6 @@
>>>>>>>>>>> #include <linux/videodev2.h>
>>>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>>>
>>>>>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>>>> -#endif
>>>>>>>>>>> -
>>>>>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>>>> #endif
>>>>>>>>>>> --
>>>>>>>>>>> 2.39.2
>>>>>>>>>>>
>>>>>>>> --
>>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>>
>>>>>> --
>>>>>> Hsia-Jun(Randy) Li
>>>>>>
>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>
>> --
>> Hsia-Jun(Randy) Li
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 04/11] media: videobuf2: Stop define VB2_MAX_FRAME as global
@ 2023-09-07 6:54 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-09-07 6:54 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, linux-arm-kernel,
m.szyprowski, linux-mediatek, linux-arm-msm, hverkuil-cisco,
ezequiel, p.zabel, linux-rockchip, mchehab, linux-staging,
ming.qian, kernel, gregkh, nicolas.dufresne
We are little off topic here.
On 9/7/23 12:15, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Fri, Jul 28, 2023 at 4:25 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>>
>> On 7/28/23 15:03, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Fri, Jul 28, 2023 at 3:55 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>>>
>>>>
>>>>
>>>> On 7/28/23 14:46, Tomasz Figa wrote:
>>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>
>>>>>
>>>>> On Mon, Jul 17, 2023 at 4:44 PM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>>>>>
>>>>>>
>>>>>> On 7/12/23 18:48, Tomasz Figa wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 03, 2023 at 04:35:30PM +0800, Hsia-Jun Li wrote:
>>>>>>>> On 7/3/23 16:09, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Le 30/06/2023 à 11:51, Hsia-Jun Li a écrit :
>>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>>>>> safe.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> After changing bufs arrays to a dynamic allocated array
>>>>>>>>>>> VB2_MAX_FRAME doesn't mean anything for videobuf2 core.
>>>>>>>>>> I think make it 64 which is the VB2_MAX_FRAME in Android GKI kernel is
>>>>>>>>>> more reasonable.
>>>>>>>>>>
>>>>>>>>>> It would be hard to iterate the whole array, it would go worse with a
>>>>>>>>>> filter. Such iterate may need to go twice because you mix
>>>>>>>>>> post-processing buffer and decoding buffer(with MV) in the same array.
>>>>>>>>> Here I don't want to change drivers behavior so I keep the same value.
>>>>>>>>> If it happens that they need more buffers, like for dynamic resolution
>>>>>>>>> change
>>>>>>>>> feature for Verisilicon VP9 decoder, case by case patches will be needed.
>>>>>>>>>
>>>>>>>> I just don't like the idea that using a variant length array here.
>>>>>>>>
>>>>>>> "I don't like" is not an argument. We had a number of arguments for
>>>>>>> using a generic helper (originally idr, but later decided to go with
>>>>>>> XArray, because the former is now deprecated) that we pointed out in
>>>>>>> our review comments for previous revisions. It wasn't really about the
>>>>>>> size being variable, but rather avoiding open coding things in vb2 and
>>>>>>> duplicating what's already implemented in generic code.
>>>>>>
>>>>>> I just want to say I don't think we need a variable length array to
>>>>>> store the buffer here.
>>>>>>
>>>>>> And the below is the reason that such a case could be avoided in the
>>>>>> first place.
>>>>>>
>>>>>>>
>>>>>>>> And I could explain why you won't need so many buffers for the performance
>>>>>>>> of decoding.
>>>>>>>>
>>>>>>>> VP9 could support 10 reference frames in dpb.
>>>>>>>>
>>>>>>>> Even for those frequent resolution changing test set, it would only happen
>>>>>>>> to two resolutions,
>>>>>>>>
>>>>>>>> 32 would be enough for 20 buffers of two resolution plus golden frames. It
>>>>>>>> also leaves enough slots for re-order latency.
>>>>>>>>
>>>>>>>> If your case had more two resolutions, likes low->medium->high.
>>>>>>>>
>>>>>>>> I would suggest just skip the medium resolutions, just allocate the lower
>>>>>>>> one first for fast playback then the highest for all the possible
>>>>>>>>
>>>>>>>> medium cases. Reallocation happens frequently would only cause memory
>>>>>>>> fragment, nothing benefits your performance.
>>>>>>>>
>>>>>>> We have mechanisms in the kernel to deal with memory fragmentation
>>>>>>> (migration/compaction) and it would still only matters for the
>>>>>>> pathologic cases of hardware that require physically contiguous memory.
>>>>>>> Modern hardware with proper DMA capabilities can either scatter-gather
>>>>>>> or are equipped with an IOMMU, so the allocation always happens in page
>>>>>>> granularity and fragmentation is avoided.
>>>>>>
>>>>>> Unfortunately, there are more devices that didn't have a IOMMU attached
>>>>>> to it, supporting scatter gather is more odd.
>>>>>>
>>>>>> It would be more likely that IOMMU would be disabled for the performance
>>>>>> reason.
>>>>>
>>>>> These days IOMMU is totally mandatory if you want to think about
>>>>> having any level of security in your system. Sure, there could be some
>>>>> systems that are completely isolated from any external environment,
>>>>> like some offline industry automation machines, but then arguably
>>>>> their running conditions would also be quite static and require very
>>>>> little memory re-allocation.
>>>> Vendor just decided not to included such hardware.
>>>> That is why From ION to DMA-heap, people like to allocate from a cavout
>>>> out memory.
>>>>>
>>>>> I also don't buy the performance reason. CPUs have been behind MMUs
>>>>> for ages and nobody is running them with paging disabled for
>>>>> performance reasons. Similarly, most of the modern consumer systems
>>>> Page lookup would increase the delay. Besides a few upstream devices
>>>> prove them only use a level 1 page table without TBL.
>>>
>>> That's just an excuse for a bad hardware design/implementation. As I
>>> said, there are good IOMMU implementations out there that don't suffer
>>> from performance issues.
>>>
>> I could do nothing about that.
>> Besides, even with TLB, cache missing could happen frequently,
>> especially we need to access many (5~16, 10 usually) buffers and more
>> 11MBytes each in a hardware processing.
>> You can't have a very large TLB.
>
> Right, but as I wrote in my previous emails, we have the right methods
> in the kernel for providing drivers with contiguous memory and those
> can be used for those special cases.
>
I think we were talking about whether the IOMMU should be considered
mandatory for the future possible hardware.
About the right methods, I don't think we have a protocol for negotiate
the allocation hints between drivers or anybody.
I sent an email to dri-devel to start(continue) the discussion of that.
>>>>> (mobile phones, PCs) run with IOMMUs enabled for pretty much anything
>>>>> because of the security reason and they don't seem to be having any
>>>> If the page is secure, you can't operate it in a insecure IOMMU or MMU.
>>>> The most security way here, we should use a dedicated memory(or a zone
>>>> in unified memory).
>>>
>>> You still need something to enforce that the hardware is not accessing
>>> memory that it's not supposed to access. How do you do that without an
>>> IOMMU?
>>>
>> If you know the arm security pipeline and security controller, you could
>> found we could reserved a range of memory for a security id(devices in
>> secure world may be a different security domain).
>> Besides, a MMU or security MPU could mark some pages for the secure
>> world access only, it doesn't mean the device need an IOMMU to access
>> them. The MPU could filter the access through the AXI id.
>>>> I believe there are more users in mobile for DMA-heap than kernel's dma
>>>> allocation API.
>>>
>>> Yes, but that's completely separate from whether there is an IOMMU or
>>> not. It's just a different allocation API.
>>>
>> The memory heap would mean a dedicated memory usually(we don't talk
>> about system heap or why there are many vendor heaps). Dedicated memory
>> means contiguous memory in the most of cases.
>
> No, and no.
> First no - DMA-buf heap doesn't imply dedicated memory and usually one
> wants to completely avoid carving out memory, because it becomes
> useless if specific use case is not active.
> Second no - there are ways to provide dedicated memory regions to the
> DMA mapping API, such as shared or restricted DMA pool [1].
>
> [1] https://elixir.bootlin.com/linux/latest/source/Documentation/devicetree/bindings/reserved-memory/shared-dma-pool.yaml
>
I think the reserved memory exits even before the ION was in kernel.
Why Android prefer to use ION and now DMA-heap, I thinik it is a
google's problem.
Youl can say what reserved memory didn't do is allowing allocate from
the userspace. That is how gralloc in Android works.
Allocation decided by the users could be a important feature, for
example, a video codec could decode either secure or non secure buffer.
We could make the driver only allocate the non secure memory while the
usersapce could feed the secure memory, that could satisfy both generic
application and DRM video playback.
> Best regards,
> Tomasz
>
>>>>> performance issues. In fact, it can improve the performance, because
>>>>> memory allocation is much easier and without contiguous careouts (as
>>>>> we used to have long ago on Android devices) the extra memory can be
>>>>> used for buffers and caches to improve system performance.
>>>>>
>>>>> Best regards,
>>>>> Tomasz
>>>>>
>>>>>>
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Tomasz
>>>>>>>
>>>>>>>>>>> Remove it from the core definitions but keep it for drivers internal
>>>>>>>>>>> needs.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>>>>> ---
>>>>>>>>>>> drivers/media/common/videobuf2/videobuf2-core.c | 2 ++
>>>>>>>>>>> drivers/media/platform/amphion/vdec.c | 1 +
>>>>>>>>>>> .../media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c | 2 ++
>>>>>>>>>>> drivers/media/platform/qcom/venus/hfi.h | 2 ++
>>>>>>>>>>> drivers/media/platform/verisilicon/hantro_hw.h | 2 ++
>>>>>>>>>>> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 ++
>>>>>>>>>>> include/media/videobuf2-core.h | 1 -
>>>>>>>>>>> include/media/videobuf2-v4l2.h | 4 ----
>>>>>>>>>>> 8 files changed, 11 insertions(+), 5 deletions(-)
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> index 86e1e926fa45..899783f67580 100644
>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> @@ -31,6 +31,8 @@
>>>>>>>>>>>
>>>>>>>>>>> #include <trace/events/vb2.h>
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> static int debug;
>>>>>>>>>>> module_param(debug, int, 0644);
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> b/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> index 3fa1a74a2e20..b3219f6d17fa 100644
>>>>>>>>>>> --- a/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> +++ b/drivers/media/platform/amphion/vdec.c
>>>>>>>>>>> @@ -28,6 +28,7 @@
>>>>>>>>>>>
>>>>>>>>>>> #define VDEC_MIN_BUFFER_CAP 8
>>>>>>>>>>> #define VDEC_MIN_BUFFER_OUT 8
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>>
>>>>>>>>>>> struct vdec_fs_info {
>>>>>>>>>>> char name[8];
>>>>>>>>>>> diff --git
>>>>>>>>>>> a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> index 6532a69f1fa8..a1e0f24bb91c 100644
>>>>>>>>>>> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_vp9_req_lat_if.c
>>>>>>>>>>> @@ -16,6 +16,8 @@
>>>>>>>>>>> #include "../vdec_drv_if.h"
>>>>>>>>>>> #include "../vdec_vpu_if.h"
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> /* reset_frame_context defined in VP9 spec */
>>>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE0 0
>>>>>>>>>>> #define VP9_RESET_FRAME_CONTEXT_NONE1 1
>>>>>>>>>>> diff --git a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> index f25d412d6553..bd5ca5a8b945 100644
>>>>>>>>>>> --- a/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> +++ b/drivers/media/platform/qcom/venus/hfi.h
>>>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>>>>
>>>>>>>>>>> #include "hfi_helper.h"
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> #define VIDC_SESSION_TYPE_VPE 0
>>>>>>>>>>> #define VIDC_SESSION_TYPE_ENC 1
>>>>>>>>>>> #define VIDC_SESSION_TYPE_DEC 2
>>>>>>>>>>> diff --git a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> index e83f0c523a30..9e8faf7ba6fb 100644
>>>>>>>>>>> --- a/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> +++ b/drivers/media/platform/verisilicon/hantro_hw.h
>>>>>>>>>>> @@ -15,6 +15,8 @@
>>>>>>>>>>> #include <media/v4l2-vp9.h>
>>>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> #define DEC_8190_ALIGN_MASK 0x07U
>>>>>>>>>>>
>>>>>>>>>>> #define MB_DIM 16
>>>>>>>>>>> diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> index e530767e80a5..6627b5c2d4d6 100644
>>>>>>>>>>> --- a/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> +++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
>>>>>>>>>>> @@ -10,6 +10,8 @@
>>>>>>>>>>> #include "ipu3.h"
>>>>>>>>>>> #include "ipu3-dmamap.h"
>>>>>>>>>>>
>>>>>>>>>>> +#define VB2_MAX_FRAME 32
>>>>>>>>>>> +
>>>>>>>>>>> /******************** v4l2_subdev_ops ********************/
>>>>>>>>>>>
>>>>>>>>>>> #define IPU3_RUNNING_MODE_VIDEO 0
>>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>>> index 77921cf894ef..080b783d608d 100644
>>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>>> @@ -20,7 +20,6 @@
>>>>>>>>>>> #include <media/media-request.h>
>>>>>>>>>>> #include <media/frame_vector.h>
>>>>>>>>>>>
>>>>>>>>>>> -#define VB2_MAX_FRAME (32)
>>>>>>>>>>> #define VB2_MAX_PLANES (8)
>>>>>>>>>>>
>>>>>>>>>>> /**
>>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> index 5a845887850b..88a7a565170e 100644
>>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> @@ -15,10 +15,6 @@
>>>>>>>>>>> #include <linux/videodev2.h>
>>>>>>>>>>> #include <media/videobuf2-core.h>
>>>>>>>>>>>
>>>>>>>>>>> -#if VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>>>> -#error VB2_MAX_FRAME != VIDEO_MAX_FRAME
>>>>>>>>>>> -#endif
>>>>>>>>>>> -
>>>>>>>>>>> #if VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>>>> #error VB2_MAX_PLANES != VIDEO_MAX_PLANES
>>>>>>>>>>> #endif
>>>>>>>>>>> --
>>>>>>>>>>> 2.39.2
>>>>>>>>>>>
>>>>>>>> --
>>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>>
>>>>>> --
>>>>>> Hsia-Jun(Randy) Li
>>>>>>
>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>
>> --
>> Hsia-Jun(Randy) Li
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread
* [PATCH v3 05/11] media: verisilicon: Refactor postprocessor to store more buffers
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Like for vb queues remove VB2_MAX_FRAME upper limit on postprocessor
buffers storage capacity.
If a buffer hasn't been allocated when setting up the queue make
possible to alloc it on the fly while streaming. This happens
when resolution change between keyframes.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro.h | 6 +-
.../media/platform/verisilicon/hantro_hw.h | 6 +-
.../platform/verisilicon/hantro_postproc.c | 109 ++++++++++++------
.../media/platform/verisilicon/hantro_v4l2.c | 12 +-
4 files changed, 82 insertions(+), 51 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro.h b/drivers/media/platform/verisilicon/hantro.h
index 2989ebc631cc..bd0dca11b90a 100644
--- a/drivers/media/platform/verisilicon/hantro.h
+++ b/drivers/media/platform/verisilicon/hantro.h
@@ -460,12 +460,14 @@ hantro_get_dst_buf(struct hantro_ctx *ctx)
bool hantro_needs_postproc(const struct hantro_ctx *ctx,
const struct hantro_fmt *fmt);
+dma_addr_t
+hantro_postproc_get_dec_buf_addr(struct hantro_ctx *ctx, int index);
static inline dma_addr_t
hantro_get_dec_buf_addr(struct hantro_ctx *ctx, struct vb2_buffer *vb)
{
if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt))
- return ctx->postproc.dec_q[vb->index].dma;
+ return hantro_postproc_get_dec_buf_addr(ctx, vb->index);
return vb2_dma_contig_plane_dma_addr(vb, 0);
}
@@ -477,8 +479,8 @@ vb2_to_hantro_decoded_buf(struct vb2_buffer *buf)
void hantro_postproc_disable(struct hantro_ctx *ctx);
void hantro_postproc_enable(struct hantro_ctx *ctx);
+int hantro_postproc_init(struct hantro_ctx *ctx);
void hantro_postproc_free(struct hantro_ctx *ctx);
-int hantro_postproc_alloc(struct hantro_ctx *ctx);
int hanto_postproc_enum_framesizes(struct hantro_ctx *ctx,
struct v4l2_frmsizeenum *fsize);
diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
index 9e8faf7ba6fb..5c7b6e04aded 100644
--- a/drivers/media/platform/verisilicon/hantro_hw.h
+++ b/drivers/media/platform/verisilicon/hantro_hw.h
@@ -15,8 +15,6 @@
#include <media/v4l2-vp9.h>
#include <media/videobuf2-core.h>
-#define VB2_MAX_FRAME 32
-
#define DEC_8190_ALIGN_MASK 0x07U
#define MB_DIM 16
@@ -252,10 +250,10 @@ struct hantro_vp9_dec_hw_ctx {
/**
* struct hantro_postproc_ctx
*
- * @dec_q: References buffers, in decoder format.
+ * @dec_refs_xa: References buffers, in decoder format.
*/
struct hantro_postproc_ctx {
- struct hantro_aux_buf dec_q[VB2_MAX_FRAME];
+ struct xarray dec_refs_xa;
};
/**
diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
index 6437423ccf3a..a6b67d0cc66c 100644
--- a/drivers/media/platform/verisilicon/hantro_postproc.c
+++ b/drivers/media/platform/verisilicon/hantro_postproc.c
@@ -170,36 +170,23 @@ static int hantro_postproc_g2_enum_framesizes(struct hantro_ctx *ctx,
return 0;
}
-void hantro_postproc_free(struct hantro_ctx *ctx)
+static int hantro_postproc_alloc(struct hantro_ctx *ctx, int index)
{
struct hantro_dev *vpu = ctx->dev;
- unsigned int i;
-
- for (i = 0; i < VB2_MAX_FRAME; ++i) {
- struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
-
- if (priv->cpu) {
- dma_free_attrs(vpu->dev, priv->size, priv->cpu,
- priv->dma, priv->attrs);
- priv->cpu = NULL;
- }
- }
-}
-
-int hantro_postproc_alloc(struct hantro_ctx *ctx)
-{
- struct hantro_dev *vpu = ctx->dev;
- struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
- struct vb2_queue *cap_queue = &m2m_ctx->cap_q_ctx.q;
- unsigned int num_buffers = cap_queue->num_buffers;
struct v4l2_pix_format_mplane pix_mp;
const struct hantro_fmt *fmt;
- unsigned int i, buf_size;
+ struct hantro_aux_buf *priv;
+ unsigned int buf_size;
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
/* this should always pick native format */
fmt = hantro_get_default_fmt(ctx, false, ctx->bit_depth);
if (!fmt)
return -EINVAL;
+
v4l2_fill_pixfmt_mp(&pix_mp, fmt->fourcc, ctx->src_fmt.width,
ctx->src_fmt.height);
@@ -214,20 +201,74 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
buf_size += hantro_hevc_mv_size(pix_mp.width,
pix_mp.height);
- for (i = 0; i < num_buffers; ++i) {
- struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
-
- /*
- * The buffers on this queue are meant as intermediate
- * buffers for the decoder, so no mapping is needed.
- */
- priv->attrs = DMA_ATTR_NO_KERNEL_MAPPING;
- priv->cpu = dma_alloc_attrs(vpu->dev, buf_size, &priv->dma,
- GFP_KERNEL, priv->attrs);
- if (!priv->cpu)
- return -ENOMEM;
- priv->size = buf_size;
+ /*
+ * The buffers on this queue are meant as intermediate
+ * buffers for the decoder, so no mapping is needed.
+ */
+ priv->attrs = DMA_ATTR_NO_KERNEL_MAPPING;
+ priv->cpu = dma_alloc_attrs(vpu->dev, buf_size, &priv->dma, GFP_KERNEL, priv->attrs);
+ if (!priv->cpu)
+ return -ENOMEM;
+
+ priv->size = buf_size;
+
+ xa_store(&ctx->postproc.dec_refs_xa, index, priv, GFP_KERNEL);
+
+ return 0;
+}
+
+void hantro_postproc_free(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ struct hantro_aux_buf *priv;
+ unsigned long i;
+
+ xa_for_each(&ctx->postproc.dec_refs_xa, i, priv) {
+ dma_free_attrs(vpu->dev, priv->size, priv->cpu, priv->dma, priv->attrs);
+ kfree(priv);
+ }
+
+ xa_destroy(&ctx->postproc.dec_refs_xa);
+}
+
+int hantro_postproc_init(struct hantro_ctx *ctx)
+{
+ struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+ struct vb2_queue *cap_queue = &m2m_ctx->cap_q_ctx.q;
+ unsigned int num_buffers = cap_queue->num_buffers;
+ int i, ret = 0;
+
+ xa_init_flags(&ctx->postproc.dec_refs_xa, XA_FLAGS_ALLOC);
+
+ for (i = 0; i < num_buffers; i++) {
+ ret = hantro_postproc_alloc(ctx, i);
+ if (ret < 0) {
+ hantro_postproc_free(ctx);
+ return ret;
+ }
}
+
+ return ret;
+}
+
+dma_addr_t
+hantro_postproc_get_dec_buf_addr(struct hantro_ctx *ctx, int index)
+{
+ struct hantro_aux_buf *priv = xa_load(&ctx->postproc.dec_refs_xa, index);
+ int ret;
+
+ if (priv)
+ return priv->dma;
+
+ /* buffer not already allocated, try getting a new one */
+ ret = hantro_postproc_alloc(ctx, index);
+ if (ret < 0)
+ return 0;
+
+ priv = xa_load(&ctx->postproc.dec_refs_xa, index);
+ if (priv)
+ return priv->dma;
+
return 0;
}
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 61cfaaf4e927..0f615c1001c0 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -894,20 +894,10 @@ static int hantro_start_streaming(struct vb2_queue *q, unsigned int count)
ret = ctx->codec_ops->init(ctx);
if (ret)
return ret;
- }
-
- if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt)) {
- ret = hantro_postproc_alloc(ctx);
- if (ret)
- goto err_codec_exit;
+ ret = hantro_postproc_init(ctx);
}
}
return ret;
-
-err_codec_exit:
- if (ctx->codec_ops->exit)
- ctx->codec_ops->exit(ctx);
- return ret;
}
static void
--
2.39.2
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 05/11] media: verisilicon: Refactor postprocessor to store more buffers
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Like for vb queues remove VB2_MAX_FRAME upper limit on postprocessor
buffers storage capacity.
If a buffer hasn't been allocated when setting up the queue make
possible to alloc it on the fly while streaming. This happens
when resolution change between keyframes.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro.h | 6 +-
.../media/platform/verisilicon/hantro_hw.h | 6 +-
.../platform/verisilicon/hantro_postproc.c | 109 ++++++++++++------
.../media/platform/verisilicon/hantro_v4l2.c | 12 +-
4 files changed, 82 insertions(+), 51 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro.h b/drivers/media/platform/verisilicon/hantro.h
index 2989ebc631cc..bd0dca11b90a 100644
--- a/drivers/media/platform/verisilicon/hantro.h
+++ b/drivers/media/platform/verisilicon/hantro.h
@@ -460,12 +460,14 @@ hantro_get_dst_buf(struct hantro_ctx *ctx)
bool hantro_needs_postproc(const struct hantro_ctx *ctx,
const struct hantro_fmt *fmt);
+dma_addr_t
+hantro_postproc_get_dec_buf_addr(struct hantro_ctx *ctx, int index);
static inline dma_addr_t
hantro_get_dec_buf_addr(struct hantro_ctx *ctx, struct vb2_buffer *vb)
{
if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt))
- return ctx->postproc.dec_q[vb->index].dma;
+ return hantro_postproc_get_dec_buf_addr(ctx, vb->index);
return vb2_dma_contig_plane_dma_addr(vb, 0);
}
@@ -477,8 +479,8 @@ vb2_to_hantro_decoded_buf(struct vb2_buffer *buf)
void hantro_postproc_disable(struct hantro_ctx *ctx);
void hantro_postproc_enable(struct hantro_ctx *ctx);
+int hantro_postproc_init(struct hantro_ctx *ctx);
void hantro_postproc_free(struct hantro_ctx *ctx);
-int hantro_postproc_alloc(struct hantro_ctx *ctx);
int hanto_postproc_enum_framesizes(struct hantro_ctx *ctx,
struct v4l2_frmsizeenum *fsize);
diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
index 9e8faf7ba6fb..5c7b6e04aded 100644
--- a/drivers/media/platform/verisilicon/hantro_hw.h
+++ b/drivers/media/platform/verisilicon/hantro_hw.h
@@ -15,8 +15,6 @@
#include <media/v4l2-vp9.h>
#include <media/videobuf2-core.h>
-#define VB2_MAX_FRAME 32
-
#define DEC_8190_ALIGN_MASK 0x07U
#define MB_DIM 16
@@ -252,10 +250,10 @@ struct hantro_vp9_dec_hw_ctx {
/**
* struct hantro_postproc_ctx
*
- * @dec_q: References buffers, in decoder format.
+ * @dec_refs_xa: References buffers, in decoder format.
*/
struct hantro_postproc_ctx {
- struct hantro_aux_buf dec_q[VB2_MAX_FRAME];
+ struct xarray dec_refs_xa;
};
/**
diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
index 6437423ccf3a..a6b67d0cc66c 100644
--- a/drivers/media/platform/verisilicon/hantro_postproc.c
+++ b/drivers/media/platform/verisilicon/hantro_postproc.c
@@ -170,36 +170,23 @@ static int hantro_postproc_g2_enum_framesizes(struct hantro_ctx *ctx,
return 0;
}
-void hantro_postproc_free(struct hantro_ctx *ctx)
+static int hantro_postproc_alloc(struct hantro_ctx *ctx, int index)
{
struct hantro_dev *vpu = ctx->dev;
- unsigned int i;
-
- for (i = 0; i < VB2_MAX_FRAME; ++i) {
- struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
-
- if (priv->cpu) {
- dma_free_attrs(vpu->dev, priv->size, priv->cpu,
- priv->dma, priv->attrs);
- priv->cpu = NULL;
- }
- }
-}
-
-int hantro_postproc_alloc(struct hantro_ctx *ctx)
-{
- struct hantro_dev *vpu = ctx->dev;
- struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
- struct vb2_queue *cap_queue = &m2m_ctx->cap_q_ctx.q;
- unsigned int num_buffers = cap_queue->num_buffers;
struct v4l2_pix_format_mplane pix_mp;
const struct hantro_fmt *fmt;
- unsigned int i, buf_size;
+ struct hantro_aux_buf *priv;
+ unsigned int buf_size;
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
/* this should always pick native format */
fmt = hantro_get_default_fmt(ctx, false, ctx->bit_depth);
if (!fmt)
return -EINVAL;
+
v4l2_fill_pixfmt_mp(&pix_mp, fmt->fourcc, ctx->src_fmt.width,
ctx->src_fmt.height);
@@ -214,20 +201,74 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
buf_size += hantro_hevc_mv_size(pix_mp.width,
pix_mp.height);
- for (i = 0; i < num_buffers; ++i) {
- struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
-
- /*
- * The buffers on this queue are meant as intermediate
- * buffers for the decoder, so no mapping is needed.
- */
- priv->attrs = DMA_ATTR_NO_KERNEL_MAPPING;
- priv->cpu = dma_alloc_attrs(vpu->dev, buf_size, &priv->dma,
- GFP_KERNEL, priv->attrs);
- if (!priv->cpu)
- return -ENOMEM;
- priv->size = buf_size;
+ /*
+ * The buffers on this queue are meant as intermediate
+ * buffers for the decoder, so no mapping is needed.
+ */
+ priv->attrs = DMA_ATTR_NO_KERNEL_MAPPING;
+ priv->cpu = dma_alloc_attrs(vpu->dev, buf_size, &priv->dma, GFP_KERNEL, priv->attrs);
+ if (!priv->cpu)
+ return -ENOMEM;
+
+ priv->size = buf_size;
+
+ xa_store(&ctx->postproc.dec_refs_xa, index, priv, GFP_KERNEL);
+
+ return 0;
+}
+
+void hantro_postproc_free(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ struct hantro_aux_buf *priv;
+ unsigned long i;
+
+ xa_for_each(&ctx->postproc.dec_refs_xa, i, priv) {
+ dma_free_attrs(vpu->dev, priv->size, priv->cpu, priv->dma, priv->attrs);
+ kfree(priv);
+ }
+
+ xa_destroy(&ctx->postproc.dec_refs_xa);
+}
+
+int hantro_postproc_init(struct hantro_ctx *ctx)
+{
+ struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+ struct vb2_queue *cap_queue = &m2m_ctx->cap_q_ctx.q;
+ unsigned int num_buffers = cap_queue->num_buffers;
+ int i, ret = 0;
+
+ xa_init_flags(&ctx->postproc.dec_refs_xa, XA_FLAGS_ALLOC);
+
+ for (i = 0; i < num_buffers; i++) {
+ ret = hantro_postproc_alloc(ctx, i);
+ if (ret < 0) {
+ hantro_postproc_free(ctx);
+ return ret;
+ }
}
+
+ return ret;
+}
+
+dma_addr_t
+hantro_postproc_get_dec_buf_addr(struct hantro_ctx *ctx, int index)
+{
+ struct hantro_aux_buf *priv = xa_load(&ctx->postproc.dec_refs_xa, index);
+ int ret;
+
+ if (priv)
+ return priv->dma;
+
+ /* buffer not already allocated, try getting a new one */
+ ret = hantro_postproc_alloc(ctx, index);
+ if (ret < 0)
+ return 0;
+
+ priv = xa_load(&ctx->postproc.dec_refs_xa, index);
+ if (priv)
+ return priv->dma;
+
return 0;
}
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 61cfaaf4e927..0f615c1001c0 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -894,20 +894,10 @@ static int hantro_start_streaming(struct vb2_queue *q, unsigned int count)
ret = ctx->codec_ops->init(ctx);
if (ret)
return ret;
- }
-
- if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt)) {
- ret = hantro_postproc_alloc(ctx);
- if (ret)
- goto err_codec_exit;
+ ret = hantro_postproc_init(ctx);
}
}
return ret;
-
-err_codec_exit:
- if (ctx->codec_ops->exit)
- ctx->codec_ops->exit(ctx);
- return ret;
}
static void
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 05/11] media: verisilicon: Refactor postprocessor to store more buffers
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Like for vb queues remove VB2_MAX_FRAME upper limit on postprocessor
buffers storage capacity.
If a buffer hasn't been allocated when setting up the queue make
possible to alloc it on the fly while streaming. This happens
when resolution change between keyframes.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro.h | 6 +-
.../media/platform/verisilicon/hantro_hw.h | 6 +-
.../platform/verisilicon/hantro_postproc.c | 109 ++++++++++++------
.../media/platform/verisilicon/hantro_v4l2.c | 12 +-
4 files changed, 82 insertions(+), 51 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro.h b/drivers/media/platform/verisilicon/hantro.h
index 2989ebc631cc..bd0dca11b90a 100644
--- a/drivers/media/platform/verisilicon/hantro.h
+++ b/drivers/media/platform/verisilicon/hantro.h
@@ -460,12 +460,14 @@ hantro_get_dst_buf(struct hantro_ctx *ctx)
bool hantro_needs_postproc(const struct hantro_ctx *ctx,
const struct hantro_fmt *fmt);
+dma_addr_t
+hantro_postproc_get_dec_buf_addr(struct hantro_ctx *ctx, int index);
static inline dma_addr_t
hantro_get_dec_buf_addr(struct hantro_ctx *ctx, struct vb2_buffer *vb)
{
if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt))
- return ctx->postproc.dec_q[vb->index].dma;
+ return hantro_postproc_get_dec_buf_addr(ctx, vb->index);
return vb2_dma_contig_plane_dma_addr(vb, 0);
}
@@ -477,8 +479,8 @@ vb2_to_hantro_decoded_buf(struct vb2_buffer *buf)
void hantro_postproc_disable(struct hantro_ctx *ctx);
void hantro_postproc_enable(struct hantro_ctx *ctx);
+int hantro_postproc_init(struct hantro_ctx *ctx);
void hantro_postproc_free(struct hantro_ctx *ctx);
-int hantro_postproc_alloc(struct hantro_ctx *ctx);
int hanto_postproc_enum_framesizes(struct hantro_ctx *ctx,
struct v4l2_frmsizeenum *fsize);
diff --git a/drivers/media/platform/verisilicon/hantro_hw.h b/drivers/media/platform/verisilicon/hantro_hw.h
index 9e8faf7ba6fb..5c7b6e04aded 100644
--- a/drivers/media/platform/verisilicon/hantro_hw.h
+++ b/drivers/media/platform/verisilicon/hantro_hw.h
@@ -15,8 +15,6 @@
#include <media/v4l2-vp9.h>
#include <media/videobuf2-core.h>
-#define VB2_MAX_FRAME 32
-
#define DEC_8190_ALIGN_MASK 0x07U
#define MB_DIM 16
@@ -252,10 +250,10 @@ struct hantro_vp9_dec_hw_ctx {
/**
* struct hantro_postproc_ctx
*
- * @dec_q: References buffers, in decoder format.
+ * @dec_refs_xa: References buffers, in decoder format.
*/
struct hantro_postproc_ctx {
- struct hantro_aux_buf dec_q[VB2_MAX_FRAME];
+ struct xarray dec_refs_xa;
};
/**
diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
index 6437423ccf3a..a6b67d0cc66c 100644
--- a/drivers/media/platform/verisilicon/hantro_postproc.c
+++ b/drivers/media/platform/verisilicon/hantro_postproc.c
@@ -170,36 +170,23 @@ static int hantro_postproc_g2_enum_framesizes(struct hantro_ctx *ctx,
return 0;
}
-void hantro_postproc_free(struct hantro_ctx *ctx)
+static int hantro_postproc_alloc(struct hantro_ctx *ctx, int index)
{
struct hantro_dev *vpu = ctx->dev;
- unsigned int i;
-
- for (i = 0; i < VB2_MAX_FRAME; ++i) {
- struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
-
- if (priv->cpu) {
- dma_free_attrs(vpu->dev, priv->size, priv->cpu,
- priv->dma, priv->attrs);
- priv->cpu = NULL;
- }
- }
-}
-
-int hantro_postproc_alloc(struct hantro_ctx *ctx)
-{
- struct hantro_dev *vpu = ctx->dev;
- struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
- struct vb2_queue *cap_queue = &m2m_ctx->cap_q_ctx.q;
- unsigned int num_buffers = cap_queue->num_buffers;
struct v4l2_pix_format_mplane pix_mp;
const struct hantro_fmt *fmt;
- unsigned int i, buf_size;
+ struct hantro_aux_buf *priv;
+ unsigned int buf_size;
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
/* this should always pick native format */
fmt = hantro_get_default_fmt(ctx, false, ctx->bit_depth);
if (!fmt)
return -EINVAL;
+
v4l2_fill_pixfmt_mp(&pix_mp, fmt->fourcc, ctx->src_fmt.width,
ctx->src_fmt.height);
@@ -214,20 +201,74 @@ int hantro_postproc_alloc(struct hantro_ctx *ctx)
buf_size += hantro_hevc_mv_size(pix_mp.width,
pix_mp.height);
- for (i = 0; i < num_buffers; ++i) {
- struct hantro_aux_buf *priv = &ctx->postproc.dec_q[i];
-
- /*
- * The buffers on this queue are meant as intermediate
- * buffers for the decoder, so no mapping is needed.
- */
- priv->attrs = DMA_ATTR_NO_KERNEL_MAPPING;
- priv->cpu = dma_alloc_attrs(vpu->dev, buf_size, &priv->dma,
- GFP_KERNEL, priv->attrs);
- if (!priv->cpu)
- return -ENOMEM;
- priv->size = buf_size;
+ /*
+ * The buffers on this queue are meant as intermediate
+ * buffers for the decoder, so no mapping is needed.
+ */
+ priv->attrs = DMA_ATTR_NO_KERNEL_MAPPING;
+ priv->cpu = dma_alloc_attrs(vpu->dev, buf_size, &priv->dma, GFP_KERNEL, priv->attrs);
+ if (!priv->cpu)
+ return -ENOMEM;
+
+ priv->size = buf_size;
+
+ xa_store(&ctx->postproc.dec_refs_xa, index, priv, GFP_KERNEL);
+
+ return 0;
+}
+
+void hantro_postproc_free(struct hantro_ctx *ctx)
+{
+ struct hantro_dev *vpu = ctx->dev;
+ struct hantro_aux_buf *priv;
+ unsigned long i;
+
+ xa_for_each(&ctx->postproc.dec_refs_xa, i, priv) {
+ dma_free_attrs(vpu->dev, priv->size, priv->cpu, priv->dma, priv->attrs);
+ kfree(priv);
+ }
+
+ xa_destroy(&ctx->postproc.dec_refs_xa);
+}
+
+int hantro_postproc_init(struct hantro_ctx *ctx)
+{
+ struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+ struct vb2_queue *cap_queue = &m2m_ctx->cap_q_ctx.q;
+ unsigned int num_buffers = cap_queue->num_buffers;
+ int i, ret = 0;
+
+ xa_init_flags(&ctx->postproc.dec_refs_xa, XA_FLAGS_ALLOC);
+
+ for (i = 0; i < num_buffers; i++) {
+ ret = hantro_postproc_alloc(ctx, i);
+ if (ret < 0) {
+ hantro_postproc_free(ctx);
+ return ret;
+ }
}
+
+ return ret;
+}
+
+dma_addr_t
+hantro_postproc_get_dec_buf_addr(struct hantro_ctx *ctx, int index)
+{
+ struct hantro_aux_buf *priv = xa_load(&ctx->postproc.dec_refs_xa, index);
+ int ret;
+
+ if (priv)
+ return priv->dma;
+
+ /* buffer not already allocated, try getting a new one */
+ ret = hantro_postproc_alloc(ctx, index);
+ if (ret < 0)
+ return 0;
+
+ priv = xa_load(&ctx->postproc.dec_refs_xa, index);
+ if (priv)
+ return priv->dma;
+
return 0;
}
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 61cfaaf4e927..0f615c1001c0 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -894,20 +894,10 @@ static int hantro_start_streaming(struct vb2_queue *q, unsigned int count)
ret = ctx->codec_ops->init(ctx);
if (ret)
return ret;
- }
-
- if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt)) {
- ret = hantro_postproc_alloc(ctx);
- if (ret)
- goto err_codec_exit;
+ ret = hantro_postproc_init(ctx);
}
}
return ret;
-
-err_codec_exit:
- if (ctx->codec_ops->exit)
- ctx->codec_ops->exit(ctx);
- return ret;
}
static void
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 183+ messages in thread
* [PATCH v3 06/11] media: verisilicon: Store chroma and motion vectors offset
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Store computed values of chroma and motion vectors offset because
they depends on width and height values which change if the resolution
change.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro.h | 2 ++
drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c | 6 ++++--
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro.h b/drivers/media/platform/verisilicon/hantro.h
index bd0dca11b90a..16c7e9bafde3 100644
--- a/drivers/media/platform/verisilicon/hantro.h
+++ b/drivers/media/platform/verisilicon/hantro.h
@@ -320,6 +320,8 @@ struct hantro_vp9_decoded_buffer_info {
/* Info needed when the decoded frame serves as a reference frame. */
unsigned short width;
unsigned short height;
+ size_t chroma_offset;
+ size_t mv_offset;
u32 bit_depth : 4;
};
diff --git a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
index 6fc4b555517f..6db1c32fce4d 100644
--- a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
+++ b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
@@ -158,9 +158,11 @@ static void config_output(struct hantro_ctx *ctx,
chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
hantro_write_addr(ctx->dev, G2_OUT_CHROMA_ADDR, chroma_addr);
+ dst->vp9.chroma_offset = chroma_offset(ctx, dec_params);
mv_addr = luma_addr + mv_offset(ctx, dec_params);
hantro_write_addr(ctx->dev, G2_OUT_MV_ADDR, mv_addr);
+ dst->vp9.mv_offset = mv_offset(ctx, dec_params);
}
struct hantro_vp9_ref_reg {
@@ -195,7 +197,7 @@ static void config_ref(struct hantro_ctx *ctx,
luma_addr = hantro_get_dec_buf_addr(ctx, &buf->base.vb.vb2_buf);
hantro_write_addr(ctx->dev, ref_reg->y_base, luma_addr);
- chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
+ chroma_addr = luma_addr + buf->vp9.chroma_offset;
hantro_write_addr(ctx->dev, ref_reg->c_base, chroma_addr);
}
@@ -238,7 +240,7 @@ static void config_ref_registers(struct hantro_ctx *ctx,
config_ref(ctx, dst, &ref_regs[2], dec_params, dec_params->alt_frame_ts);
mv_addr = hantro_get_dec_buf_addr(ctx, &mv_ref->base.vb.vb2_buf) +
- mv_offset(ctx, dec_params);
+ mv_ref->vp9.mv_offset;
hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_addr);
hantro_reg_write(ctx->dev, &vp9_last_sign_bias,
--
2.39.2
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 06/11] media: verisilicon: Store chroma and motion vectors offset
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Store computed values of chroma and motion vectors offset because
they depends on width and height values which change if the resolution
change.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro.h | 2 ++
drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c | 6 ++++--
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro.h b/drivers/media/platform/verisilicon/hantro.h
index bd0dca11b90a..16c7e9bafde3 100644
--- a/drivers/media/platform/verisilicon/hantro.h
+++ b/drivers/media/platform/verisilicon/hantro.h
@@ -320,6 +320,8 @@ struct hantro_vp9_decoded_buffer_info {
/* Info needed when the decoded frame serves as a reference frame. */
unsigned short width;
unsigned short height;
+ size_t chroma_offset;
+ size_t mv_offset;
u32 bit_depth : 4;
};
diff --git a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
index 6fc4b555517f..6db1c32fce4d 100644
--- a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
+++ b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
@@ -158,9 +158,11 @@ static void config_output(struct hantro_ctx *ctx,
chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
hantro_write_addr(ctx->dev, G2_OUT_CHROMA_ADDR, chroma_addr);
+ dst->vp9.chroma_offset = chroma_offset(ctx, dec_params);
mv_addr = luma_addr + mv_offset(ctx, dec_params);
hantro_write_addr(ctx->dev, G2_OUT_MV_ADDR, mv_addr);
+ dst->vp9.mv_offset = mv_offset(ctx, dec_params);
}
struct hantro_vp9_ref_reg {
@@ -195,7 +197,7 @@ static void config_ref(struct hantro_ctx *ctx,
luma_addr = hantro_get_dec_buf_addr(ctx, &buf->base.vb.vb2_buf);
hantro_write_addr(ctx->dev, ref_reg->y_base, luma_addr);
- chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
+ chroma_addr = luma_addr + buf->vp9.chroma_offset;
hantro_write_addr(ctx->dev, ref_reg->c_base, chroma_addr);
}
@@ -238,7 +240,7 @@ static void config_ref_registers(struct hantro_ctx *ctx,
config_ref(ctx, dst, &ref_regs[2], dec_params, dec_params->alt_frame_ts);
mv_addr = hantro_get_dec_buf_addr(ctx, &mv_ref->base.vb.vb2_buf) +
- mv_offset(ctx, dec_params);
+ mv_ref->vp9.mv_offset;
hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_addr);
hantro_reg_write(ctx->dev, &vp9_last_sign_bias,
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 06/11] media: verisilicon: Store chroma and motion vectors offset
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Store computed values of chroma and motion vectors offset because
they depends on width and height values which change if the resolution
change.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro.h | 2 ++
drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c | 6 ++++--
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro.h b/drivers/media/platform/verisilicon/hantro.h
index bd0dca11b90a..16c7e9bafde3 100644
--- a/drivers/media/platform/verisilicon/hantro.h
+++ b/drivers/media/platform/verisilicon/hantro.h
@@ -320,6 +320,8 @@ struct hantro_vp9_decoded_buffer_info {
/* Info needed when the decoded frame serves as a reference frame. */
unsigned short width;
unsigned short height;
+ size_t chroma_offset;
+ size_t mv_offset;
u32 bit_depth : 4;
};
diff --git a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
index 6fc4b555517f..6db1c32fce4d 100644
--- a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
+++ b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
@@ -158,9 +158,11 @@ static void config_output(struct hantro_ctx *ctx,
chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
hantro_write_addr(ctx->dev, G2_OUT_CHROMA_ADDR, chroma_addr);
+ dst->vp9.chroma_offset = chroma_offset(ctx, dec_params);
mv_addr = luma_addr + mv_offset(ctx, dec_params);
hantro_write_addr(ctx->dev, G2_OUT_MV_ADDR, mv_addr);
+ dst->vp9.mv_offset = mv_offset(ctx, dec_params);
}
struct hantro_vp9_ref_reg {
@@ -195,7 +197,7 @@ static void config_ref(struct hantro_ctx *ctx,
luma_addr = hantro_get_dec_buf_addr(ctx, &buf->base.vb.vb2_buf);
hantro_write_addr(ctx->dev, ref_reg->y_base, luma_addr);
- chroma_addr = luma_addr + chroma_offset(ctx, dec_params);
+ chroma_addr = luma_addr + buf->vp9.chroma_offset;
hantro_write_addr(ctx->dev, ref_reg->c_base, chroma_addr);
}
@@ -238,7 +240,7 @@ static void config_ref_registers(struct hantro_ctx *ctx,
config_ref(ctx, dst, &ref_regs[2], dec_params, dec_params->alt_frame_ts);
mv_addr = hantro_get_dec_buf_addr(ctx, &mv_ref->base.vb.vb2_buf) +
- mv_offset(ctx, dec_params);
+ mv_ref->vp9.mv_offset;
hantro_write_addr(ctx->dev, G2_REF_MV_ADDR(0), mv_addr);
hantro_reg_write(ctx->dev, &vp9_last_sign_bias,
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 183+ messages in thread* Re: [PATCH v3 06/11] media: verisilicon: Store chroma and motion vectors offset
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-07-02 7:48 ` Markus Elfring
-1 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 7:48 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
…
> they depends on …
Please avoid a typo in a subsequent change description.
Regards,
Markus
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 06/11] media: verisilicon: Store chroma and motion vectors offset
@ 2023-07-02 7:48 ` Markus Elfring
0 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 7:48 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
…
> they depends on …
Please avoid a typo in a subsequent change description.
Regards,
Markus
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 06/11] media: verisilicon: Store chroma and motion vectors offset
@ 2023-07-02 7:48 ` Markus Elfring
0 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 7:48 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
…
> they depends on …
Please avoid a typo in a subsequent change description.
Regards,
Markus
^ permalink raw reply [flat|nested] 183+ messages in thread
* [PATCH v3 07/11] media: verisilicon: vp9: Use destination buffer height to compute chroma offset
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Source and destination buffer height may not be the same because
alignment constraint are different.
Use destination height to compute chroma offset because we target
this buffer as hardware output.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
index 6db1c32fce4d..1f3f5e7ce978 100644
--- a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
+++ b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
@@ -93,9 +93,7 @@ static int start_prepare_run(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_
static size_t chroma_offset(const struct hantro_ctx *ctx,
const struct v4l2_ctrl_vp9_frame *dec_params)
{
- int bytes_per_pixel = dec_params->bit_depth == 8 ? 1 : 2;
-
- return ctx->src_fmt.width * ctx->src_fmt.height * bytes_per_pixel;
+ return ctx->dst_fmt.width * ctx->dst_fmt.height * ctx->bit_depth / 8;
}
static size_t mv_offset(const struct hantro_ctx *ctx,
--
2.39.2
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 07/11] media: verisilicon: vp9: Use destination buffer height to compute chroma offset
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Source and destination buffer height may not be the same because
alignment constraint are different.
Use destination height to compute chroma offset because we target
this buffer as hardware output.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
index 6db1c32fce4d..1f3f5e7ce978 100644
--- a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
+++ b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
@@ -93,9 +93,7 @@ static int start_prepare_run(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_
static size_t chroma_offset(const struct hantro_ctx *ctx,
const struct v4l2_ctrl_vp9_frame *dec_params)
{
- int bytes_per_pixel = dec_params->bit_depth == 8 ? 1 : 2;
-
- return ctx->src_fmt.width * ctx->src_fmt.height * bytes_per_pixel;
+ return ctx->dst_fmt.width * ctx->dst_fmt.height * ctx->bit_depth / 8;
}
static size_t mv_offset(const struct hantro_ctx *ctx,
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 07/11] media: verisilicon: vp9: Use destination buffer height to compute chroma offset
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Source and destination buffer height may not be the same because
alignment constraint are different.
Use destination height to compute chroma offset because we target
this buffer as hardware output.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
index 6db1c32fce4d..1f3f5e7ce978 100644
--- a/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
+++ b/drivers/media/platform/verisilicon/hantro_g2_vp9_dec.c
@@ -93,9 +93,7 @@ static int start_prepare_run(struct hantro_ctx *ctx, const struct v4l2_ctrl_vp9_
static size_t chroma_offset(const struct hantro_ctx *ctx,
const struct v4l2_ctrl_vp9_frame *dec_params)
{
- int bytes_per_pixel = dec_params->bit_depth == 8 ? 1 : 2;
-
- return ctx->src_fmt.width * ctx->src_fmt.height * bytes_per_pixel;
+ return ctx->dst_fmt.width * ctx->dst_fmt.height * ctx->bit_depth / 8;
}
static size_t mv_offset(const struct hantro_ctx *ctx,
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 183+ messages in thread* Re: [PATCH v3 07/11] media: verisilicon: vp9: Use destination buffer height to compute chroma offset
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-07-02 9:14 ` Markus Elfring
-1 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 9:14 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
> Source and destination buffer height may not be the same because
> alignment constraint are different.
How do you think about to use the wording variant “alignment constraints are different”
in a subsequent change description?
Regards,
Markus
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 07/11] media: verisilicon: vp9: Use destination buffer height to compute chroma offset
@ 2023-07-02 9:14 ` Markus Elfring
0 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 9:14 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
> Source and destination buffer height may not be the same because
> alignment constraint are different.
How do you think about to use the wording variant “alignment constraints are different”
in a subsequent change description?
Regards,
Markus
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 07/11] media: verisilicon: vp9: Use destination buffer height to compute chroma offset
@ 2023-07-02 9:14 ` Markus Elfring
0 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 9:14 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
> Source and destination buffer height may not be the same because
> alignment constraint are different.
How do you think about to use the wording variant “alignment constraints are different”
in a subsequent change description?
Regards,
Markus
^ permalink raw reply [flat|nested] 183+ messages in thread
* [PATCH v3 08/11] media: verisilicon: postproc: Fix down scale test
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Do not allow down scaling if the source buffer resolution is
smaller than destination one.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro_postproc.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
index a6b67d0cc66c..a4f2c5e8932c 100644
--- a/drivers/media/platform/verisilicon/hantro_postproc.c
+++ b/drivers/media/platform/verisilicon/hantro_postproc.c
@@ -103,7 +103,10 @@ static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
static int down_scale_factor(struct hantro_ctx *ctx)
{
- if (ctx->src_fmt.width == ctx->dst_fmt.width)
+ if (ctx->src_fmt.width <= ctx->dst_fmt.width)
+ return 0;
+
+ if (ctx->src_fmt.height <= ctx->dst_fmt.height)
return 0;
return DIV_ROUND_CLOSEST(ctx->src_fmt.width, ctx->dst_fmt.width);
--
2.39.2
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 08/11] media: verisilicon: postproc: Fix down scale test
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Do not allow down scaling if the source buffer resolution is
smaller than destination one.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro_postproc.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
index a6b67d0cc66c..a4f2c5e8932c 100644
--- a/drivers/media/platform/verisilicon/hantro_postproc.c
+++ b/drivers/media/platform/verisilicon/hantro_postproc.c
@@ -103,7 +103,10 @@ static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
static int down_scale_factor(struct hantro_ctx *ctx)
{
- if (ctx->src_fmt.width == ctx->dst_fmt.width)
+ if (ctx->src_fmt.width <= ctx->dst_fmt.width)
+ return 0;
+
+ if (ctx->src_fmt.height <= ctx->dst_fmt.height)
return 0;
return DIV_ROUND_CLOSEST(ctx->src_fmt.width, ctx->dst_fmt.width);
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 08/11] media: verisilicon: postproc: Fix down scale test
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Do not allow down scaling if the source buffer resolution is
smaller than destination one.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
drivers/media/platform/verisilicon/hantro_postproc.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
index a6b67d0cc66c..a4f2c5e8932c 100644
--- a/drivers/media/platform/verisilicon/hantro_postproc.c
+++ b/drivers/media/platform/verisilicon/hantro_postproc.c
@@ -103,7 +103,10 @@ static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
static int down_scale_factor(struct hantro_ctx *ctx)
{
- if (ctx->src_fmt.width == ctx->dst_fmt.width)
+ if (ctx->src_fmt.width <= ctx->dst_fmt.width)
+ return 0;
+
+ if (ctx->src_fmt.height <= ctx->dst_fmt.height)
return 0;
return DIV_ROUND_CLOSEST(ctx->src_fmt.width, ctx->dst_fmt.width);
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 183+ messages in thread* Re: [PATCH v3 08/11] media: verisilicon: postproc: Fix down scale test
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 15:27 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 15:27 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Le 22/06/2023 à 15:13, Benjamin Gaignard a écrit :
> Do not allow down scaling if the source buffer resolution is
> smaller than destination one.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> drivers/media/platform/verisilicon/hantro_postproc.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
> index a6b67d0cc66c..a4f2c5e8932c 100644
> --- a/drivers/media/platform/verisilicon/hantro_postproc.c
> +++ b/drivers/media/platform/verisilicon/hantro_postproc.c
> @@ -103,7 +103,10 @@ static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
>
> static int down_scale_factor(struct hantro_ctx *ctx)
> {
> - if (ctx->src_fmt.width == ctx->dst_fmt.width)
> + if (ctx->src_fmt.width <= ctx->dst_fmt.width)
> + return 0;
> +
> + if (ctx->src_fmt.height <= ctx->dst_fmt.height)
One test will be enough.
I will fix that in v4.
Benjamin
> return 0;
>
> return DIV_ROUND_CLOSEST(ctx->src_fmt.width, ctx->dst_fmt.width);
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 08/11] media: verisilicon: postproc: Fix down scale test
@ 2023-06-22 15:27 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 15:27 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Le 22/06/2023 à 15:13, Benjamin Gaignard a écrit :
> Do not allow down scaling if the source buffer resolution is
> smaller than destination one.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> drivers/media/platform/verisilicon/hantro_postproc.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
> index a6b67d0cc66c..a4f2c5e8932c 100644
> --- a/drivers/media/platform/verisilicon/hantro_postproc.c
> +++ b/drivers/media/platform/verisilicon/hantro_postproc.c
> @@ -103,7 +103,10 @@ static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
>
> static int down_scale_factor(struct hantro_ctx *ctx)
> {
> - if (ctx->src_fmt.width == ctx->dst_fmt.width)
> + if (ctx->src_fmt.width <= ctx->dst_fmt.width)
> + return 0;
> +
> + if (ctx->src_fmt.height <= ctx->dst_fmt.height)
One test will be enough.
I will fix that in v4.
Benjamin
> return 0;
>
> return DIV_ROUND_CLOSEST(ctx->src_fmt.width, ctx->dst_fmt.width);
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 08/11] media: verisilicon: postproc: Fix down scale test
@ 2023-06-22 15:27 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 15:27 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Le 22/06/2023 à 15:13, Benjamin Gaignard a écrit :
> Do not allow down scaling if the source buffer resolution is
> smaller than destination one.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> drivers/media/platform/verisilicon/hantro_postproc.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/verisilicon/hantro_postproc.c b/drivers/media/platform/verisilicon/hantro_postproc.c
> index a6b67d0cc66c..a4f2c5e8932c 100644
> --- a/drivers/media/platform/verisilicon/hantro_postproc.c
> +++ b/drivers/media/platform/verisilicon/hantro_postproc.c
> @@ -103,7 +103,10 @@ static void hantro_postproc_g1_enable(struct hantro_ctx *ctx)
>
> static int down_scale_factor(struct hantro_ctx *ctx)
> {
> - if (ctx->src_fmt.width == ctx->dst_fmt.width)
> + if (ctx->src_fmt.width <= ctx->dst_fmt.width)
> + return 0;
> +
> + if (ctx->src_fmt.height <= ctx->dst_fmt.height)
One test will be enough.
I will fix that in v4.
Benjamin
> return 0;
>
> return DIV_ROUND_CLOSEST(ctx->src_fmt.width, ctx->dst_fmt.width);
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* [PATCH v3 09/11] media: verisilicon: vp9: Allow to change resolution while streaming
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Remove all checks that prohibit to set a new format while streaming.
This allow to change dynamically the resolution if the pixel format
remains the same.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/platform/verisilicon/hantro_v4l2.c | 24 +++----------------
1 file changed, 3 insertions(+), 21 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 0f615c1001c0..cdddfb4179aa 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -480,25 +480,14 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx,
return ret;
if (!ctx->is_encoder) {
- struct vb2_queue *peer_vq;
-
/*
* In order to support dynamic resolution change,
* the decoder admits a resolution change, as long
- * as the pixelformat remains. Can't be done if streaming.
- */
- if (vb2_is_streaming(vq) || (vb2_is_busy(vq) &&
- pix_mp->pixelformat != ctx->src_fmt.pixelformat))
- return -EBUSY;
- /*
- * Since format change on the OUTPUT queue will reset
- * the CAPTURE queue, we can't allow doing so
- * when the CAPTURE queue has buffers allocated.
+ * as the pixelformat remains.
*/
- peer_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
- V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
- if (vb2_is_busy(peer_vq))
+ if (vb2_is_streaming(vq) && pix_mp->pixelformat != ctx->src_fmt.pixelformat) {
return -EBUSY;
+ }
} else {
/*
* The encoder doesn't admit a format change if
@@ -541,15 +530,8 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx,
static int hantro_set_fmt_cap(struct hantro_ctx *ctx,
struct v4l2_pix_format_mplane *pix_mp)
{
- struct vb2_queue *vq;
int ret;
- /* Change not allowed if queue is busy. */
- vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
- V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
- if (vb2_is_busy(vq))
- return -EBUSY;
-
if (ctx->is_encoder) {
struct vb2_queue *peer_vq;
--
2.39.2
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 09/11] media: verisilicon: vp9: Allow to change resolution while streaming
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Remove all checks that prohibit to set a new format while streaming.
This allow to change dynamically the resolution if the pixel format
remains the same.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/platform/verisilicon/hantro_v4l2.c | 24 +++----------------
1 file changed, 3 insertions(+), 21 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 0f615c1001c0..cdddfb4179aa 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -480,25 +480,14 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx,
return ret;
if (!ctx->is_encoder) {
- struct vb2_queue *peer_vq;
-
/*
* In order to support dynamic resolution change,
* the decoder admits a resolution change, as long
- * as the pixelformat remains. Can't be done if streaming.
- */
- if (vb2_is_streaming(vq) || (vb2_is_busy(vq) &&
- pix_mp->pixelformat != ctx->src_fmt.pixelformat))
- return -EBUSY;
- /*
- * Since format change on the OUTPUT queue will reset
- * the CAPTURE queue, we can't allow doing so
- * when the CAPTURE queue has buffers allocated.
+ * as the pixelformat remains.
*/
- peer_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
- V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
- if (vb2_is_busy(peer_vq))
+ if (vb2_is_streaming(vq) && pix_mp->pixelformat != ctx->src_fmt.pixelformat) {
return -EBUSY;
+ }
} else {
/*
* The encoder doesn't admit a format change if
@@ -541,15 +530,8 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx,
static int hantro_set_fmt_cap(struct hantro_ctx *ctx,
struct v4l2_pix_format_mplane *pix_mp)
{
- struct vb2_queue *vq;
int ret;
- /* Change not allowed if queue is busy. */
- vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
- V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
- if (vb2_is_busy(vq))
- return -EBUSY;
-
if (ctx->is_encoder) {
struct vb2_queue *peer_vq;
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 09/11] media: verisilicon: vp9: Allow to change resolution while streaming
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Remove all checks that prohibit to set a new format while streaming.
This allow to change dynamically the resolution if the pixel format
remains the same.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/platform/verisilicon/hantro_v4l2.c | 24 +++----------------
1 file changed, 3 insertions(+), 21 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 0f615c1001c0..cdddfb4179aa 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -480,25 +480,14 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx,
return ret;
if (!ctx->is_encoder) {
- struct vb2_queue *peer_vq;
-
/*
* In order to support dynamic resolution change,
* the decoder admits a resolution change, as long
- * as the pixelformat remains. Can't be done if streaming.
- */
- if (vb2_is_streaming(vq) || (vb2_is_busy(vq) &&
- pix_mp->pixelformat != ctx->src_fmt.pixelformat))
- return -EBUSY;
- /*
- * Since format change on the OUTPUT queue will reset
- * the CAPTURE queue, we can't allow doing so
- * when the CAPTURE queue has buffers allocated.
+ * as the pixelformat remains.
*/
- peer_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
- V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
- if (vb2_is_busy(peer_vq))
+ if (vb2_is_streaming(vq) && pix_mp->pixelformat != ctx->src_fmt.pixelformat) {
return -EBUSY;
+ }
} else {
/*
* The encoder doesn't admit a format change if
@@ -541,15 +530,8 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx,
static int hantro_set_fmt_cap(struct hantro_ctx *ctx,
struct v4l2_pix_format_mplane *pix_mp)
{
- struct vb2_queue *vq;
int ret;
- /* Change not allowed if queue is busy. */
- vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
- V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
- if (vb2_is_busy(vq))
- return -EBUSY;
-
if (ctx->is_encoder) {
struct vb2_queue *peer_vq;
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 183+ messages in thread* Re: [PATCH v3 09/11] media: verisilicon: vp9: Allow to change resolution while streaming
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-07-02 9:33 ` Markus Elfring
-1 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 9:33 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
…
> This allow to change dynamically the resolution if the pixel format
> remains the same.
Would the wording “This adjustment allows …” be more appropriate
for a subsequent change description?
Regards,
Markus
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 09/11] media: verisilicon: vp9: Allow to change resolution while streaming
@ 2023-07-02 9:33 ` Markus Elfring
0 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 9:33 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
…
> This allow to change dynamically the resolution if the pixel format
> remains the same.
Would the wording “This adjustment allows …” be more appropriate
for a subsequent change description?
Regards,
Markus
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 09/11] media: verisilicon: vp9: Allow to change resolution while streaming
@ 2023-07-02 9:33 ` Markus Elfring
0 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 9:33 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
…
> This allow to change dynamically the resolution if the pixel format
> remains the same.
Would the wording “This adjustment allows …” be more appropriate
for a subsequent change description?
Regards,
Markus
^ permalink raw reply [flat|nested] 183+ messages in thread
* [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../userspace-api/media/v4l/user-func.rst | 1 +
.../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
.../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
.../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
drivers/media/v4l2-core/v4l2-dev.c | 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
include/media/v4l2-ioctl.h | 4 ++
include/media/videobuf2-core.h | 9 ++++
include/media/videobuf2-v4l2.h | 11 ++++
include/uapi/linux/videodev2.h | 2 +
10 files changed, 128 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst
index 15ff0bf7bbe6..8c74016e12fd 100644
--- a/Documentation/userspace-api/media/v4l/user-func.rst
+++ b/Documentation/userspace-api/media/v4l/user-func.rst
@@ -17,6 +17,7 @@ Function Reference
vidioc-dbg-g-chip-info
vidioc-dbg-g-register
vidioc-decoder-cmd
+ vidioc-delete-buf
vidioc-dqevent
vidioc-dv-timings-cap
vidioc-encoder-cmd
diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
new file mode 100644
index 000000000000..0e7ce58f91bc
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
@@ -0,0 +1,51 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+.. c:namespace:: V4L
+
+.. _VIDIOC_DELETE_BUF:
+
+************************
+ioctl VIDIOC_DELETE_BUF
+************************
+
+Name
+====
+
+VIDIOC_DELETE_BUF - Delete a buffer from a queue
+
+Synopsis
+========
+
+.. c:macro:: VIDIOC_DELETE_BUF
+
+``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
+
+Arguments
+=========
+
+``fd``
+ File descriptor returned by :c:func:`open()`.
+
+``argp``
+ Pointer to struct :c:type:`v4l2_buffer`.
+
+Description
+===========
+
+Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
+delete a buffer from a queue.
+
+The struct :c:type:`v4l2_buffer` structure is specified in
+:ref:`buffer`.
+
+Return Value
+============
+
+On success 0 is returned, on error -1 and the ``errno`` variable is set
+appropriately. The generic error codes are described at the
+:ref:`Generic Error Codes <gen-errors>` chapter.
+
+EBUSY
+ File I/O is in progress.
+
+EINVAL
+ The buffer ``index`` doesn't exist in the queue.
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 899783f67580..aa546c972c3d 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
}
EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
+{
+ struct vb2_buffer *vb;
+
+ vb = vb2_get_buffer(q, index);
+ if (!vb) {
+ dprintk(q, 1, "invalid buffer index %d\n", index);
+ return -EINVAL;
+ }
+
+ if (vb->state != VB2_BUF_STATE_DEQUEUED) {
+ dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
+ return -EINVAL;
+ }
+
+ if (vb->planes[0].mem_priv)
+ call_void_vb_qop(vb, buf_cleanup, vb);
+
+ /* Free MMAP buffers or release USERPTR buffers */
+ if (q->memory == VB2_MEMORY_MMAP)
+ __vb2_buf_mem_free(vb);
+ else if (q->memory == VB2_MEMORY_DMABUF)
+ __vb2_buf_dmabuf_put(vb);
+ else
+ __vb2_buf_userptr_put(vb);
+
+ vb2_queue_remove_buffer(q, vb);
+ kfree(vb);
+
+ dprintk(q, 2, "buffer %d deleted\n", index);
+ return 0;
+}
+
/*
* vb2_start_streaming() - Attempt to start streaming.
* @q: videobuf2 queue
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 724135d41f7f..cea666c17b41 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
}
EXPORT_SYMBOL_GPL(vb2_prepare_buf);
+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
+{
+ return vb2_core_delete_buf(q, b->index);
+}
+EXPORT_SYMBOL_GPL(vb2_delete_buf);
+
int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
{
unsigned requested_planes = 1;
diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
index f81279492682..80ace2e1e932 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device *vdev)
SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
+ SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF, vidioc_delete_buf);
}
if (is_vid || is_vbi || is_meta) {
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index a858acea6547..1c737279d3ef 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct v4l2_ioctl_ops *ops,
return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
}
+static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
+ struct file *file, void *fh, void *arg)
+{
+ struct v4l2_buffer *b = arg;
+ int ret = check_fmt(file, b->type);
+
+ return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
+}
+
static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
struct file *file, void *fh, void *arg)
{
@@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0),
IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
+ IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer, INFO_FL_QUEUE),
};
#define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
index edb733f21604..2f232ed884c7 100644
--- a/include/media/v4l2-ioctl.h
+++ b/include/media/v4l2-ioctl.h
@@ -163,6 +163,8 @@ struct v4l2_fh;
* :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
* @vidioc_prepare_buf: pointer to the function that implements
* :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
+ * @vidioc_delete_buf: pointer to the function that implements
+ * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
* @vidioc_overlay: pointer to the function that implements
* :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
* @vidioc_g_fbuf: pointer to the function that implements
@@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
struct v4l2_create_buffers *b);
int (*vidioc_prepare_buf)(struct file *file, void *fh,
struct v4l2_buffer *b);
+ int (*vidioc_delete_buf)(struct file *file, void *fh,
+ struct v4l2_buffer *b);
int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
int (*vidioc_g_fbuf)(struct file *file, void *fh,
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 080b783d608d..0f9e68f76b77 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
*/
int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
+/**
+ * vb2_core_delete_buf() -
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @index: id number of the buffer.
+ *
+ * Return: returns zero on success; an error code otherwise.
+ */
+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
+
/**
* vb2_core_qbuf() - Queue a buffer from userspace
*
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 88a7a565170e..3beeb4c735f0 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
*/
int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
struct v4l2_buffer *b);
+/**
+ * vb2_delete_buf() - Delete the buffer from the queue
+ *
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @b: buffer structure passed from userspace to
+ * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
+ *
+ * The return values from this function are intended to be directly returned
+ * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
+ */
+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
/**
* vb2_qbuf() - Queue a buffer from userspace
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index aee75eb9e686..31bba1915642 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
#define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
#define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
+#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
+
/* Reminder: when adding new ioctls please add support for them to
drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
--
2.39.2
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../userspace-api/media/v4l/user-func.rst | 1 +
.../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
.../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
.../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
drivers/media/v4l2-core/v4l2-dev.c | 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
include/media/v4l2-ioctl.h | 4 ++
include/media/videobuf2-core.h | 9 ++++
include/media/videobuf2-v4l2.h | 11 ++++
include/uapi/linux/videodev2.h | 2 +
10 files changed, 128 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst
index 15ff0bf7bbe6..8c74016e12fd 100644
--- a/Documentation/userspace-api/media/v4l/user-func.rst
+++ b/Documentation/userspace-api/media/v4l/user-func.rst
@@ -17,6 +17,7 @@ Function Reference
vidioc-dbg-g-chip-info
vidioc-dbg-g-register
vidioc-decoder-cmd
+ vidioc-delete-buf
vidioc-dqevent
vidioc-dv-timings-cap
vidioc-encoder-cmd
diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
new file mode 100644
index 000000000000..0e7ce58f91bc
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
@@ -0,0 +1,51 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+.. c:namespace:: V4L
+
+.. _VIDIOC_DELETE_BUF:
+
+************************
+ioctl VIDIOC_DELETE_BUF
+************************
+
+Name
+====
+
+VIDIOC_DELETE_BUF - Delete a buffer from a queue
+
+Synopsis
+========
+
+.. c:macro:: VIDIOC_DELETE_BUF
+
+``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
+
+Arguments
+=========
+
+``fd``
+ File descriptor returned by :c:func:`open()`.
+
+``argp``
+ Pointer to struct :c:type:`v4l2_buffer`.
+
+Description
+===========
+
+Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
+delete a buffer from a queue.
+
+The struct :c:type:`v4l2_buffer` structure is specified in
+:ref:`buffer`.
+
+Return Value
+============
+
+On success 0 is returned, on error -1 and the ``errno`` variable is set
+appropriately. The generic error codes are described at the
+:ref:`Generic Error Codes <gen-errors>` chapter.
+
+EBUSY
+ File I/O is in progress.
+
+EINVAL
+ The buffer ``index`` doesn't exist in the queue.
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 899783f67580..aa546c972c3d 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
}
EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
+{
+ struct vb2_buffer *vb;
+
+ vb = vb2_get_buffer(q, index);
+ if (!vb) {
+ dprintk(q, 1, "invalid buffer index %d\n", index);
+ return -EINVAL;
+ }
+
+ if (vb->state != VB2_BUF_STATE_DEQUEUED) {
+ dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
+ return -EINVAL;
+ }
+
+ if (vb->planes[0].mem_priv)
+ call_void_vb_qop(vb, buf_cleanup, vb);
+
+ /* Free MMAP buffers or release USERPTR buffers */
+ if (q->memory == VB2_MEMORY_MMAP)
+ __vb2_buf_mem_free(vb);
+ else if (q->memory == VB2_MEMORY_DMABUF)
+ __vb2_buf_dmabuf_put(vb);
+ else
+ __vb2_buf_userptr_put(vb);
+
+ vb2_queue_remove_buffer(q, vb);
+ kfree(vb);
+
+ dprintk(q, 2, "buffer %d deleted\n", index);
+ return 0;
+}
+
/*
* vb2_start_streaming() - Attempt to start streaming.
* @q: videobuf2 queue
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 724135d41f7f..cea666c17b41 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
}
EXPORT_SYMBOL_GPL(vb2_prepare_buf);
+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
+{
+ return vb2_core_delete_buf(q, b->index);
+}
+EXPORT_SYMBOL_GPL(vb2_delete_buf);
+
int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
{
unsigned requested_planes = 1;
diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
index f81279492682..80ace2e1e932 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device *vdev)
SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
+ SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF, vidioc_delete_buf);
}
if (is_vid || is_vbi || is_meta) {
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index a858acea6547..1c737279d3ef 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct v4l2_ioctl_ops *ops,
return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
}
+static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
+ struct file *file, void *fh, void *arg)
+{
+ struct v4l2_buffer *b = arg;
+ int ret = check_fmt(file, b->type);
+
+ return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
+}
+
static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
struct file *file, void *fh, void *arg)
{
@@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0),
IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
+ IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer, INFO_FL_QUEUE),
};
#define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
index edb733f21604..2f232ed884c7 100644
--- a/include/media/v4l2-ioctl.h
+++ b/include/media/v4l2-ioctl.h
@@ -163,6 +163,8 @@ struct v4l2_fh;
* :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
* @vidioc_prepare_buf: pointer to the function that implements
* :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
+ * @vidioc_delete_buf: pointer to the function that implements
+ * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
* @vidioc_overlay: pointer to the function that implements
* :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
* @vidioc_g_fbuf: pointer to the function that implements
@@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
struct v4l2_create_buffers *b);
int (*vidioc_prepare_buf)(struct file *file, void *fh,
struct v4l2_buffer *b);
+ int (*vidioc_delete_buf)(struct file *file, void *fh,
+ struct v4l2_buffer *b);
int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
int (*vidioc_g_fbuf)(struct file *file, void *fh,
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 080b783d608d..0f9e68f76b77 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
*/
int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
+/**
+ * vb2_core_delete_buf() -
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @index: id number of the buffer.
+ *
+ * Return: returns zero on success; an error code otherwise.
+ */
+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
+
/**
* vb2_core_qbuf() - Queue a buffer from userspace
*
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 88a7a565170e..3beeb4c735f0 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
*/
int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
struct v4l2_buffer *b);
+/**
+ * vb2_delete_buf() - Delete the buffer from the queue
+ *
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @b: buffer structure passed from userspace to
+ * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
+ *
+ * The return values from this function are intended to be directly returned
+ * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
+ */
+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
/**
* vb2_qbuf() - Queue a buffer from userspace
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index aee75eb9e686..31bba1915642 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
#define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
#define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
+#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
+
/* Reminder: when adding new ioctls please add support for them to
drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../userspace-api/media/v4l/user-func.rst | 1 +
.../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
.../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
.../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
drivers/media/v4l2-core/v4l2-dev.c | 1 +
drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
include/media/v4l2-ioctl.h | 4 ++
include/media/videobuf2-core.h | 9 ++++
include/media/videobuf2-v4l2.h | 11 ++++
include/uapi/linux/videodev2.h | 2 +
10 files changed, 128 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst
index 15ff0bf7bbe6..8c74016e12fd 100644
--- a/Documentation/userspace-api/media/v4l/user-func.rst
+++ b/Documentation/userspace-api/media/v4l/user-func.rst
@@ -17,6 +17,7 @@ Function Reference
vidioc-dbg-g-chip-info
vidioc-dbg-g-register
vidioc-decoder-cmd
+ vidioc-delete-buf
vidioc-dqevent
vidioc-dv-timings-cap
vidioc-encoder-cmd
diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
new file mode 100644
index 000000000000..0e7ce58f91bc
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
@@ -0,0 +1,51 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+.. c:namespace:: V4L
+
+.. _VIDIOC_DELETE_BUF:
+
+************************
+ioctl VIDIOC_DELETE_BUF
+************************
+
+Name
+====
+
+VIDIOC_DELETE_BUF - Delete a buffer from a queue
+
+Synopsis
+========
+
+.. c:macro:: VIDIOC_DELETE_BUF
+
+``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
+
+Arguments
+=========
+
+``fd``
+ File descriptor returned by :c:func:`open()`.
+
+``argp``
+ Pointer to struct :c:type:`v4l2_buffer`.
+
+Description
+===========
+
+Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
+delete a buffer from a queue.
+
+The struct :c:type:`v4l2_buffer` structure is specified in
+:ref:`buffer`.
+
+Return Value
+============
+
+On success 0 is returned, on error -1 and the ``errno`` variable is set
+appropriately. The generic error codes are described at the
+:ref:`Generic Error Codes <gen-errors>` chapter.
+
+EBUSY
+ File I/O is in progress.
+
+EINVAL
+ The buffer ``index`` doesn't exist in the queue.
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 899783f67580..aa546c972c3d 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
}
EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
+{
+ struct vb2_buffer *vb;
+
+ vb = vb2_get_buffer(q, index);
+ if (!vb) {
+ dprintk(q, 1, "invalid buffer index %d\n", index);
+ return -EINVAL;
+ }
+
+ if (vb->state != VB2_BUF_STATE_DEQUEUED) {
+ dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
+ return -EINVAL;
+ }
+
+ if (vb->planes[0].mem_priv)
+ call_void_vb_qop(vb, buf_cleanup, vb);
+
+ /* Free MMAP buffers or release USERPTR buffers */
+ if (q->memory == VB2_MEMORY_MMAP)
+ __vb2_buf_mem_free(vb);
+ else if (q->memory == VB2_MEMORY_DMABUF)
+ __vb2_buf_dmabuf_put(vb);
+ else
+ __vb2_buf_userptr_put(vb);
+
+ vb2_queue_remove_buffer(q, vb);
+ kfree(vb);
+
+ dprintk(q, 2, "buffer %d deleted\n", index);
+ return 0;
+}
+
/*
* vb2_start_streaming() - Attempt to start streaming.
* @q: videobuf2 queue
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 724135d41f7f..cea666c17b41 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
}
EXPORT_SYMBOL_GPL(vb2_prepare_buf);
+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
+{
+ return vb2_core_delete_buf(q, b->index);
+}
+EXPORT_SYMBOL_GPL(vb2_delete_buf);
+
int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
{
unsigned requested_planes = 1;
diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
index f81279492682..80ace2e1e932 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device *vdev)
SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
+ SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF, vidioc_delete_buf);
}
if (is_vid || is_vbi || is_meta) {
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index a858acea6547..1c737279d3ef 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct v4l2_ioctl_ops *ops,
return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
}
+static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
+ struct file *file, void *fh, void *arg)
+{
+ struct v4l2_buffer *b = arg;
+ int ret = check_fmt(file, b->type);
+
+ return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
+}
+
static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
struct file *file, void *fh, void *arg)
{
@@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0),
IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
+ IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer, INFO_FL_QUEUE),
};
#define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
index edb733f21604..2f232ed884c7 100644
--- a/include/media/v4l2-ioctl.h
+++ b/include/media/v4l2-ioctl.h
@@ -163,6 +163,8 @@ struct v4l2_fh;
* :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
* @vidioc_prepare_buf: pointer to the function that implements
* :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
+ * @vidioc_delete_buf: pointer to the function that implements
+ * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
* @vidioc_overlay: pointer to the function that implements
* :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
* @vidioc_g_fbuf: pointer to the function that implements
@@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
struct v4l2_create_buffers *b);
int (*vidioc_prepare_buf)(struct file *file, void *fh,
struct v4l2_buffer *b);
+ int (*vidioc_delete_buf)(struct file *file, void *fh,
+ struct v4l2_buffer *b);
int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
int (*vidioc_g_fbuf)(struct file *file, void *fh,
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 080b783d608d..0f9e68f76b77 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
*/
int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
+/**
+ * vb2_core_delete_buf() -
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @index: id number of the buffer.
+ *
+ * Return: returns zero on success; an error code otherwise.
+ */
+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
+
/**
* vb2_core_qbuf() - Queue a buffer from userspace
*
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 88a7a565170e..3beeb4c735f0 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
*/
int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
struct v4l2_buffer *b);
+/**
+ * vb2_delete_buf() - Delete the buffer from the queue
+ *
+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
+ * @b: buffer structure passed from userspace to
+ * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
+ *
+ * The return values from this function are intended to be directly returned
+ * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
+ */
+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
/**
* vb2_qbuf() - Queue a buffer from userspace
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index aee75eb9e686..31bba1915642 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
#define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
#define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
+#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
+
/* Reminder: when adding new ioctls please add support for them to
drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 23:12 ` kernel test robot
-1 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-22 23:12 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: llvm, oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7]
[cannot apply to next-20230622]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-11-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
config: i386-buildonly-randconfig-r004-20230622 (https://download.01.org/0day-ci/archive/20230623/202306230742.KaUokotT-lkp@intel.com/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project.git 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce: (https://download.01.org/0day-ci/archive/20230623/202306230742.KaUokotT-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306230742.KaUokotT-lkp@intel.com/
All errors (new ones prefixed by >>, old ones prefixed by <<):
>> ERROR: modpost: "vb2_core_delete_buf" [drivers/media/common/videobuf2/videobuf2-v4l2.ko] undefined!
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-22 23:12 ` kernel test robot
0 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-22 23:12 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: llvm, oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7]
[cannot apply to next-20230622]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-11-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
config: i386-buildonly-randconfig-r004-20230622 (https://download.01.org/0day-ci/archive/20230623/202306230742.KaUokotT-lkp@intel.com/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project.git 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce: (https://download.01.org/0day-ci/archive/20230623/202306230742.KaUokotT-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306230742.KaUokotT-lkp@intel.com/
All errors (new ones prefixed by >>, old ones prefixed by <<):
>> ERROR: modpost: "vb2_core_delete_buf" [drivers/media/common/videobuf2/videobuf2-v4l2.ko] undefined!
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-22 23:12 ` kernel test robot
0 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-22 23:12 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: llvm, oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7]
[cannot apply to next-20230622]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-11-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
config: i386-buildonly-randconfig-r004-20230622 (https://download.01.org/0day-ci/archive/20230623/202306230742.KaUokotT-lkp@intel.com/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project.git 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce: (https://download.01.org/0day-ci/archive/20230623/202306230742.KaUokotT-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306230742.KaUokotT-lkp@intel.com/
All errors (new ones prefixed by >>, old ones prefixed by <<):
>> ERROR: modpost: "vb2_core_delete_buf" [drivers/media/common/videobuf2/videobuf2-v4l2.ko] undefined!
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-23 0:25 ` kernel test robot
-1 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-23 0:25 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7]
[cannot apply to next-20230622]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-11-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
config: i386-randconfig-i012-20230622 (https://download.01.org/0day-ci/archive/20230623/202306230822.0CCLmVt5-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230623/202306230822.0CCLmVt5-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306230822.0CCLmVt5-lkp@intel.com/
All errors (new ones prefixed by >>, old ones prefixed by <<):
>> ERROR: modpost: "vb2_core_delete_buf" [drivers/media/common/videobuf2/videobuf2-v4l2.ko] undefined!
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-23 0:25 ` kernel test robot
0 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-23 0:25 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7]
[cannot apply to next-20230622]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-11-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
config: i386-randconfig-i012-20230622 (https://download.01.org/0day-ci/archive/20230623/202306230822.0CCLmVt5-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230623/202306230822.0CCLmVt5-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306230822.0CCLmVt5-lkp@intel.com/
All errors (new ones prefixed by >>, old ones prefixed by <<):
>> ERROR: modpost: "vb2_core_delete_buf" [drivers/media/common/videobuf2/videobuf2-v4l2.ko] undefined!
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-23 0:25 ` kernel test robot
0 siblings, 0 replies; 183+ messages in thread
From: kernel test robot @ 2023-06-23 0:25 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ming.qian,
ezequiel, p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: oe-kbuild-all, linux-media, linux-kernel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, linux-staging,
kernel, Benjamin Gaignard
Hi Benjamin,
kernel test robot noticed the following build errors:
[auto build test ERROR on media-tree/master]
[also build test ERROR on linus/master v6.4-rc7]
[cannot apply to next-20230622]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Benjamin-Gaignard/media-videobuf2-Access-vb2_queue-bufs-array-through-helper-functions/20230622-214122
base: git://linuxtv.org/media_tree.git master
patch link: https://lore.kernel.org/r/20230622131349.144160-11-benjamin.gaignard%40collabora.com
patch subject: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
config: i386-randconfig-i012-20230622 (https://download.01.org/0day-ci/archive/20230623/202306230822.0CCLmVt5-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230623/202306230822.0CCLmVt5-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202306230822.0CCLmVt5-lkp@intel.com/
All errors (new ones prefixed by >>, old ones prefixed by <<):
>> ERROR: modpost: "vb2_core_delete_buf" [drivers/media/common/videobuf2/videobuf2-v4l2.ko] undefined!
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* RE: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-26 7:08 ` Ming Qian
-1 siblings, 0 replies; 183+ messages in thread
From: Ming Qian @ 2023-06-26 7:08 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ezequiel,
p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Hi Benjamin,
>-----Original Message-----
>From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>Sent: 2023年6月22日 21:14
>To: mchehab@kernel.org; tfiga@chromium.org; m.szyprowski@samsung.com;
>Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>cisco@xs4all.nl; nicolas.dufresne@collabora.com
>Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
><benjamin.gaignard@collabora.com>
>Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>
>Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>---
> .../userspace-api/media/v4l/user-func.rst | 1 +
> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> include/media/v4l2-ioctl.h | 4 ++
> include/media/videobuf2-core.h | 9 ++++
> include/media/videobuf2-v4l2.h | 11 ++++
> include/uapi/linux/videodev2.h | 2 +
> 10 files changed, 128 insertions(+)
> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>buf.rst
>
>diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>b/Documentation/userspace-api/media/v4l/user-func.rst
>index 15ff0bf7bbe6..8c74016e12fd 100644
>--- a/Documentation/userspace-api/media/v4l/user-func.rst
>+++ b/Documentation/userspace-api/media/v4l/user-func.rst
>@@ -17,6 +17,7 @@ Function Reference
> vidioc-dbg-g-chip-info
> vidioc-dbg-g-register
> vidioc-decoder-cmd
>+ vidioc-delete-buf
> vidioc-dqevent
> vidioc-dv-timings-cap
> vidioc-encoder-cmd
>diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>new file mode 100644
>index 000000000000..0e7ce58f91bc
>--- /dev/null
>+++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>@@ -0,0 +1,51 @@
>+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>+c:namespace:: V4L
>+
>+.. _VIDIOC_DELETE_BUF:
>+
>+************************
>+ioctl VIDIOC_DELETE_BUF
>+************************
>+
>+Name
>+====
>+
>+VIDIOC_DELETE_BUF - Delete a buffer from a queue
>+
>+Synopsis
>+========
>+
>+.. c:macro:: VIDIOC_DELETE_BUF
>+
>+``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>+
>+Arguments
>+=========
>+
>+``fd``
>+ File descriptor returned by :c:func:`open()`.
>+
>+``argp``
>+ Pointer to struct :c:type:`v4l2_buffer`.
>+
>+Description
>+===========
>+
>+Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>+delete a buffer from a queue.
>+
>+The struct :c:type:`v4l2_buffer` structure is specified in
>+:ref:`buffer`.
>+
>+Return Value
>+============
>+
>+On success 0 is returned, on error -1 and the ``errno`` variable is set
>+appropriately. The generic error codes are described at the
>+:ref:`Generic Error Codes <gen-errors>` chapter.
>+
>+EBUSY
>+ File I/O is in progress.
>+
>+EINVAL
>+ The buffer ``index`` doesn't exist in the queue.
>diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>b/drivers/media/common/videobuf2/videobuf2-core.c
>index 899783f67580..aa546c972c3d 100644
>--- a/drivers/media/common/videobuf2/videobuf2-core.c
>+++ b/drivers/media/common/videobuf2/videobuf2-core.c
>@@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>unsigned int index, void *pb) }
>EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>
>+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>+ struct vb2_buffer *vb;
>+
>+ vb = vb2_get_buffer(q, index);
>+ if (!vb) {
>+ dprintk(q, 1, "invalid buffer index %d\n", index);
>+ return -EINVAL;
>+ }
>+
>+ if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>+ dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
>+ return -EINVAL;
>+ }
>+
>+ if (vb->planes[0].mem_priv)
>+ call_void_vb_qop(vb, buf_cleanup, vb);
>+
>+ /* Free MMAP buffers or release USERPTR buffers */
>+ if (q->memory == VB2_MEMORY_MMAP)
>+ __vb2_buf_mem_free(vb);
>+ else if (q->memory == VB2_MEMORY_DMABUF)
>+ __vb2_buf_dmabuf_put(vb);
>+ else
>+ __vb2_buf_userptr_put(vb);
>+
>+ vb2_queue_remove_buffer(q, vb);
>+ kfree(vb);
The num_buffers is not changed, Is that on purpose?
num_buffers doesn't mean the number of vb2 buffers, but only decides the max index of allocated vb2 buffer?
Once the index is deleted, then it won't be used anymore, unless free the vb2 queue? is it a hole in vb2 queue?
And you can use xa_for_each() instead the for loop to iterate over the present buffers in vb2 queue.
>+
>+ dprintk(q, 2, "buffer %d deleted\n", index);
>+ return 0;
>+}
>+
> /*
> * vb2_start_streaming() - Attempt to start streaming.
> * @q: videobuf2 queue
>diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>index 724135d41f7f..cea666c17b41 100644
>--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>@@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>
>+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>+ return vb2_core_delete_buf(q, b->index); }
>+EXPORT_SYMBOL_GPL(vb2_delete_buf);
>+
> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
>{
> unsigned requested_planes = 1;
>diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>core/v4l2-dev.c
>index f81279492682..80ace2e1e932 100644
>--- a/drivers/media/v4l2-core/v4l2-dev.c
>+++ b/drivers/media/v4l2-core/v4l2-dev.c
>@@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device
>*vdev)
> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
>+ SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>+ vidioc_delete_buf);
> }
>
> if (is_vid || is_vbi || is_meta) { diff --git a/drivers/media/v4l2-core/v4l2-
>ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>index a858acea6547..1c737279d3ef 100644
>--- a/drivers/media/v4l2-core/v4l2-ioctl.c
>+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>@@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>v4l2_ioctl_ops *ops,
> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>
>+static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>+ struct file *file, void *fh, void *arg) {
>+ struct v4l2_buffer *b = arg;
>+ int ret = check_fmt(file, b->type);
>+
>+ return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>+
> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> struct file *file, void *fh, void *arg) { @@ -2905,6 +2914,7
>@@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>v4l_print_freq_band, 0),
> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>id)),
>+ IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>+ INFO_FL_QUEUE),
> };
> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>
>diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index
>edb733f21604..2f232ed884c7 100644
>--- a/include/media/v4l2-ioctl.h
>+++ b/include/media/v4l2-ioctl.h
>@@ -163,6 +163,8 @@ struct v4l2_fh;
> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> * @vidioc_prepare_buf: pointer to the function that implements
> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>+ * @vidioc_delete_buf: pointer to the function that implements
>+ * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> * @vidioc_overlay: pointer to the function that implements
> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6 +424,8
>@@ struct v4l2_ioctl_ops {
> struct v4l2_create_buffers *b);
> int (*vidioc_prepare_buf)(struct file *file, void *fh,
> struct v4l2_buffer *b);
>+ int (*vidioc_delete_buf)(struct file *file, void *fh,
>+ struct v4l2_buffer *b);
>
> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>080b783d608d..0f9e68f76b77 100644
>--- a/include/media/videobuf2-core.h
>+++ b/include/media/videobuf2-core.h
>@@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>enum vb2_memory memory,
> */
> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>
>+/**
>+ * vb2_core_delete_buf() -
>+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
>+ * @index: id number of the buffer.
>+ *
>+ * Return: returns zero on success; an error code otherwise.
>+ */
>+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>+
> /**
> * vb2_core_qbuf() - Queue a buffer from userspace
> *
>diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
>index 88a7a565170e..3beeb4c735f0 100644
>--- a/include/media/videobuf2-v4l2.h
>+++ b/include/media/videobuf2-v4l2.h
>@@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>v4l2_create_buffers *create);
> */
> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> struct v4l2_buffer *b);
>+/**
>+ * vb2_delete_buf() - Delete the buffer from the queue
>+ *
>+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
>+ * @b: buffer structure passed from userspace to
>+ * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>+ *
>+ * The return values from this function are intended to be directly
>+returned
>+ * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>+ */
>+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>
> /**
> * vb2_qbuf() - Queue a buffer from userspace diff --git
>a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>aee75eb9e686..31bba1915642 100644
>--- a/include/uapi/linux/videodev2.h
>+++ b/include/uapi/linux/videodev2.h
>@@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>
> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
>+#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>+
>
> /* Reminder: when adding new ioctls please add support for them to
> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>--
>2.39.2
^ permalink raw reply [flat|nested] 183+ messages in thread* RE: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-26 7:08 ` Ming Qian
0 siblings, 0 replies; 183+ messages in thread
From: Ming Qian @ 2023-06-26 7:08 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ezequiel,
p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Hi Benjamin,
>-----Original Message-----
>From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>Sent: 2023年6月22日 21:14
>To: mchehab@kernel.org; tfiga@chromium.org; m.szyprowski@samsung.com;
>Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>cisco@xs4all.nl; nicolas.dufresne@collabora.com
>Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
><benjamin.gaignard@collabora.com>
>Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>
>Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>---
> .../userspace-api/media/v4l/user-func.rst | 1 +
> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> include/media/v4l2-ioctl.h | 4 ++
> include/media/videobuf2-core.h | 9 ++++
> include/media/videobuf2-v4l2.h | 11 ++++
> include/uapi/linux/videodev2.h | 2 +
> 10 files changed, 128 insertions(+)
> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>buf.rst
>
>diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>b/Documentation/userspace-api/media/v4l/user-func.rst
>index 15ff0bf7bbe6..8c74016e12fd 100644
>--- a/Documentation/userspace-api/media/v4l/user-func.rst
>+++ b/Documentation/userspace-api/media/v4l/user-func.rst
>@@ -17,6 +17,7 @@ Function Reference
> vidioc-dbg-g-chip-info
> vidioc-dbg-g-register
> vidioc-decoder-cmd
>+ vidioc-delete-buf
> vidioc-dqevent
> vidioc-dv-timings-cap
> vidioc-encoder-cmd
>diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>new file mode 100644
>index 000000000000..0e7ce58f91bc
>--- /dev/null
>+++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>@@ -0,0 +1,51 @@
>+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>+c:namespace:: V4L
>+
>+.. _VIDIOC_DELETE_BUF:
>+
>+************************
>+ioctl VIDIOC_DELETE_BUF
>+************************
>+
>+Name
>+====
>+
>+VIDIOC_DELETE_BUF - Delete a buffer from a queue
>+
>+Synopsis
>+========
>+
>+.. c:macro:: VIDIOC_DELETE_BUF
>+
>+``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>+
>+Arguments
>+=========
>+
>+``fd``
>+ File descriptor returned by :c:func:`open()`.
>+
>+``argp``
>+ Pointer to struct :c:type:`v4l2_buffer`.
>+
>+Description
>+===========
>+
>+Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>+delete a buffer from a queue.
>+
>+The struct :c:type:`v4l2_buffer` structure is specified in
>+:ref:`buffer`.
>+
>+Return Value
>+============
>+
>+On success 0 is returned, on error -1 and the ``errno`` variable is set
>+appropriately. The generic error codes are described at the
>+:ref:`Generic Error Codes <gen-errors>` chapter.
>+
>+EBUSY
>+ File I/O is in progress.
>+
>+EINVAL
>+ The buffer ``index`` doesn't exist in the queue.
>diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>b/drivers/media/common/videobuf2/videobuf2-core.c
>index 899783f67580..aa546c972c3d 100644
>--- a/drivers/media/common/videobuf2/videobuf2-core.c
>+++ b/drivers/media/common/videobuf2/videobuf2-core.c
>@@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>unsigned int index, void *pb) }
>EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>
>+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>+ struct vb2_buffer *vb;
>+
>+ vb = vb2_get_buffer(q, index);
>+ if (!vb) {
>+ dprintk(q, 1, "invalid buffer index %d\n", index);
>+ return -EINVAL;
>+ }
>+
>+ if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>+ dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
>+ return -EINVAL;
>+ }
>+
>+ if (vb->planes[0].mem_priv)
>+ call_void_vb_qop(vb, buf_cleanup, vb);
>+
>+ /* Free MMAP buffers or release USERPTR buffers */
>+ if (q->memory == VB2_MEMORY_MMAP)
>+ __vb2_buf_mem_free(vb);
>+ else if (q->memory == VB2_MEMORY_DMABUF)
>+ __vb2_buf_dmabuf_put(vb);
>+ else
>+ __vb2_buf_userptr_put(vb);
>+
>+ vb2_queue_remove_buffer(q, vb);
>+ kfree(vb);
The num_buffers is not changed, Is that on purpose?
num_buffers doesn't mean the number of vb2 buffers, but only decides the max index of allocated vb2 buffer?
Once the index is deleted, then it won't be used anymore, unless free the vb2 queue? is it a hole in vb2 queue?
And you can use xa_for_each() instead the for loop to iterate over the present buffers in vb2 queue.
>+
>+ dprintk(q, 2, "buffer %d deleted\n", index);
>+ return 0;
>+}
>+
> /*
> * vb2_start_streaming() - Attempt to start streaming.
> * @q: videobuf2 queue
>diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>index 724135d41f7f..cea666c17b41 100644
>--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>@@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>
>+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>+ return vb2_core_delete_buf(q, b->index); }
>+EXPORT_SYMBOL_GPL(vb2_delete_buf);
>+
> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
>{
> unsigned requested_planes = 1;
>diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>core/v4l2-dev.c
>index f81279492682..80ace2e1e932 100644
>--- a/drivers/media/v4l2-core/v4l2-dev.c
>+++ b/drivers/media/v4l2-core/v4l2-dev.c
>@@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device
>*vdev)
> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
>+ SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>+ vidioc_delete_buf);
> }
>
> if (is_vid || is_vbi || is_meta) { diff --git a/drivers/media/v4l2-core/v4l2-
>ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>index a858acea6547..1c737279d3ef 100644
>--- a/drivers/media/v4l2-core/v4l2-ioctl.c
>+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>@@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>v4l2_ioctl_ops *ops,
> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>
>+static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>+ struct file *file, void *fh, void *arg) {
>+ struct v4l2_buffer *b = arg;
>+ int ret = check_fmt(file, b->type);
>+
>+ return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>+
> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> struct file *file, void *fh, void *arg) { @@ -2905,6 +2914,7
>@@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>v4l_print_freq_band, 0),
> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>id)),
>+ IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>+ INFO_FL_QUEUE),
> };
> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>
>diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index
>edb733f21604..2f232ed884c7 100644
>--- a/include/media/v4l2-ioctl.h
>+++ b/include/media/v4l2-ioctl.h
>@@ -163,6 +163,8 @@ struct v4l2_fh;
> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> * @vidioc_prepare_buf: pointer to the function that implements
> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>+ * @vidioc_delete_buf: pointer to the function that implements
>+ * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> * @vidioc_overlay: pointer to the function that implements
> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6 +424,8
>@@ struct v4l2_ioctl_ops {
> struct v4l2_create_buffers *b);
> int (*vidioc_prepare_buf)(struct file *file, void *fh,
> struct v4l2_buffer *b);
>+ int (*vidioc_delete_buf)(struct file *file, void *fh,
>+ struct v4l2_buffer *b);
>
> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>080b783d608d..0f9e68f76b77 100644
>--- a/include/media/videobuf2-core.h
>+++ b/include/media/videobuf2-core.h
>@@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>enum vb2_memory memory,
> */
> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>
>+/**
>+ * vb2_core_delete_buf() -
>+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
>+ * @index: id number of the buffer.
>+ *
>+ * Return: returns zero on success; an error code otherwise.
>+ */
>+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>+
> /**
> * vb2_core_qbuf() - Queue a buffer from userspace
> *
>diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
>index 88a7a565170e..3beeb4c735f0 100644
>--- a/include/media/videobuf2-v4l2.h
>+++ b/include/media/videobuf2-v4l2.h
>@@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>v4l2_create_buffers *create);
> */
> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> struct v4l2_buffer *b);
>+/**
>+ * vb2_delete_buf() - Delete the buffer from the queue
>+ *
>+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
>+ * @b: buffer structure passed from userspace to
>+ * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>+ *
>+ * The return values from this function are intended to be directly
>+returned
>+ * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>+ */
>+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>
> /**
> * vb2_qbuf() - Queue a buffer from userspace diff --git
>a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>aee75eb9e686..31bba1915642 100644
>--- a/include/uapi/linux/videodev2.h
>+++ b/include/uapi/linux/videodev2.h
>@@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>
> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
>+#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>+
>
> /* Reminder: when adding new ioctls please add support for them to
> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>--
>2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* RE: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-26 7:08 ` Ming Qian
0 siblings, 0 replies; 183+ messages in thread
From: Ming Qian @ 2023-06-26 7:08 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ezequiel,
p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Hi Benjamin,
>-----Original Message-----
>From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>Sent: 2023年6月22日 21:14
>To: mchehab@kernel.org; tfiga@chromium.org; m.szyprowski@samsung.com;
>Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>cisco@xs4all.nl; nicolas.dufresne@collabora.com
>Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
><benjamin.gaignard@collabora.com>
>Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>
>Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>---
> .../userspace-api/media/v4l/user-func.rst | 1 +
> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> include/media/v4l2-ioctl.h | 4 ++
> include/media/videobuf2-core.h | 9 ++++
> include/media/videobuf2-v4l2.h | 11 ++++
> include/uapi/linux/videodev2.h | 2 +
> 10 files changed, 128 insertions(+)
> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>buf.rst
>
>diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>b/Documentation/userspace-api/media/v4l/user-func.rst
>index 15ff0bf7bbe6..8c74016e12fd 100644
>--- a/Documentation/userspace-api/media/v4l/user-func.rst
>+++ b/Documentation/userspace-api/media/v4l/user-func.rst
>@@ -17,6 +17,7 @@ Function Reference
> vidioc-dbg-g-chip-info
> vidioc-dbg-g-register
> vidioc-decoder-cmd
>+ vidioc-delete-buf
> vidioc-dqevent
> vidioc-dv-timings-cap
> vidioc-encoder-cmd
>diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>new file mode 100644
>index 000000000000..0e7ce58f91bc
>--- /dev/null
>+++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>@@ -0,0 +1,51 @@
>+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>+c:namespace:: V4L
>+
>+.. _VIDIOC_DELETE_BUF:
>+
>+************************
>+ioctl VIDIOC_DELETE_BUF
>+************************
>+
>+Name
>+====
>+
>+VIDIOC_DELETE_BUF - Delete a buffer from a queue
>+
>+Synopsis
>+========
>+
>+.. c:macro:: VIDIOC_DELETE_BUF
>+
>+``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>+
>+Arguments
>+=========
>+
>+``fd``
>+ File descriptor returned by :c:func:`open()`.
>+
>+``argp``
>+ Pointer to struct :c:type:`v4l2_buffer`.
>+
>+Description
>+===========
>+
>+Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>+delete a buffer from a queue.
>+
>+The struct :c:type:`v4l2_buffer` structure is specified in
>+:ref:`buffer`.
>+
>+Return Value
>+============
>+
>+On success 0 is returned, on error -1 and the ``errno`` variable is set
>+appropriately. The generic error codes are described at the
>+:ref:`Generic Error Codes <gen-errors>` chapter.
>+
>+EBUSY
>+ File I/O is in progress.
>+
>+EINVAL
>+ The buffer ``index`` doesn't exist in the queue.
>diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>b/drivers/media/common/videobuf2/videobuf2-core.c
>index 899783f67580..aa546c972c3d 100644
>--- a/drivers/media/common/videobuf2/videobuf2-core.c
>+++ b/drivers/media/common/videobuf2/videobuf2-core.c
>@@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>unsigned int index, void *pb) }
>EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>
>+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>+ struct vb2_buffer *vb;
>+
>+ vb = vb2_get_buffer(q, index);
>+ if (!vb) {
>+ dprintk(q, 1, "invalid buffer index %d\n", index);
>+ return -EINVAL;
>+ }
>+
>+ if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>+ dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
>+ return -EINVAL;
>+ }
>+
>+ if (vb->planes[0].mem_priv)
>+ call_void_vb_qop(vb, buf_cleanup, vb);
>+
>+ /* Free MMAP buffers or release USERPTR buffers */
>+ if (q->memory == VB2_MEMORY_MMAP)
>+ __vb2_buf_mem_free(vb);
>+ else if (q->memory == VB2_MEMORY_DMABUF)
>+ __vb2_buf_dmabuf_put(vb);
>+ else
>+ __vb2_buf_userptr_put(vb);
>+
>+ vb2_queue_remove_buffer(q, vb);
>+ kfree(vb);
The num_buffers is not changed, Is that on purpose?
num_buffers doesn't mean the number of vb2 buffers, but only decides the max index of allocated vb2 buffer?
Once the index is deleted, then it won't be used anymore, unless free the vb2 queue? is it a hole in vb2 queue?
And you can use xa_for_each() instead the for loop to iterate over the present buffers in vb2 queue.
>+
>+ dprintk(q, 2, "buffer %d deleted\n", index);
>+ return 0;
>+}
>+
> /*
> * vb2_start_streaming() - Attempt to start streaming.
> * @q: videobuf2 queue
>diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>index 724135d41f7f..cea666c17b41 100644
>--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>@@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>
>+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>+ return vb2_core_delete_buf(q, b->index); }
>+EXPORT_SYMBOL_GPL(vb2_delete_buf);
>+
> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
>{
> unsigned requested_planes = 1;
>diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>core/v4l2-dev.c
>index f81279492682..80ace2e1e932 100644
>--- a/drivers/media/v4l2-core/v4l2-dev.c
>+++ b/drivers/media/v4l2-core/v4l2-dev.c
>@@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device
>*vdev)
> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
>+ SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>+ vidioc_delete_buf);
> }
>
> if (is_vid || is_vbi || is_meta) { diff --git a/drivers/media/v4l2-core/v4l2-
>ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>index a858acea6547..1c737279d3ef 100644
>--- a/drivers/media/v4l2-core/v4l2-ioctl.c
>+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>@@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>v4l2_ioctl_ops *ops,
> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>
>+static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>+ struct file *file, void *fh, void *arg) {
>+ struct v4l2_buffer *b = arg;
>+ int ret = check_fmt(file, b->type);
>+
>+ return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>+
> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> struct file *file, void *fh, void *arg) { @@ -2905,6 +2914,7
>@@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>v4l_print_freq_band, 0),
> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>id)),
>+ IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>+ INFO_FL_QUEUE),
> };
> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>
>diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index
>edb733f21604..2f232ed884c7 100644
>--- a/include/media/v4l2-ioctl.h
>+++ b/include/media/v4l2-ioctl.h
>@@ -163,6 +163,8 @@ struct v4l2_fh;
> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> * @vidioc_prepare_buf: pointer to the function that implements
> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>+ * @vidioc_delete_buf: pointer to the function that implements
>+ * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> * @vidioc_overlay: pointer to the function that implements
> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6 +424,8
>@@ struct v4l2_ioctl_ops {
> struct v4l2_create_buffers *b);
> int (*vidioc_prepare_buf)(struct file *file, void *fh,
> struct v4l2_buffer *b);
>+ int (*vidioc_delete_buf)(struct file *file, void *fh,
>+ struct v4l2_buffer *b);
>
> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>080b783d608d..0f9e68f76b77 100644
>--- a/include/media/videobuf2-core.h
>+++ b/include/media/videobuf2-core.h
>@@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>enum vb2_memory memory,
> */
> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>
>+/**
>+ * vb2_core_delete_buf() -
>+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
>+ * @index: id number of the buffer.
>+ *
>+ * Return: returns zero on success; an error code otherwise.
>+ */
>+int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>+
> /**
> * vb2_core_qbuf() - Queue a buffer from userspace
> *
>diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
>index 88a7a565170e..3beeb4c735f0 100644
>--- a/include/media/videobuf2-v4l2.h
>+++ b/include/media/videobuf2-v4l2.h
>@@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>v4l2_create_buffers *create);
> */
> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> struct v4l2_buffer *b);
>+/**
>+ * vb2_delete_buf() - Delete the buffer from the queue
>+ *
>+ * @q: pointer to &struct vb2_queue with videobuf2 queue.
>+ * @b: buffer structure passed from userspace to
>+ * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>+ *
>+ * The return values from this function are intended to be directly
>+returned
>+ * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>+ */
>+int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>
> /**
> * vb2_qbuf() - Queue a buffer from userspace diff --git
>a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>aee75eb9e686..31bba1915642 100644
>--- a/include/uapi/linux/videodev2.h
>+++ b/include/uapi/linux/videodev2.h
>@@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>
> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
>+#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>+
>
> /* Reminder: when adding new ioctls please add support for them to
> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>--
>2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-26 7:08 ` Ming Qian
(?)
@ 2023-06-26 7:48 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-26 7:48 UTC (permalink / raw)
To: Ming Qian, mchehab, tfiga, m.szyprowski, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Le 26/06/2023 à 09:08, Ming Qian a écrit :
> Hi Benjamin,
>
>> -----Original Message-----
>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> Sent: 2023年6月22日 21:14
>> To: mchehab@kernel.org; tfiga@chromium.org; m.szyprowski@samsung.com;
>> Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>> p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>> cisco@xs4all.nl; nicolas.dufresne@collabora.com
>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>> kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>> msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>> staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
>> <benjamin.gaignard@collabora.com>
>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>
>> Caution: This is an external email. Please take care when clicking links or
>> opening attachments. When in doubt, report the message using the 'Report
>> this email' button
>>
>>
>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> .../userspace-api/media/v4l/user-func.rst | 1 +
>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>> include/media/v4l2-ioctl.h | 4 ++
>> include/media/videobuf2-core.h | 9 ++++
>> include/media/videobuf2-v4l2.h | 11 ++++
>> include/uapi/linux/videodev2.h | 2 +
>> 10 files changed, 128 insertions(+)
>> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>> buf.rst
>>
>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>> b/Documentation/userspace-api/media/v4l/user-func.rst
>> index 15ff0bf7bbe6..8c74016e12fd 100644
>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>> @@ -17,6 +17,7 @@ Function Reference
>> vidioc-dbg-g-chip-info
>> vidioc-dbg-g-register
>> vidioc-decoder-cmd
>> + vidioc-delete-buf
>> vidioc-dqevent
>> vidioc-dv-timings-cap
>> vidioc-encoder-cmd
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> new file mode 100644
>> index 000000000000..0e7ce58f91bc
>> --- /dev/null
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> @@ -0,0 +1,51 @@
>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>> +c:namespace:: V4L
>> +
>> +.. _VIDIOC_DELETE_BUF:
>> +
>> +************************
>> +ioctl VIDIOC_DELETE_BUF
>> +************************
>> +
>> +Name
>> +====
>> +
>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>> +
>> +Synopsis
>> +========
>> +
>> +.. c:macro:: VIDIOC_DELETE_BUF
>> +
>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>> +
>> +Arguments
>> +=========
>> +
>> +``fd``
>> + File descriptor returned by :c:func:`open()`.
>> +
>> +``argp``
>> + Pointer to struct :c:type:`v4l2_buffer`.
>> +
>> +Description
>> +===========
>> +
>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>> +delete a buffer from a queue.
>> +
>> +The struct :c:type:`v4l2_buffer` structure is specified in
>> +:ref:`buffer`.
>> +
>> +Return Value
>> +============
>> +
>> +On success 0 is returned, on error -1 and the ``errno`` variable is set
>> +appropriately. The generic error codes are described at the
>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>> +
>> +EBUSY
>> + File I/O is in progress.
>> +
>> +EINVAL
>> + The buffer ``index`` doesn't exist in the queue.
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index 899783f67580..aa546c972c3d 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>> unsigned int index, void *pb) }
>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>> + struct vb2_buffer *vb;
>> +
>> + vb = vb2_get_buffer(q, index);
>> + if (!vb) {
>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>> + return -EINVAL;
>> + }
>> +
>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>> + dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
>> + return -EINVAL;
>> + }
>> +
>> + if (vb->planes[0].mem_priv)
>> + call_void_vb_qop(vb, buf_cleanup, vb);
>> +
>> + /* Free MMAP buffers or release USERPTR buffers */
>> + if (q->memory == VB2_MEMORY_MMAP)
>> + __vb2_buf_mem_free(vb);
>> + else if (q->memory == VB2_MEMORY_DMABUF)
>> + __vb2_buf_dmabuf_put(vb);
>> + else
>> + __vb2_buf_userptr_put(vb);
>> +
>> + vb2_queue_remove_buffer(q, vb);
>> + kfree(vb);
> The num_buffers is not changed, Is that on purpose?
> num_buffers doesn't mean the number of vb2 buffers, but only decides the max index of allocated vb2 buffer?
> Once the index is deleted, then it won't be used anymore, unless free the vb2 queue? is it a hole in vb2 queue?
Yes num_buffers isn't change and still represent the highest index of allocated buffer.
DELETE_BUF remove the buffer from the list and free the memory.
That could a create a hole in vb2 queue, that why the first patch of this series change
all places where vb2 queue array was used by calls to helpers function and also check the return value.
num_buffers is used as the lowest possible value when finding the free indexes for buffers (see vb2_queue_add_buffer())
this way I can guaranty that the indexes a continuous which is a requirement for create_bufs.
I choose this solution because XArray API doesn't offer way to find continuous free range.
It doesn't seem impossible to add it but this series is already big enough from my point of view.
Regards,
Benjamin
>
> And you can use xa_for_each() instead the for loop to iterate over the present buffers in vb2 queue.
>
>> +
>> + dprintk(q, 2, "buffer %d deleted\n", index);
>> + return 0;
>> +}
>> +
>> /*
>> * vb2_start_streaming() - Attempt to start streaming.
>> * @q: videobuf2 queue
>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> index 724135d41f7f..cea666c17b41 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>> + return vb2_core_delete_buf(q, b->index); }
>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>> +
>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
>> {
>> unsigned requested_planes = 1;
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>> core/v4l2-dev.c
>> index f81279492682..80ace2e1e932 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device
>> *vdev)
>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>> + vidioc_delete_buf);
>> }
>>
>> if (is_vid || is_vbi || is_meta) { diff --git a/drivers/media/v4l2-core/v4l2-
>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index a858acea6547..1c737279d3ef 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>> v4l2_ioctl_ops *ops,
>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>
>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>> + struct file *file, void *fh, void *arg) {
>> + struct v4l2_buffer *b = arg;
>> + int ret = check_fmt(file, b->type);
>> +
>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>> +
>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>> struct file *file, void *fh, void *arg) { @@ -2905,6 +2914,7
>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>> v4l_print_freq_band, 0),
>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>> v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>> id)),
>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>> + INFO_FL_QUEUE),
>> };
>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>
>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index
>> edb733f21604..2f232ed884c7 100644
>> --- a/include/media/v4l2-ioctl.h
>> +++ b/include/media/v4l2-ioctl.h
>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>> * @vidioc_prepare_buf: pointer to the function that implements
>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>> + * @vidioc_delete_buf: pointer to the function that implements
>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>> * @vidioc_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>> * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6 +424,8
>> @@ struct v4l2_ioctl_ops {
>> struct v4l2_create_buffers *b);
>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>> struct v4l2_buffer *b);
>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>> + struct v4l2_buffer *b);
>>
>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>> 080b783d608d..0f9e68f76b77 100644
>> --- a/include/media/videobuf2-core.h
>> +++ b/include/media/videobuf2-core.h
>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>> enum vb2_memory memory,
>> */
>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>>
>> +/**
>> + * vb2_core_delete_buf() -
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @index: id number of the buffer.
>> + *
>> + * Return: returns zero on success; an error code otherwise.
>> + */
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>> +
>> /**
>> * vb2_core_qbuf() - Queue a buffer from userspace
>> *
>> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
>> index 88a7a565170e..3beeb4c735f0 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>> v4l2_create_buffers *create);
>> */
>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>> struct v4l2_buffer *b);
>> +/**
>> + * vb2_delete_buf() - Delete the buffer from the queue
>> + *
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @b: buffer structure passed from userspace to
>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>> + *
>> + * The return values from this function are intended to be directly
>> +returned
>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>> + */
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>
>> /**
>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>> aee75eb9e686..31bba1915642 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>
>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>> +
>>
>> /* Reminder: when adding new ioctls please add support for them to
>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>> --
>> 2.39.2
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-26 7:48 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-26 7:48 UTC (permalink / raw)
To: Ming Qian, mchehab, tfiga, m.szyprowski, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Le 26/06/2023 à 09:08, Ming Qian a écrit :
> Hi Benjamin,
>
>> -----Original Message-----
>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> Sent: 2023年6月22日 21:14
>> To: mchehab@kernel.org; tfiga@chromium.org; m.szyprowski@samsung.com;
>> Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>> p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>> cisco@xs4all.nl; nicolas.dufresne@collabora.com
>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>> kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>> msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>> staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
>> <benjamin.gaignard@collabora.com>
>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>
>> Caution: This is an external email. Please take care when clicking links or
>> opening attachments. When in doubt, report the message using the 'Report
>> this email' button
>>
>>
>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> .../userspace-api/media/v4l/user-func.rst | 1 +
>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>> include/media/v4l2-ioctl.h | 4 ++
>> include/media/videobuf2-core.h | 9 ++++
>> include/media/videobuf2-v4l2.h | 11 ++++
>> include/uapi/linux/videodev2.h | 2 +
>> 10 files changed, 128 insertions(+)
>> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>> buf.rst
>>
>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>> b/Documentation/userspace-api/media/v4l/user-func.rst
>> index 15ff0bf7bbe6..8c74016e12fd 100644
>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>> @@ -17,6 +17,7 @@ Function Reference
>> vidioc-dbg-g-chip-info
>> vidioc-dbg-g-register
>> vidioc-decoder-cmd
>> + vidioc-delete-buf
>> vidioc-dqevent
>> vidioc-dv-timings-cap
>> vidioc-encoder-cmd
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> new file mode 100644
>> index 000000000000..0e7ce58f91bc
>> --- /dev/null
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> @@ -0,0 +1,51 @@
>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>> +c:namespace:: V4L
>> +
>> +.. _VIDIOC_DELETE_BUF:
>> +
>> +************************
>> +ioctl VIDIOC_DELETE_BUF
>> +************************
>> +
>> +Name
>> +====
>> +
>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>> +
>> +Synopsis
>> +========
>> +
>> +.. c:macro:: VIDIOC_DELETE_BUF
>> +
>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>> +
>> +Arguments
>> +=========
>> +
>> +``fd``
>> + File descriptor returned by :c:func:`open()`.
>> +
>> +``argp``
>> + Pointer to struct :c:type:`v4l2_buffer`.
>> +
>> +Description
>> +===========
>> +
>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>> +delete a buffer from a queue.
>> +
>> +The struct :c:type:`v4l2_buffer` structure is specified in
>> +:ref:`buffer`.
>> +
>> +Return Value
>> +============
>> +
>> +On success 0 is returned, on error -1 and the ``errno`` variable is set
>> +appropriately. The generic error codes are described at the
>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>> +
>> +EBUSY
>> + File I/O is in progress.
>> +
>> +EINVAL
>> + The buffer ``index`` doesn't exist in the queue.
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index 899783f67580..aa546c972c3d 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>> unsigned int index, void *pb) }
>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>> + struct vb2_buffer *vb;
>> +
>> + vb = vb2_get_buffer(q, index);
>> + if (!vb) {
>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>> + return -EINVAL;
>> + }
>> +
>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>> + dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
>> + return -EINVAL;
>> + }
>> +
>> + if (vb->planes[0].mem_priv)
>> + call_void_vb_qop(vb, buf_cleanup, vb);
>> +
>> + /* Free MMAP buffers or release USERPTR buffers */
>> + if (q->memory == VB2_MEMORY_MMAP)
>> + __vb2_buf_mem_free(vb);
>> + else if (q->memory == VB2_MEMORY_DMABUF)
>> + __vb2_buf_dmabuf_put(vb);
>> + else
>> + __vb2_buf_userptr_put(vb);
>> +
>> + vb2_queue_remove_buffer(q, vb);
>> + kfree(vb);
> The num_buffers is not changed, Is that on purpose?
> num_buffers doesn't mean the number of vb2 buffers, but only decides the max index of allocated vb2 buffer?
> Once the index is deleted, then it won't be used anymore, unless free the vb2 queue? is it a hole in vb2 queue?
Yes num_buffers isn't change and still represent the highest index of allocated buffer.
DELETE_BUF remove the buffer from the list and free the memory.
That could a create a hole in vb2 queue, that why the first patch of this series change
all places where vb2 queue array was used by calls to helpers function and also check the return value.
num_buffers is used as the lowest possible value when finding the free indexes for buffers (see vb2_queue_add_buffer())
this way I can guaranty that the indexes a continuous which is a requirement for create_bufs.
I choose this solution because XArray API doesn't offer way to find continuous free range.
It doesn't seem impossible to add it but this series is already big enough from my point of view.
Regards,
Benjamin
>
> And you can use xa_for_each() instead the for loop to iterate over the present buffers in vb2 queue.
>
>> +
>> + dprintk(q, 2, "buffer %d deleted\n", index);
>> + return 0;
>> +}
>> +
>> /*
>> * vb2_start_streaming() - Attempt to start streaming.
>> * @q: videobuf2 queue
>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> index 724135d41f7f..cea666c17b41 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>> + return vb2_core_delete_buf(q, b->index); }
>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>> +
>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
>> {
>> unsigned requested_planes = 1;
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>> core/v4l2-dev.c
>> index f81279492682..80ace2e1e932 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device
>> *vdev)
>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>> + vidioc_delete_buf);
>> }
>>
>> if (is_vid || is_vbi || is_meta) { diff --git a/drivers/media/v4l2-core/v4l2-
>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index a858acea6547..1c737279d3ef 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>> v4l2_ioctl_ops *ops,
>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>
>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>> + struct file *file, void *fh, void *arg) {
>> + struct v4l2_buffer *b = arg;
>> + int ret = check_fmt(file, b->type);
>> +
>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>> +
>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>> struct file *file, void *fh, void *arg) { @@ -2905,6 +2914,7
>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>> v4l_print_freq_band, 0),
>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>> v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>> id)),
>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>> + INFO_FL_QUEUE),
>> };
>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>
>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index
>> edb733f21604..2f232ed884c7 100644
>> --- a/include/media/v4l2-ioctl.h
>> +++ b/include/media/v4l2-ioctl.h
>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>> * @vidioc_prepare_buf: pointer to the function that implements
>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>> + * @vidioc_delete_buf: pointer to the function that implements
>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>> * @vidioc_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>> * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6 +424,8
>> @@ struct v4l2_ioctl_ops {
>> struct v4l2_create_buffers *b);
>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>> struct v4l2_buffer *b);
>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>> + struct v4l2_buffer *b);
>>
>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>> 080b783d608d..0f9e68f76b77 100644
>> --- a/include/media/videobuf2-core.h
>> +++ b/include/media/videobuf2-core.h
>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>> enum vb2_memory memory,
>> */
>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>>
>> +/**
>> + * vb2_core_delete_buf() -
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @index: id number of the buffer.
>> + *
>> + * Return: returns zero on success; an error code otherwise.
>> + */
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>> +
>> /**
>> * vb2_core_qbuf() - Queue a buffer from userspace
>> *
>> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
>> index 88a7a565170e..3beeb4c735f0 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>> v4l2_create_buffers *create);
>> */
>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>> struct v4l2_buffer *b);
>> +/**
>> + * vb2_delete_buf() - Delete the buffer from the queue
>> + *
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @b: buffer structure passed from userspace to
>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>> + *
>> + * The return values from this function are intended to be directly
>> +returned
>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>> + */
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>
>> /**
>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>> aee75eb9e686..31bba1915642 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>
>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>> +
>>
>> /* Reminder: when adding new ioctls please add support for them to
>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>> --
>> 2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-26 7:48 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-26 7:48 UTC (permalink / raw)
To: Ming Qian, mchehab, tfiga, m.szyprowski, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Le 26/06/2023 à 09:08, Ming Qian a écrit :
> Hi Benjamin,
>
>> -----Original Message-----
>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> Sent: 2023年6月22日 21:14
>> To: mchehab@kernel.org; tfiga@chromium.org; m.szyprowski@samsung.com;
>> Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>> p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>> cisco@xs4all.nl; nicolas.dufresne@collabora.com
>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>> kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>> msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>> staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
>> <benjamin.gaignard@collabora.com>
>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>
>> Caution: This is an external email. Please take care when clicking links or
>> opening attachments. When in doubt, report the message using the 'Report
>> this email' button
>>
>>
>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> .../userspace-api/media/v4l/user-func.rst | 1 +
>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>> include/media/v4l2-ioctl.h | 4 ++
>> include/media/videobuf2-core.h | 9 ++++
>> include/media/videobuf2-v4l2.h | 11 ++++
>> include/uapi/linux/videodev2.h | 2 +
>> 10 files changed, 128 insertions(+)
>> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>> buf.rst
>>
>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>> b/Documentation/userspace-api/media/v4l/user-func.rst
>> index 15ff0bf7bbe6..8c74016e12fd 100644
>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>> @@ -17,6 +17,7 @@ Function Reference
>> vidioc-dbg-g-chip-info
>> vidioc-dbg-g-register
>> vidioc-decoder-cmd
>> + vidioc-delete-buf
>> vidioc-dqevent
>> vidioc-dv-timings-cap
>> vidioc-encoder-cmd
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> new file mode 100644
>> index 000000000000..0e7ce58f91bc
>> --- /dev/null
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> @@ -0,0 +1,51 @@
>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>> +c:namespace:: V4L
>> +
>> +.. _VIDIOC_DELETE_BUF:
>> +
>> +************************
>> +ioctl VIDIOC_DELETE_BUF
>> +************************
>> +
>> +Name
>> +====
>> +
>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>> +
>> +Synopsis
>> +========
>> +
>> +.. c:macro:: VIDIOC_DELETE_BUF
>> +
>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>> +
>> +Arguments
>> +=========
>> +
>> +``fd``
>> + File descriptor returned by :c:func:`open()`.
>> +
>> +``argp``
>> + Pointer to struct :c:type:`v4l2_buffer`.
>> +
>> +Description
>> +===========
>> +
>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>> +delete a buffer from a queue.
>> +
>> +The struct :c:type:`v4l2_buffer` structure is specified in
>> +:ref:`buffer`.
>> +
>> +Return Value
>> +============
>> +
>> +On success 0 is returned, on error -1 and the ``errno`` variable is set
>> +appropriately. The generic error codes are described at the
>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>> +
>> +EBUSY
>> + File I/O is in progress.
>> +
>> +EINVAL
>> + The buffer ``index`` doesn't exist in the queue.
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index 899783f67580..aa546c972c3d 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>> unsigned int index, void *pb) }
>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>> + struct vb2_buffer *vb;
>> +
>> + vb = vb2_get_buffer(q, index);
>> + if (!vb) {
>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>> + return -EINVAL;
>> + }
>> +
>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>> + dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
>> + return -EINVAL;
>> + }
>> +
>> + if (vb->planes[0].mem_priv)
>> + call_void_vb_qop(vb, buf_cleanup, vb);
>> +
>> + /* Free MMAP buffers or release USERPTR buffers */
>> + if (q->memory == VB2_MEMORY_MMAP)
>> + __vb2_buf_mem_free(vb);
>> + else if (q->memory == VB2_MEMORY_DMABUF)
>> + __vb2_buf_dmabuf_put(vb);
>> + else
>> + __vb2_buf_userptr_put(vb);
>> +
>> + vb2_queue_remove_buffer(q, vb);
>> + kfree(vb);
> The num_buffers is not changed, Is that on purpose?
> num_buffers doesn't mean the number of vb2 buffers, but only decides the max index of allocated vb2 buffer?
> Once the index is deleted, then it won't be used anymore, unless free the vb2 queue? is it a hole in vb2 queue?
Yes num_buffers isn't change and still represent the highest index of allocated buffer.
DELETE_BUF remove the buffer from the list and free the memory.
That could a create a hole in vb2 queue, that why the first patch of this series change
all places where vb2 queue array was used by calls to helpers function and also check the return value.
num_buffers is used as the lowest possible value when finding the free indexes for buffers (see vb2_queue_add_buffer())
this way I can guaranty that the indexes a continuous which is a requirement for create_bufs.
I choose this solution because XArray API doesn't offer way to find continuous free range.
It doesn't seem impossible to add it but this series is already big enough from my point of view.
Regards,
Benjamin
>
> And you can use xa_for_each() instead the for loop to iterate over the present buffers in vb2 queue.
>
>> +
>> + dprintk(q, 2, "buffer %d deleted\n", index);
>> + return 0;
>> +}
>> +
>> /*
>> * vb2_start_streaming() - Attempt to start streaming.
>> * @q: videobuf2 queue
>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> index 724135d41f7f..cea666c17b41 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>> + return vb2_core_delete_buf(q, b->index); }
>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>> +
>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
>> {
>> unsigned requested_planes = 1;
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>> core/v4l2-dev.c
>> index f81279492682..80ace2e1e932 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device
>> *vdev)
>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>> + vidioc_delete_buf);
>> }
>>
>> if (is_vid || is_vbi || is_meta) { diff --git a/drivers/media/v4l2-core/v4l2-
>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index a858acea6547..1c737279d3ef 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>> v4l2_ioctl_ops *ops,
>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>
>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>> + struct file *file, void *fh, void *arg) {
>> + struct v4l2_buffer *b = arg;
>> + int ret = check_fmt(file, b->type);
>> +
>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>> +
>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>> struct file *file, void *fh, void *arg) { @@ -2905,6 +2914,7
>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>> v4l_print_freq_band, 0),
>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>> v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>> id)),
>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>> + INFO_FL_QUEUE),
>> };
>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>
>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index
>> edb733f21604..2f232ed884c7 100644
>> --- a/include/media/v4l2-ioctl.h
>> +++ b/include/media/v4l2-ioctl.h
>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>> * @vidioc_prepare_buf: pointer to the function that implements
>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>> + * @vidioc_delete_buf: pointer to the function that implements
>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>> * @vidioc_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>> * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6 +424,8
>> @@ struct v4l2_ioctl_ops {
>> struct v4l2_create_buffers *b);
>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>> struct v4l2_buffer *b);
>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>> + struct v4l2_buffer *b);
>>
>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>> 080b783d608d..0f9e68f76b77 100644
>> --- a/include/media/videobuf2-core.h
>> +++ b/include/media/videobuf2-core.h
>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>> enum vb2_memory memory,
>> */
>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>>
>> +/**
>> + * vb2_core_delete_buf() -
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @index: id number of the buffer.
>> + *
>> + * Return: returns zero on success; an error code otherwise.
>> + */
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>> +
>> /**
>> * vb2_core_qbuf() - Queue a buffer from userspace
>> *
>> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
>> index 88a7a565170e..3beeb4c735f0 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>> v4l2_create_buffers *create);
>> */
>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>> struct v4l2_buffer *b);
>> +/**
>> + * vb2_delete_buf() - Delete the buffer from the queue
>> + *
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @b: buffer structure passed from userspace to
>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>> + *
>> + * The return values from this function are intended to be directly
>> +returned
>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>> + */
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>
>> /**
>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>> aee75eb9e686..31bba1915642 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>
>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>> +
>>
>> /* Reminder: when adding new ioctls please add support for them to
>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>> --
>> 2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-26 7:48 ` Benjamin Gaignard
(?)
@ 2023-06-26 7:50 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-26 7:50 UTC (permalink / raw)
To: Ming Qian, mchehab, tfiga, m.szyprowski, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Le 26/06/2023 à 09:48, Benjamin Gaignard a écrit :
>
> Le 26/06/2023 à 09:08, Ming Qian a écrit :
>> Hi Benjamin,
>>
>>> -----Original Message-----
>>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> Sent: 2023年6月22日 21:14
>>> To: mchehab@kernel.org; tfiga@chromium.org; m.szyprowski@samsung.com;
>>> Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>>> p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>>> cisco@xs4all.nl; nicolas.dufresne@collabora.com
>>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>>> linux-arm-
>>> kernel@lists.infradead.org; linux-mediatek@lists.infradead.org;
>>> linux-arm-
>>> msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>>> staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
>>> <benjamin.gaignard@collabora.com>
>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>
>>> Caution: This is an external email. Please take care when clicking
>>> links or
>>> opening attachments. When in doubt, report the message using the
>>> 'Report
>>> this email' button
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>> include/media/v4l2-ioctl.h | 4 ++
>>> include/media/videobuf2-core.h | 9 ++++
>>> include/media/videobuf2-v4l2.h | 11 ++++
>>> include/uapi/linux/videodev2.h | 2 +
>>> 10 files changed, 128 insertions(+)
>>> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>>> buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>> vidioc-dbg-g-chip-info
>>> vidioc-dbg-g-register
>>> vidioc-decoder-cmd
>>> + vidioc-delete-buf
>>> vidioc-dqevent
>>> vidioc-dv-timings-cap
>>> vidioc-encoder-cmd
>>> diff --git
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>> +c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> + File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>>> +delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>> set
>>> +appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> + File I/O is in progress.
>>> +
>>> +EINVAL
>>> + The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb) }
>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>> + struct vb2_buffer *vb;
>>> +
>>> + vb = vb2_get_buffer(q, index);
>>> + if (!vb) {
>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>> index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->planes[0].mem_priv)
>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> + /* Free MMAP buffers or release USERPTR buffers */
>>> + if (q->memory == VB2_MEMORY_MMAP)
>>> + __vb2_buf_mem_free(vb);
>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>> + __vb2_buf_dmabuf_put(vb);
>>> + else
>>> + __vb2_buf_userptr_put(vb);
>>> +
>>> + vb2_queue_remove_buffer(q, vb);
>>> + kfree(vb);
>> The num_buffers is not changed, Is that on purpose?
>> num_buffers doesn't mean the number of vb2 buffers, but only decides
>> the max index of allocated vb2 buffer?
>> Once the index is deleted, then it won't be used anymore, unless free
>> the vb2 queue? is it a hole in vb2 queue?
>
> Yes num_buffers isn't change and still represent the highest index of
> allocated buffer.
> DELETE_BUF remove the buffer from the list and free the memory.
> That could a create a hole in vb2 queue, that why the first patch of
> this series change
> all places where vb2 queue array was used by calls to helpers function
> and also check the return value.
> num_buffers is used as the lowest possible value when finding the free
> indexes for buffers (see vb2_queue_add_buffer())
> this way I can guaranty that the indexes a continuous which is a
> requirement for create_bufs.
> I choose this solution because XArray API doesn't offer way to find
> continuous free range.
> It doesn't seem impossible to add it but this series is already big
> enough from my point of view.
>
> Regards,
> Benjamin
>
>>
>> And you can use xa_for_each() instead the for loop to iterate over
>> the present buffers in vb2 queue.
Sorry I forgot this comment.
Where do you think I can improve the code ?
Thanks,
Benjamin
>>
>>> +
>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>> + return 0;
>>> +}
>>> +
>>> /*
>>> * vb2_start_streaming() - Attempt to start streaming.
>>> * @q: videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>> + return vb2_core_delete_buf(q, b->index); }
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>> *create)
>>> {
>>> unsigned requested_planes = 1;
>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>> core/v4l2-dev.c
>>> index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>> video_device
>>> *vdev)
>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>> vidioc_prepare_buf);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>> vidioc_streamoff);
>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> + vidioc_delete_buf);
>>> }
>>>
>>> if (is_vid || is_vbi || is_meta) { diff --git
>>> a/drivers/media/v4l2-core/v4l2-
>>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> + struct file *file, void *fh, void *arg) {
>>> + struct v4l2_buffer *b = arg;
>>> + int ret = check_fmt(file, b->type);
>>> +
>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>> +
>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>> struct file *file, void *fh, void
>>> *arg) { @@ -2905,6 +2914,7
>>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>> id)),
>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>>> + INFO_FL_QUEUE),
>>> };
>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>> index
>>> edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>> * @vidioc_prepare_buf: pointer to the function that implements
>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>> * @vidioc_overlay: pointer to the function that implements
>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>> * @vidioc_g_fbuf: pointer to the function that implements @@
>>> -422,6 +424,8
>>> @@ struct v4l2_ioctl_ops {
>>> struct v4l2_create_buffers *b);
>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>> struct v4l2_buffer *b);
>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> + struct v4l2_buffer *b);
>>>
>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>> int i);
>>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>>> 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>> */
>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index: id number of the buffer.
>>> + *
>>> + * Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>> /**
>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>> *
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h
>>> index 88a7a565170e..3beeb4c735f0 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>> */
>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>> struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b: buffer structure passed from userspace to
>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> +returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>> /**
>>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>>> aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>
>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>> /* Reminder: when adding new ioctls please add support for them to
>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> --
>>> 2.39.2
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-26 7:50 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-26 7:50 UTC (permalink / raw)
To: Ming Qian, mchehab, tfiga, m.szyprowski, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Le 26/06/2023 à 09:48, Benjamin Gaignard a écrit :
>
> Le 26/06/2023 à 09:08, Ming Qian a écrit :
>> Hi Benjamin,
>>
>>> -----Original Message-----
>>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> Sent: 2023年6月22日 21:14
>>> To: mchehab@kernel.org; tfiga@chromium.org; m.szyprowski@samsung.com;
>>> Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>>> p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>>> cisco@xs4all.nl; nicolas.dufresne@collabora.com
>>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>>> linux-arm-
>>> kernel@lists.infradead.org; linux-mediatek@lists.infradead.org;
>>> linux-arm-
>>> msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>>> staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
>>> <benjamin.gaignard@collabora.com>
>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>
>>> Caution: This is an external email. Please take care when clicking
>>> links or
>>> opening attachments. When in doubt, report the message using the
>>> 'Report
>>> this email' button
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>> include/media/v4l2-ioctl.h | 4 ++
>>> include/media/videobuf2-core.h | 9 ++++
>>> include/media/videobuf2-v4l2.h | 11 ++++
>>> include/uapi/linux/videodev2.h | 2 +
>>> 10 files changed, 128 insertions(+)
>>> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>>> buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>> vidioc-dbg-g-chip-info
>>> vidioc-dbg-g-register
>>> vidioc-decoder-cmd
>>> + vidioc-delete-buf
>>> vidioc-dqevent
>>> vidioc-dv-timings-cap
>>> vidioc-encoder-cmd
>>> diff --git
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>> +c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> + File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>>> +delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>> set
>>> +appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> + File I/O is in progress.
>>> +
>>> +EINVAL
>>> + The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb) }
>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>> + struct vb2_buffer *vb;
>>> +
>>> + vb = vb2_get_buffer(q, index);
>>> + if (!vb) {
>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>> index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->planes[0].mem_priv)
>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> + /* Free MMAP buffers or release USERPTR buffers */
>>> + if (q->memory == VB2_MEMORY_MMAP)
>>> + __vb2_buf_mem_free(vb);
>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>> + __vb2_buf_dmabuf_put(vb);
>>> + else
>>> + __vb2_buf_userptr_put(vb);
>>> +
>>> + vb2_queue_remove_buffer(q, vb);
>>> + kfree(vb);
>> The num_buffers is not changed, Is that on purpose?
>> num_buffers doesn't mean the number of vb2 buffers, but only decides
>> the max index of allocated vb2 buffer?
>> Once the index is deleted, then it won't be used anymore, unless free
>> the vb2 queue? is it a hole in vb2 queue?
>
> Yes num_buffers isn't change and still represent the highest index of
> allocated buffer.
> DELETE_BUF remove the buffer from the list and free the memory.
> That could a create a hole in vb2 queue, that why the first patch of
> this series change
> all places where vb2 queue array was used by calls to helpers function
> and also check the return value.
> num_buffers is used as the lowest possible value when finding the free
> indexes for buffers (see vb2_queue_add_buffer())
> this way I can guaranty that the indexes a continuous which is a
> requirement for create_bufs.
> I choose this solution because XArray API doesn't offer way to find
> continuous free range.
> It doesn't seem impossible to add it but this series is already big
> enough from my point of view.
>
> Regards,
> Benjamin
>
>>
>> And you can use xa_for_each() instead the for loop to iterate over
>> the present buffers in vb2 queue.
Sorry I forgot this comment.
Where do you think I can improve the code ?
Thanks,
Benjamin
>>
>>> +
>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>> + return 0;
>>> +}
>>> +
>>> /*
>>> * vb2_start_streaming() - Attempt to start streaming.
>>> * @q: videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>> + return vb2_core_delete_buf(q, b->index); }
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>> *create)
>>> {
>>> unsigned requested_planes = 1;
>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>> core/v4l2-dev.c
>>> index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>> video_device
>>> *vdev)
>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>> vidioc_prepare_buf);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>> vidioc_streamoff);
>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> + vidioc_delete_buf);
>>> }
>>>
>>> if (is_vid || is_vbi || is_meta) { diff --git
>>> a/drivers/media/v4l2-core/v4l2-
>>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> + struct file *file, void *fh, void *arg) {
>>> + struct v4l2_buffer *b = arg;
>>> + int ret = check_fmt(file, b->type);
>>> +
>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>> +
>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>> struct file *file, void *fh, void
>>> *arg) { @@ -2905,6 +2914,7
>>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>> id)),
>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>>> + INFO_FL_QUEUE),
>>> };
>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>> index
>>> edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>> * @vidioc_prepare_buf: pointer to the function that implements
>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>> * @vidioc_overlay: pointer to the function that implements
>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>> * @vidioc_g_fbuf: pointer to the function that implements @@
>>> -422,6 +424,8
>>> @@ struct v4l2_ioctl_ops {
>>> struct v4l2_create_buffers *b);
>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>> struct v4l2_buffer *b);
>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> + struct v4l2_buffer *b);
>>>
>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>> int i);
>>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>>> 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>> */
>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index: id number of the buffer.
>>> + *
>>> + * Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>> /**
>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>> *
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h
>>> index 88a7a565170e..3beeb4c735f0 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>> */
>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>> struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b: buffer structure passed from userspace to
>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> +returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>> /**
>>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>>> aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>
>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>> /* Reminder: when adding new ioctls please add support for them to
>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> --
>>> 2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-26 7:50 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-26 7:50 UTC (permalink / raw)
To: Ming Qian, mchehab, tfiga, m.szyprowski, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
Le 26/06/2023 à 09:48, Benjamin Gaignard a écrit :
>
> Le 26/06/2023 à 09:08, Ming Qian a écrit :
>> Hi Benjamin,
>>
>>> -----Original Message-----
>>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> Sent: 2023年6月22日 21:14
>>> To: mchehab@kernel.org; tfiga@chromium.org; m.szyprowski@samsung.com;
>>> Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>>> p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>>> cisco@xs4all.nl; nicolas.dufresne@collabora.com
>>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>>> linux-arm-
>>> kernel@lists.infradead.org; linux-mediatek@lists.infradead.org;
>>> linux-arm-
>>> msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>>> staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
>>> <benjamin.gaignard@collabora.com>
>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>
>>> Caution: This is an external email. Please take care when clicking
>>> links or
>>> opening attachments. When in doubt, report the message using the
>>> 'Report
>>> this email' button
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>> include/media/v4l2-ioctl.h | 4 ++
>>> include/media/videobuf2-core.h | 9 ++++
>>> include/media/videobuf2-v4l2.h | 11 ++++
>>> include/uapi/linux/videodev2.h | 2 +
>>> 10 files changed, 128 insertions(+)
>>> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-
>>> buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>> vidioc-dbg-g-chip-info
>>> vidioc-dbg-g-register
>>> vidioc-decoder-cmd
>>> + vidioc-delete-buf
>>> vidioc-dqevent
>>> vidioc-dv-timings-cap
>>> vidioc-encoder-cmd
>>> diff --git
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>> +c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> + File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>>> +delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>> set
>>> +appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> + File I/O is in progress.
>>> +
>>> +EINVAL
>>> + The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb) }
>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>> + struct vb2_buffer *vb;
>>> +
>>> + vb = vb2_get_buffer(q, index);
>>> + if (!vb) {
>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>> index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->planes[0].mem_priv)
>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> + /* Free MMAP buffers or release USERPTR buffers */
>>> + if (q->memory == VB2_MEMORY_MMAP)
>>> + __vb2_buf_mem_free(vb);
>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>> + __vb2_buf_dmabuf_put(vb);
>>> + else
>>> + __vb2_buf_userptr_put(vb);
>>> +
>>> + vb2_queue_remove_buffer(q, vb);
>>> + kfree(vb);
>> The num_buffers is not changed, Is that on purpose?
>> num_buffers doesn't mean the number of vb2 buffers, but only decides
>> the max index of allocated vb2 buffer?
>> Once the index is deleted, then it won't be used anymore, unless free
>> the vb2 queue? is it a hole in vb2 queue?
>
> Yes num_buffers isn't change and still represent the highest index of
> allocated buffer.
> DELETE_BUF remove the buffer from the list and free the memory.
> That could a create a hole in vb2 queue, that why the first patch of
> this series change
> all places where vb2 queue array was used by calls to helpers function
> and also check the return value.
> num_buffers is used as the lowest possible value when finding the free
> indexes for buffers (see vb2_queue_add_buffer())
> this way I can guaranty that the indexes a continuous which is a
> requirement for create_bufs.
> I choose this solution because XArray API doesn't offer way to find
> continuous free range.
> It doesn't seem impossible to add it but this series is already big
> enough from my point of view.
>
> Regards,
> Benjamin
>
>>
>> And you can use xa_for_each() instead the for loop to iterate over
>> the present buffers in vb2 queue.
Sorry I forgot this comment.
Where do you think I can improve the code ?
Thanks,
Benjamin
>>
>>> +
>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>> + return 0;
>>> +}
>>> +
>>> /*
>>> * vb2_start_streaming() - Attempt to start streaming.
>>> * @q: videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>> + return vb2_core_delete_buf(q, b->index); }
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>> *create)
>>> {
>>> unsigned requested_planes = 1;
>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>> core/v4l2-dev.c
>>> index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>> video_device
>>> *vdev)
>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>> vidioc_prepare_buf);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>> vidioc_streamoff);
>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> + vidioc_delete_buf);
>>> }
>>>
>>> if (is_vid || is_vbi || is_meta) { diff --git
>>> a/drivers/media/v4l2-core/v4l2-
>>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> + struct file *file, void *fh, void *arg) {
>>> + struct v4l2_buffer *b = arg;
>>> + int ret = check_fmt(file, b->type);
>>> +
>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>> +
>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>> struct file *file, void *fh, void
>>> *arg) { @@ -2905,6 +2914,7
>>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>> id)),
>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer,
>>> + INFO_FL_QUEUE),
>>> };
>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>> index
>>> edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>> * @vidioc_prepare_buf: pointer to the function that implements
>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>> * @vidioc_overlay: pointer to the function that implements
>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>> * @vidioc_g_fbuf: pointer to the function that implements @@
>>> -422,6 +424,8
>>> @@ struct v4l2_ioctl_ops {
>>> struct v4l2_create_buffers *b);
>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>> struct v4l2_buffer *b);
>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> + struct v4l2_buffer *b);
>>>
>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>> int i);
>>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h index
>>> 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>> */
>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index: id number of the buffer.
>>> + *
>>> + * Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>> /**
>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>> *
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h
>>> index 88a7a565170e..3beeb4c735f0 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>> */
>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>> struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b: buffer structure passed from userspace to
>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> +returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>> /**
>>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index
>>> aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>
>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>> /* Reminder: when adding new ioctls please add support for them to
>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> --
>>> 2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* RE: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-26 7:50 ` Benjamin Gaignard
(?)
@ 2023-06-26 8:13 ` Ming Qian
-1 siblings, 0 replies; 183+ messages in thread
From: Ming Qian @ 2023-06-26 8:13 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ezequiel,
p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
>From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>Sent: 2023年6月26日 15:50
>To: Ming Qian <ming.qian@nxp.com>; mchehab@kernel.org;
>tfiga@chromium.org; m.szyprowski@samsung.com;
>ezequiel@vanguardiasur.com.ar; p.zabel@pengutronix.de;
>gregkh@linuxfoundation.org; hverkuil-cisco@xs4all.nl;
>nicolas.dufresne@collabora.com
>Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>staging@lists.linux.dev; kernel@collabora.com
>Subject: Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>Le 26/06/2023 à 09:48, Benjamin Gaignard a écrit :
>>
>> Le 26/06/2023 à 09:08, Ming Qian a écrit :
>>> Hi Benjamin,
>>>
>>>> -----Original Message-----
>>>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> Sent: 2023年6月22日 21:14
>>>> To: mchehab@kernel.org; tfiga@chromium.org;
>>>> m.szyprowski@samsung.com; Ming Qian <ming.qian@nxp.com>;
>>>> ezequiel@vanguardiasur.com.ar; p.zabel@pengutronix.de;
>>>> gregkh@linuxfoundation.org; hverkuil- cisco@xs4all.nl;
>>>> nicolas.dufresne@collabora.com
>>>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>>>> linux-arm-
>>>> kernel@lists.infradead.org; linux-mediatek@lists.infradead.org;
>>>> linux-arm-
>>>> msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>>>> staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
>>>> <benjamin.gaignard@collabora.com>
>>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>>
>>>> Caution: This is an external email. Please take care when clicking
>>>> links or opening attachments. When in doubt, report the message
>>>> using the 'Report this email' button
>>>>
>>>>
>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> ---
>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>> include/media/v4l2-ioctl.h | 4 ++
>>>> include/media/videobuf2-core.h | 9 ++++
>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>> include/uapi/linux/videodev2.h | 2 +
>>>> 10 files changed, 128 insertions(+)
>>>> create mode 100644
>>>> Documentation/userspace-api/media/v4l/vidioc-delete-
>>>> buf.rst
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> @@ -17,6 +17,7 @@ Function Reference
>>>> vidioc-dbg-g-chip-info
>>>> vidioc-dbg-g-register
>>>> vidioc-decoder-cmd
>>>> + vidioc-delete-buf
>>>> vidioc-dqevent
>>>> vidioc-dv-timings-cap
>>>> vidioc-encoder-cmd
>>>> diff --git
>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> new file mode 100644
>>>> index 000000000000..0e7ce58f91bc
>>>> --- /dev/null
>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> @@ -0,0 +1,51 @@
>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>>> +c:namespace:: V4L
>>>> +
>>>> +.. _VIDIOC_DELETE_BUF:
>>>> +
>>>> +************************
>>>> +ioctl VIDIOC_DELETE_BUF
>>>> +************************
>>>> +
>>>> +Name
>>>> +====
>>>> +
>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>> +
>>>> +Synopsis
>>>> +========
>>>> +
>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>> +
>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>> +
>>>> +Arguments
>>>> +=========
>>>> +
>>>> +``fd``
>>>> + File descriptor returned by :c:func:`open()`.
>>>> +
>>>> +``argp``
>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>> +
>>>> +Description
>>>> +===========
>>>> +
>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl
>>>> +to delete a buffer from a queue.
>>>> +
>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>> +:ref:`buffer`.
>>>> +
>>>> +Return Value
>>>> +============
>>>> +
>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>> set
>>>> +appropriately. The generic error codes are described at the
>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>> +
>>>> +EBUSY
>>>> + File I/O is in progress.
>>>> +
>>>> +EINVAL
>>>> + The buffer ``index`` doesn't exist in the queue.
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index 899783f67580..aa546c972c3d 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>*q,
>>>> unsigned int index, void *pb) }
>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>>> + struct vb2_buffer *vb;
>>>> +
>>>> + vb = vb2_get_buffer(q, index);
>>>> + if (!vb) {
>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>> index %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + if (vb->planes[0].mem_priv)
>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>> +
>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>> + __vb2_buf_mem_free(vb);
>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>> + __vb2_buf_dmabuf_put(vb);
>>>> + else
>>>> + __vb2_buf_userptr_put(vb);
>>>> +
>>>> + vb2_queue_remove_buffer(q, vb);
>>>> + kfree(vb);
>>> The num_buffers is not changed, Is that on purpose?
>>> num_buffers doesn't mean the number of vb2 buffers, but only decides
>>> the max index of allocated vb2 buffer?
>>> Once the index is deleted, then it won't be used anymore, unless free
>>> the vb2 queue? is it a hole in vb2 queue?
>>
>> Yes num_buffers isn't change and still represent the highest index of
>> allocated buffer.
>> DELETE_BUF remove the buffer from the list and free the memory.
>> That could a create a hole in vb2 queue, that why the first patch of
>> this series change all places where vb2 queue array was used by calls
>> to helpers function and also check the return value.
>> num_buffers is used as the lowest possible value when finding the free
>> indexes for buffers (see vb2_queue_add_buffer()) this way I can
>> guaranty that the indexes a continuous which is a requirement for
>> create_bufs.
>> I choose this solution because XArray API doesn't offer way to find
>> continuous free range.
>> It doesn't seem impossible to add it but this series is already big
>> enough from my point of view.
>>
>> Regards,
>> Benjamin
>>
>>>
>>> And you can use xa_for_each() instead the for loop to iterate over
>>> the present buffers in vb2 queue.
>
>Sorry I forgot this comment.
>Where do you think I can improve the code ?
That's what I meant is the following code can use xa_for_each() directly.
Indeed the for loop also work well, so please ignore it.
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
>
>Thanks,
>Benjamin
>
>>>
>>>> +
>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>> + return 0;
>>>> +}
>>>> +
>>>> /*
>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>> * @q: videobuf2 queue
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> index 724135d41f7f..cea666c17b41 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>>> + return vb2_core_delete_buf(q, b->index); }
>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>> +
>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>> *create)
>>>> {
>>>> unsigned requested_planes = 1; diff --git
>>>> a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>>> core/v4l2-dev.c index f81279492682..80ace2e1e932 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>> video_device
>>>> *vdev)
>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>> vidioc_prepare_buf);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>> vidioc_streamoff);
>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>> + vidioc_delete_buf);
>>>> }
>>>>
>>>> if (is_vid || is_vbi || is_meta) { diff --git
>>>> a/drivers/media/v4l2-core/v4l2-
>>>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index a858acea6547..1c737279d3ef 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>> v4l2_ioctl_ops *ops,
>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>>>
>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>> + struct file *file, void *fh, void *arg) {
>>>> + struct v4l2_buffer *b = arg;
>>>> + int ret = check_fmt(file, b->type);
>>>> +
>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>>> +
>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>> struct file *file, void *fh, void
>>>> *arg) { @@ -2905,6 +2914,7
>>>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>> v4l_print_freq_band, 0),
>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>>> id)),
>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>> + v4l_print_buffer, INFO_FL_QUEUE),
>>>> };
>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>
>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>> index
>>>> edb733f21604..2f232ed884c7 100644
>>>> --- a/include/media/v4l2-ioctl.h
>>>> +++ b/include/media/v4l2-ioctl.h
>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>> * @vidioc_overlay: pointer to the function that implements
>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>> * @vidioc_g_fbuf: pointer to the function that implements @@
>>>> -422,6 +424,8
>>>> @@ struct v4l2_ioctl_ops {
>>>> struct v4l2_create_buffers *b);
>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>> struct v4l2_buffer *b);
>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>> + struct v4l2_buffer *b);
>>>>
>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>> int i);
>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>>>> index
>>>> 080b783d608d..0f9e68f76b77 100644
>>>> --- a/include/media/videobuf2-core.h
>>>> +++ b/include/media/videobuf2-core.h
>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>> enum vb2_memory memory,
>>>> */
>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>> void *pb);
>>>>
>>>> +/**
>>>> + * vb2_core_delete_buf() -
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @index: id number of the buffer.
>>>> + *
>>>> + * Return: returns zero on success; an error code otherwise.
>>>> + */
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>> +
>>>> /**
>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>> *
>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>> b/include/media/videobuf2-v4l2.h index 88a7a565170e..3beeb4c735f0
>>>> 100644
>>>> --- a/include/media/videobuf2-v4l2.h
>>>> +++ b/include/media/videobuf2-v4l2.h
>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>> v4l2_create_buffers *create);
>>>> */
>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>> struct v4l2_buffer *b);
>>>> +/**
>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>> + *
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @b: buffer structure passed from userspace to
>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>> + *
>>>> + * The return values from this function are intended to be directly
>>>> +returned
>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>> + */
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>
>>>> /**
>>>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>> index
>>>> aee75eb9e686..31bba1915642 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>>>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>>
>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>> v4l2_query_ext_ctrl)
>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>> +
>>>>
>>>> /* Reminder: when adding new ioctls please add support for them to
>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>> --
>>>> 2.39.2
^ permalink raw reply [flat|nested] 183+ messages in thread* RE: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-26 8:13 ` Ming Qian
0 siblings, 0 replies; 183+ messages in thread
From: Ming Qian @ 2023-06-26 8:13 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ezequiel,
p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
>From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>Sent: 2023年6月26日 15:50
>To: Ming Qian <ming.qian@nxp.com>; mchehab@kernel.org;
>tfiga@chromium.org; m.szyprowski@samsung.com;
>ezequiel@vanguardiasur.com.ar; p.zabel@pengutronix.de;
>gregkh@linuxfoundation.org; hverkuil-cisco@xs4all.nl;
>nicolas.dufresne@collabora.com
>Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>staging@lists.linux.dev; kernel@collabora.com
>Subject: Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>Le 26/06/2023 à 09:48, Benjamin Gaignard a écrit :
>>
>> Le 26/06/2023 à 09:08, Ming Qian a écrit :
>>> Hi Benjamin,
>>>
>>>> -----Original Message-----
>>>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> Sent: 2023年6月22日 21:14
>>>> To: mchehab@kernel.org; tfiga@chromium.org;
>>>> m.szyprowski@samsung.com; Ming Qian <ming.qian@nxp.com>;
>>>> ezequiel@vanguardiasur.com.ar; p.zabel@pengutronix.de;
>>>> gregkh@linuxfoundation.org; hverkuil- cisco@xs4all.nl;
>>>> nicolas.dufresne@collabora.com
>>>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>>>> linux-arm-
>>>> kernel@lists.infradead.org; linux-mediatek@lists.infradead.org;
>>>> linux-arm-
>>>> msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>>>> staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
>>>> <benjamin.gaignard@collabora.com>
>>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>>
>>>> Caution: This is an external email. Please take care when clicking
>>>> links or opening attachments. When in doubt, report the message
>>>> using the 'Report this email' button
>>>>
>>>>
>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> ---
>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>> include/media/v4l2-ioctl.h | 4 ++
>>>> include/media/videobuf2-core.h | 9 ++++
>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>> include/uapi/linux/videodev2.h | 2 +
>>>> 10 files changed, 128 insertions(+)
>>>> create mode 100644
>>>> Documentation/userspace-api/media/v4l/vidioc-delete-
>>>> buf.rst
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> @@ -17,6 +17,7 @@ Function Reference
>>>> vidioc-dbg-g-chip-info
>>>> vidioc-dbg-g-register
>>>> vidioc-decoder-cmd
>>>> + vidioc-delete-buf
>>>> vidioc-dqevent
>>>> vidioc-dv-timings-cap
>>>> vidioc-encoder-cmd
>>>> diff --git
>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> new file mode 100644
>>>> index 000000000000..0e7ce58f91bc
>>>> --- /dev/null
>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> @@ -0,0 +1,51 @@
>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>>> +c:namespace:: V4L
>>>> +
>>>> +.. _VIDIOC_DELETE_BUF:
>>>> +
>>>> +************************
>>>> +ioctl VIDIOC_DELETE_BUF
>>>> +************************
>>>> +
>>>> +Name
>>>> +====
>>>> +
>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>> +
>>>> +Synopsis
>>>> +========
>>>> +
>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>> +
>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>> +
>>>> +Arguments
>>>> +=========
>>>> +
>>>> +``fd``
>>>> + File descriptor returned by :c:func:`open()`.
>>>> +
>>>> +``argp``
>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>> +
>>>> +Description
>>>> +===========
>>>> +
>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl
>>>> +to delete a buffer from a queue.
>>>> +
>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>> +:ref:`buffer`.
>>>> +
>>>> +Return Value
>>>> +============
>>>> +
>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>> set
>>>> +appropriately. The generic error codes are described at the
>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>> +
>>>> +EBUSY
>>>> + File I/O is in progress.
>>>> +
>>>> +EINVAL
>>>> + The buffer ``index`` doesn't exist in the queue.
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index 899783f67580..aa546c972c3d 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>*q,
>>>> unsigned int index, void *pb) }
>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>>> + struct vb2_buffer *vb;
>>>> +
>>>> + vb = vb2_get_buffer(q, index);
>>>> + if (!vb) {
>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>> index %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + if (vb->planes[0].mem_priv)
>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>> +
>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>> + __vb2_buf_mem_free(vb);
>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>> + __vb2_buf_dmabuf_put(vb);
>>>> + else
>>>> + __vb2_buf_userptr_put(vb);
>>>> +
>>>> + vb2_queue_remove_buffer(q, vb);
>>>> + kfree(vb);
>>> The num_buffers is not changed, Is that on purpose?
>>> num_buffers doesn't mean the number of vb2 buffers, but only decides
>>> the max index of allocated vb2 buffer?
>>> Once the index is deleted, then it won't be used anymore, unless free
>>> the vb2 queue? is it a hole in vb2 queue?
>>
>> Yes num_buffers isn't change and still represent the highest index of
>> allocated buffer.
>> DELETE_BUF remove the buffer from the list and free the memory.
>> That could a create a hole in vb2 queue, that why the first patch of
>> this series change all places where vb2 queue array was used by calls
>> to helpers function and also check the return value.
>> num_buffers is used as the lowest possible value when finding the free
>> indexes for buffers (see vb2_queue_add_buffer()) this way I can
>> guaranty that the indexes a continuous which is a requirement for
>> create_bufs.
>> I choose this solution because XArray API doesn't offer way to find
>> continuous free range.
>> It doesn't seem impossible to add it but this series is already big
>> enough from my point of view.
>>
>> Regards,
>> Benjamin
>>
>>>
>>> And you can use xa_for_each() instead the for loop to iterate over
>>> the present buffers in vb2 queue.
>
>Sorry I forgot this comment.
>Where do you think I can improve the code ?
That's what I meant is the following code can use xa_for_each() directly.
Indeed the for loop also work well, so please ignore it.
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
>
>Thanks,
>Benjamin
>
>>>
>>>> +
>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>> + return 0;
>>>> +}
>>>> +
>>>> /*
>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>> * @q: videobuf2 queue
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> index 724135d41f7f..cea666c17b41 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>>> + return vb2_core_delete_buf(q, b->index); }
>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>> +
>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>> *create)
>>>> {
>>>> unsigned requested_planes = 1; diff --git
>>>> a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>>> core/v4l2-dev.c index f81279492682..80ace2e1e932 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>> video_device
>>>> *vdev)
>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>> vidioc_prepare_buf);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>> vidioc_streamoff);
>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>> + vidioc_delete_buf);
>>>> }
>>>>
>>>> if (is_vid || is_vbi || is_meta) { diff --git
>>>> a/drivers/media/v4l2-core/v4l2-
>>>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index a858acea6547..1c737279d3ef 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>> v4l2_ioctl_ops *ops,
>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>>>
>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>> + struct file *file, void *fh, void *arg) {
>>>> + struct v4l2_buffer *b = arg;
>>>> + int ret = check_fmt(file, b->type);
>>>> +
>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>>> +
>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>> struct file *file, void *fh, void
>>>> *arg) { @@ -2905,6 +2914,7
>>>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>> v4l_print_freq_band, 0),
>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>>> id)),
>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>> + v4l_print_buffer, INFO_FL_QUEUE),
>>>> };
>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>
>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>> index
>>>> edb733f21604..2f232ed884c7 100644
>>>> --- a/include/media/v4l2-ioctl.h
>>>> +++ b/include/media/v4l2-ioctl.h
>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>> * @vidioc_overlay: pointer to the function that implements
>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>> * @vidioc_g_fbuf: pointer to the function that implements @@
>>>> -422,6 +424,8
>>>> @@ struct v4l2_ioctl_ops {
>>>> struct v4l2_create_buffers *b);
>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>> struct v4l2_buffer *b);
>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>> + struct v4l2_buffer *b);
>>>>
>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>> int i);
>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>>>> index
>>>> 080b783d608d..0f9e68f76b77 100644
>>>> --- a/include/media/videobuf2-core.h
>>>> +++ b/include/media/videobuf2-core.h
>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>> enum vb2_memory memory,
>>>> */
>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>> void *pb);
>>>>
>>>> +/**
>>>> + * vb2_core_delete_buf() -
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @index: id number of the buffer.
>>>> + *
>>>> + * Return: returns zero on success; an error code otherwise.
>>>> + */
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>> +
>>>> /**
>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>> *
>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>> b/include/media/videobuf2-v4l2.h index 88a7a565170e..3beeb4c735f0
>>>> 100644
>>>> --- a/include/media/videobuf2-v4l2.h
>>>> +++ b/include/media/videobuf2-v4l2.h
>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>> v4l2_create_buffers *create);
>>>> */
>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>> struct v4l2_buffer *b);
>>>> +/**
>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>> + *
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @b: buffer structure passed from userspace to
>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>> + *
>>>> + * The return values from this function are intended to be directly
>>>> +returned
>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>> + */
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>
>>>> /**
>>>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>> index
>>>> aee75eb9e686..31bba1915642 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>>>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>>
>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>> v4l2_query_ext_ctrl)
>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>> +
>>>>
>>>> /* Reminder: when adding new ioctls please add support for them to
>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>> --
>>>> 2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* RE: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-26 8:13 ` Ming Qian
0 siblings, 0 replies; 183+ messages in thread
From: Ming Qian @ 2023-06-26 8:13 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ezequiel,
p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
>From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>Sent: 2023年6月26日 15:50
>To: Ming Qian <ming.qian@nxp.com>; mchehab@kernel.org;
>tfiga@chromium.org; m.szyprowski@samsung.com;
>ezequiel@vanguardiasur.com.ar; p.zabel@pengutronix.de;
>gregkh@linuxfoundation.org; hverkuil-cisco@xs4all.nl;
>nicolas.dufresne@collabora.com
>Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>staging@lists.linux.dev; kernel@collabora.com
>Subject: Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>Le 26/06/2023 à 09:48, Benjamin Gaignard a écrit :
>>
>> Le 26/06/2023 à 09:08, Ming Qian a écrit :
>>> Hi Benjamin,
>>>
>>>> -----Original Message-----
>>>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> Sent: 2023年6月22日 21:14
>>>> To: mchehab@kernel.org; tfiga@chromium.org;
>>>> m.szyprowski@samsung.com; Ming Qian <ming.qian@nxp.com>;
>>>> ezequiel@vanguardiasur.com.ar; p.zabel@pengutronix.de;
>>>> gregkh@linuxfoundation.org; hverkuil- cisco@xs4all.nl;
>>>> nicolas.dufresne@collabora.com
>>>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>>>> linux-arm-
>>>> kernel@lists.infradead.org; linux-mediatek@lists.infradead.org;
>>>> linux-arm-
>>>> msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>>>> staging@lists.linux.dev; kernel@collabora.com; Benjamin Gaignard
>>>> <benjamin.gaignard@collabora.com>
>>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>>
>>>> Caution: This is an external email. Please take care when clicking
>>>> links or opening attachments. When in doubt, report the message
>>>> using the 'Report this email' button
>>>>
>>>>
>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> ---
>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>> include/media/v4l2-ioctl.h | 4 ++
>>>> include/media/videobuf2-core.h | 9 ++++
>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>> include/uapi/linux/videodev2.h | 2 +
>>>> 10 files changed, 128 insertions(+)
>>>> create mode 100644
>>>> Documentation/userspace-api/media/v4l/vidioc-delete-
>>>> buf.rst
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> @@ -17,6 +17,7 @@ Function Reference
>>>> vidioc-dbg-g-chip-info
>>>> vidioc-dbg-g-register
>>>> vidioc-decoder-cmd
>>>> + vidioc-delete-buf
>>>> vidioc-dqevent
>>>> vidioc-dv-timings-cap
>>>> vidioc-encoder-cmd
>>>> diff --git
>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> new file mode 100644
>>>> index 000000000000..0e7ce58f91bc
>>>> --- /dev/null
>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> @@ -0,0 +1,51 @@
>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>>> +c:namespace:: V4L
>>>> +
>>>> +.. _VIDIOC_DELETE_BUF:
>>>> +
>>>> +************************
>>>> +ioctl VIDIOC_DELETE_BUF
>>>> +************************
>>>> +
>>>> +Name
>>>> +====
>>>> +
>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>> +
>>>> +Synopsis
>>>> +========
>>>> +
>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>> +
>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>> +
>>>> +Arguments
>>>> +=========
>>>> +
>>>> +``fd``
>>>> + File descriptor returned by :c:func:`open()`.
>>>> +
>>>> +``argp``
>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>> +
>>>> +Description
>>>> +===========
>>>> +
>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl
>>>> +to delete a buffer from a queue.
>>>> +
>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>> +:ref:`buffer`.
>>>> +
>>>> +Return Value
>>>> +============
>>>> +
>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>> set
>>>> +appropriately. The generic error codes are described at the
>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>> +
>>>> +EBUSY
>>>> + File I/O is in progress.
>>>> +
>>>> +EINVAL
>>>> + The buffer ``index`` doesn't exist in the queue.
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index 899783f67580..aa546c972c3d 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>*q,
>>>> unsigned int index, void *pb) }
>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>>> + struct vb2_buffer *vb;
>>>> +
>>>> + vb = vb2_get_buffer(q, index);
>>>> + if (!vb) {
>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>> index %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + if (vb->planes[0].mem_priv)
>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>> +
>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>> + __vb2_buf_mem_free(vb);
>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>> + __vb2_buf_dmabuf_put(vb);
>>>> + else
>>>> + __vb2_buf_userptr_put(vb);
>>>> +
>>>> + vb2_queue_remove_buffer(q, vb);
>>>> + kfree(vb);
>>> The num_buffers is not changed, Is that on purpose?
>>> num_buffers doesn't mean the number of vb2 buffers, but only decides
>>> the max index of allocated vb2 buffer?
>>> Once the index is deleted, then it won't be used anymore, unless free
>>> the vb2 queue? is it a hole in vb2 queue?
>>
>> Yes num_buffers isn't change and still represent the highest index of
>> allocated buffer.
>> DELETE_BUF remove the buffer from the list and free the memory.
>> That could a create a hole in vb2 queue, that why the first patch of
>> this series change all places where vb2 queue array was used by calls
>> to helpers function and also check the return value.
>> num_buffers is used as the lowest possible value when finding the free
>> indexes for buffers (see vb2_queue_add_buffer()) this way I can
>> guaranty that the indexes a continuous which is a requirement for
>> create_bufs.
>> I choose this solution because XArray API doesn't offer way to find
>> continuous free range.
>> It doesn't seem impossible to add it but this series is already big
>> enough from my point of view.
>>
>> Regards,
>> Benjamin
>>
>>>
>>> And you can use xa_for_each() instead the for loop to iterate over
>>> the present buffers in vb2 queue.
>
>Sorry I forgot this comment.
>Where do you think I can improve the code ?
That's what I meant is the following code can use xa_for_each() directly.
Indeed the for loop also work well, so please ignore it.
for (buffer = 0; buffer < q->num_buffers; ++buffer) {
struct vb2_buffer *vb = vb2_get_buffer(q, buffer);
>
>Thanks,
>Benjamin
>
>>>
>>>> +
>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>> + return 0;
>>>> +}
>>>> +
>>>> /*
>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>> * @q: videobuf2 queue
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> index 724135d41f7f..cea666c17b41 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>>> + return vb2_core_delete_buf(q, b->index); }
>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>> +
>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>> *create)
>>>> {
>>>> unsigned requested_planes = 1; diff --git
>>>> a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>>> core/v4l2-dev.c index f81279492682..80ace2e1e932 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>> video_device
>>>> *vdev)
>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>> vidioc_prepare_buf);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>> vidioc_streamoff);
>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>> + vidioc_delete_buf);
>>>> }
>>>>
>>>> if (is_vid || is_vbi || is_meta) { diff --git
>>>> a/drivers/media/v4l2-core/v4l2-
>>>> ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index a858acea6547..1c737279d3ef 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>> v4l2_ioctl_ops *ops,
>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>>>
>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>> + struct file *file, void *fh, void *arg) {
>>>> + struct v4l2_buffer *b = arg;
>>>> + int ret = check_fmt(file, b->type);
>>>> +
>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>>> +
>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>> struct file *file, void *fh, void
>>>> *arg) { @@ -2905,6 +2914,7
>>>> @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>> v4l_print_freq_band, 0),
>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>>> id)),
>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>> + v4l_print_buffer, INFO_FL_QUEUE),
>>>> };
>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>
>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>> index
>>>> edb733f21604..2f232ed884c7 100644
>>>> --- a/include/media/v4l2-ioctl.h
>>>> +++ b/include/media/v4l2-ioctl.h
>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>> * @vidioc_overlay: pointer to the function that implements
>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>> * @vidioc_g_fbuf: pointer to the function that implements @@
>>>> -422,6 +424,8
>>>> @@ struct v4l2_ioctl_ops {
>>>> struct v4l2_create_buffers *b);
>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>> struct v4l2_buffer *b);
>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>> + struct v4l2_buffer *b);
>>>>
>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>> int i);
>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>>>> index
>>>> 080b783d608d..0f9e68f76b77 100644
>>>> --- a/include/media/videobuf2-core.h
>>>> +++ b/include/media/videobuf2-core.h
>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>> enum vb2_memory memory,
>>>> */
>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>> void *pb);
>>>>
>>>> +/**
>>>> + * vb2_core_delete_buf() -
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @index: id number of the buffer.
>>>> + *
>>>> + * Return: returns zero on success; an error code otherwise.
>>>> + */
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>> +
>>>> /**
>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>> *
>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>> b/include/media/videobuf2-v4l2.h index 88a7a565170e..3beeb4c735f0
>>>> 100644
>>>> --- a/include/media/videobuf2-v4l2.h
>>>> +++ b/include/media/videobuf2-v4l2.h
>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>> v4l2_create_buffers *create);
>>>> */
>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>> struct v4l2_buffer *b);
>>>> +/**
>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>> + *
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @b: buffer structure passed from userspace to
>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>> + *
>>>> + * The return values from this function are intended to be directly
>>>> +returned
>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>> + */
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>
>>>> /**
>>>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>> index
>>>> aee75eb9e686..31bba1915642 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>>>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>>
>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>> v4l2_query_ext_ctrl)
>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>> +
>>>>
>>>> /* Reminder: when adding new ioctls please add support for them to
>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>> --
>>>> 2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* RE: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-26 7:48 ` Benjamin Gaignard
(?)
@ 2023-06-26 8:04 ` Ming Qian
-1 siblings, 0 replies; 183+ messages in thread
From: Ming Qian @ 2023-06-26 8:04 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ezequiel,
p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
>-----Original Message-----
>From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>Sent: 2023年6月26日 15:49
>To: Ming Qian <ming.qian@nxp.com>; mchehab@kernel.org;
>tfiga@chromium.org; m.szyprowski@samsung.com;
>ezequiel@vanguardiasur.com.ar; p.zabel@pengutronix.de;
>gregkh@linuxfoundation.org; hverkuil-cisco@xs4all.nl;
>nicolas.dufresne@collabora.com
>Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>staging@lists.linux.dev; kernel@collabora.com
>Subject: Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>Le 26/06/2023 à 09:08, Ming Qian a écrit :
>> Hi Benjamin,
>>
>>> -----Original Message-----
>>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> Sent: 2023年6月22日 21:14
>>> To: mchehab@kernel.org; tfiga@chromium.org;
>m.szyprowski@samsung.com;
>>> Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>>> p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>>> cisco@xs4all.nl; nicolas.dufresne@collabora.com
>>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>>> linux-arm- kernel@lists.infradead.org;
>>> linux-mediatek@lists.infradead.org; linux-arm- msm@vger.kernel.org;
>>> linux-rockchip@lists.infradead.org; linux- staging@lists.linux.dev;
>>> kernel@collabora.com; Benjamin Gaignard
>>> <benjamin.gaignard@collabora.com>
>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>
>>> Caution: This is an external email. Please take care when clicking
>>> links or opening attachments. When in doubt, report the message using
>>> the 'Report this email' button
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>> include/media/v4l2-ioctl.h | 4 ++
>>> include/media/videobuf2-core.h | 9 ++++
>>> include/media/videobuf2-v4l2.h | 11 ++++
>>> include/uapi/linux/videodev2.h | 2 +
>>> 10 files changed, 128 insertions(+)
>>> create mode 100644
>>> Documentation/userspace-api/media/v4l/vidioc-delete-
>>> buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>> vidioc-dbg-g-chip-info
>>> vidioc-dbg-g-register
>>> vidioc-decoder-cmd
>>> + vidioc-delete-buf
>>> vidioc-dqevent
>>> vidioc-dv-timings-cap
>>> vidioc-encoder-cmd
>>> diff --git
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>> +c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> + File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl
>>> +to delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>> +set appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> + File I/O is in progress.
>>> +
>>> +EINVAL
>>> + The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb) }
>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>> + struct vb2_buffer *vb;
>>> +
>>> + vb = vb2_get_buffer(q, index);
>>> + if (!vb) {
>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> + dprintk(q, 1, "can't delete non dequeued buffer index %d\n",
>index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->planes[0].mem_priv)
>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> + /* Free MMAP buffers or release USERPTR buffers */
>>> + if (q->memory == VB2_MEMORY_MMAP)
>>> + __vb2_buf_mem_free(vb);
>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>> + __vb2_buf_dmabuf_put(vb);
>>> + else
>>> + __vb2_buf_userptr_put(vb);
>>> +
>>> + vb2_queue_remove_buffer(q, vb);
>>> + kfree(vb);
>> The num_buffers is not changed, Is that on purpose?
>> num_buffers doesn't mean the number of vb2 buffers, but only decides the
>max index of allocated vb2 buffer?
>> Once the index is deleted, then it won't be used anymore, unless free the
>vb2 queue? is it a hole in vb2 queue?
>
>Yes num_buffers isn't change and still represent the highest index of allocated
>buffer.
>DELETE_BUF remove the buffer from the list and free the memory.
>That could a create a hole in vb2 queue, that why the first patch of this series
>change all places where vb2 queue array was used by calls to helpers function
>and also check the return value.
>num_buffers is used as the lowest possible value when finding the free
>indexes for buffers (see vb2_queue_add_buffer()) this way I can guaranty that
>the indexes a continuous which is a requirement for create_bufs.
>I choose this solution because XArray API doesn't offer way to find continuous
>free range.
>It doesn't seem impossible to add it but this series is already big enough from
>my point of view.
>
Thank you for your detailed explanation.
>Regards,
>Benjamin
>
>>
>> And you can use xa_for_each() instead the for loop to iterate over the
>present buffers in vb2 queue.
>>
>>> +
>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>> + return 0;
>>> +}
>>> +
>>> /*
>>> * vb2_start_streaming() - Attempt to start streaming.
>>> * @q: videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>> + return vb2_core_delete_buf(q, b->index); }
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>> *create) {
>>> unsigned requested_planes = 1; diff --git
>>> a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>> core/v4l2-dev.c index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>> video_device
>>> *vdev)
>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>vidioc_prepare_buf);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>> vidioc_streamoff);
>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> + vidioc_delete_buf);
>>> }
>>>
>>> if (is_vid || is_vbi || is_meta) { diff --git
>>> a/drivers/media/v4l2-core/v4l2- ioctl.c
>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> + struct file *file, void *fh, void *arg) {
>>> + struct v4l2_buffer *b = arg;
>>> + int ret = check_fmt(file, b->type);
>>> +
>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>> +
>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>> struct file *file, void *fh, void
>>> *arg) { @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>v4l2_ioctls[] = {
>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>> id)),
>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>> + v4l_print_buffer, INFO_FL_QUEUE),
>>> };
>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>> index
>>> edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>> * @vidioc_prepare_buf: pointer to the function that implements
>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>> * @vidioc_overlay: pointer to the function that implements
>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>> * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6
>>> +424,8 @@ struct v4l2_ioctl_ops {
>>> struct v4l2_create_buffers *b);
>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>> struct v4l2_buffer *b);
>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> + struct v4l2_buffer *b);
>>>
>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
>>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>>> index
>>> 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>> */
>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index: id number of the buffer.
>>> + *
>>> + * Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>> /**
>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>> *
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h index 88a7a565170e..3beeb4c735f0
>>> 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>> */
>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>> struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b: buffer structure passed from userspace to
>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> +returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>> /**
>>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>> index
>>> aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>
>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>> /* Reminder: when adding new ioctls please add support for them to
>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> --
>>> 2.39.2
^ permalink raw reply [flat|nested] 183+ messages in thread* RE: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-26 8:04 ` Ming Qian
0 siblings, 0 replies; 183+ messages in thread
From: Ming Qian @ 2023-06-26 8:04 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ezequiel,
p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
>-----Original Message-----
>From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>Sent: 2023年6月26日 15:49
>To: Ming Qian <ming.qian@nxp.com>; mchehab@kernel.org;
>tfiga@chromium.org; m.szyprowski@samsung.com;
>ezequiel@vanguardiasur.com.ar; p.zabel@pengutronix.de;
>gregkh@linuxfoundation.org; hverkuil-cisco@xs4all.nl;
>nicolas.dufresne@collabora.com
>Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>staging@lists.linux.dev; kernel@collabora.com
>Subject: Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>Le 26/06/2023 à 09:08, Ming Qian a écrit :
>> Hi Benjamin,
>>
>>> -----Original Message-----
>>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> Sent: 2023年6月22日 21:14
>>> To: mchehab@kernel.org; tfiga@chromium.org;
>m.szyprowski@samsung.com;
>>> Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>>> p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>>> cisco@xs4all.nl; nicolas.dufresne@collabora.com
>>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>>> linux-arm- kernel@lists.infradead.org;
>>> linux-mediatek@lists.infradead.org; linux-arm- msm@vger.kernel.org;
>>> linux-rockchip@lists.infradead.org; linux- staging@lists.linux.dev;
>>> kernel@collabora.com; Benjamin Gaignard
>>> <benjamin.gaignard@collabora.com>
>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>
>>> Caution: This is an external email. Please take care when clicking
>>> links or opening attachments. When in doubt, report the message using
>>> the 'Report this email' button
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>> include/media/v4l2-ioctl.h | 4 ++
>>> include/media/videobuf2-core.h | 9 ++++
>>> include/media/videobuf2-v4l2.h | 11 ++++
>>> include/uapi/linux/videodev2.h | 2 +
>>> 10 files changed, 128 insertions(+)
>>> create mode 100644
>>> Documentation/userspace-api/media/v4l/vidioc-delete-
>>> buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>> vidioc-dbg-g-chip-info
>>> vidioc-dbg-g-register
>>> vidioc-decoder-cmd
>>> + vidioc-delete-buf
>>> vidioc-dqevent
>>> vidioc-dv-timings-cap
>>> vidioc-encoder-cmd
>>> diff --git
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>> +c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> + File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl
>>> +to delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>> +set appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> + File I/O is in progress.
>>> +
>>> +EINVAL
>>> + The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb) }
>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>> + struct vb2_buffer *vb;
>>> +
>>> + vb = vb2_get_buffer(q, index);
>>> + if (!vb) {
>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> + dprintk(q, 1, "can't delete non dequeued buffer index %d\n",
>index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->planes[0].mem_priv)
>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> + /* Free MMAP buffers or release USERPTR buffers */
>>> + if (q->memory == VB2_MEMORY_MMAP)
>>> + __vb2_buf_mem_free(vb);
>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>> + __vb2_buf_dmabuf_put(vb);
>>> + else
>>> + __vb2_buf_userptr_put(vb);
>>> +
>>> + vb2_queue_remove_buffer(q, vb);
>>> + kfree(vb);
>> The num_buffers is not changed, Is that on purpose?
>> num_buffers doesn't mean the number of vb2 buffers, but only decides the
>max index of allocated vb2 buffer?
>> Once the index is deleted, then it won't be used anymore, unless free the
>vb2 queue? is it a hole in vb2 queue?
>
>Yes num_buffers isn't change and still represent the highest index of allocated
>buffer.
>DELETE_BUF remove the buffer from the list and free the memory.
>That could a create a hole in vb2 queue, that why the first patch of this series
>change all places where vb2 queue array was used by calls to helpers function
>and also check the return value.
>num_buffers is used as the lowest possible value when finding the free
>indexes for buffers (see vb2_queue_add_buffer()) this way I can guaranty that
>the indexes a continuous which is a requirement for create_bufs.
>I choose this solution because XArray API doesn't offer way to find continuous
>free range.
>It doesn't seem impossible to add it but this series is already big enough from
>my point of view.
>
Thank you for your detailed explanation.
>Regards,
>Benjamin
>
>>
>> And you can use xa_for_each() instead the for loop to iterate over the
>present buffers in vb2 queue.
>>
>>> +
>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>> + return 0;
>>> +}
>>> +
>>> /*
>>> * vb2_start_streaming() - Attempt to start streaming.
>>> * @q: videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>> + return vb2_core_delete_buf(q, b->index); }
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>> *create) {
>>> unsigned requested_planes = 1; diff --git
>>> a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>> core/v4l2-dev.c index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>> video_device
>>> *vdev)
>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>vidioc_prepare_buf);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>> vidioc_streamoff);
>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> + vidioc_delete_buf);
>>> }
>>>
>>> if (is_vid || is_vbi || is_meta) { diff --git
>>> a/drivers/media/v4l2-core/v4l2- ioctl.c
>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> + struct file *file, void *fh, void *arg) {
>>> + struct v4l2_buffer *b = arg;
>>> + int ret = check_fmt(file, b->type);
>>> +
>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>> +
>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>> struct file *file, void *fh, void
>>> *arg) { @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>v4l2_ioctls[] = {
>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>> id)),
>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>> + v4l_print_buffer, INFO_FL_QUEUE),
>>> };
>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>> index
>>> edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>> * @vidioc_prepare_buf: pointer to the function that implements
>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>> * @vidioc_overlay: pointer to the function that implements
>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>> * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6
>>> +424,8 @@ struct v4l2_ioctl_ops {
>>> struct v4l2_create_buffers *b);
>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>> struct v4l2_buffer *b);
>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> + struct v4l2_buffer *b);
>>>
>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
>>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>>> index
>>> 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>> */
>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index: id number of the buffer.
>>> + *
>>> + * Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>> /**
>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>> *
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h index 88a7a565170e..3beeb4c735f0
>>> 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>> */
>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>> struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b: buffer structure passed from userspace to
>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> +returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>> /**
>>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>> index
>>> aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>
>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>> /* Reminder: when adding new ioctls please add support for them to
>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> --
>>> 2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* RE: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-26 8:04 ` Ming Qian
0 siblings, 0 replies; 183+ messages in thread
From: Ming Qian @ 2023-06-26 8:04 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, tfiga, m.szyprowski, ezequiel,
p.zabel, gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel
>-----Original Message-----
>From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>Sent: 2023年6月26日 15:49
>To: Ming Qian <ming.qian@nxp.com>; mchehab@kernel.org;
>tfiga@chromium.org; m.szyprowski@samsung.com;
>ezequiel@vanguardiasur.com.ar; p.zabel@pengutronix.de;
>gregkh@linuxfoundation.org; hverkuil-cisco@xs4all.nl;
>nicolas.dufresne@collabora.com
>Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org; linux-arm-
>kernel@lists.infradead.org; linux-mediatek@lists.infradead.org; linux-arm-
>msm@vger.kernel.org; linux-rockchip@lists.infradead.org; linux-
>staging@lists.linux.dev; kernel@collabora.com
>Subject: Re: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>
>Caution: This is an external email. Please take care when clicking links or
>opening attachments. When in doubt, report the message using the 'Report
>this email' button
>
>
>Le 26/06/2023 à 09:08, Ming Qian a écrit :
>> Hi Benjamin,
>>
>>> -----Original Message-----
>>> From: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> Sent: 2023年6月22日 21:14
>>> To: mchehab@kernel.org; tfiga@chromium.org;
>m.szyprowski@samsung.com;
>>> Ming Qian <ming.qian@nxp.com>; ezequiel@vanguardiasur.com.ar;
>>> p.zabel@pengutronix.de; gregkh@linuxfoundation.org; hverkuil-
>>> cisco@xs4all.nl; nicolas.dufresne@collabora.com
>>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>>> linux-arm- kernel@lists.infradead.org;
>>> linux-mediatek@lists.infradead.org; linux-arm- msm@vger.kernel.org;
>>> linux-rockchip@lists.infradead.org; linux- staging@lists.linux.dev;
>>> kernel@collabora.com; Benjamin Gaignard
>>> <benjamin.gaignard@collabora.com>
>>> Subject: [EXT] [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
>>>
>>> Caution: This is an external email. Please take care when clicking
>>> links or opening attachments. When in doubt, report the message using
>>> the 'Report this email' button
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>> include/media/v4l2-ioctl.h | 4 ++
>>> include/media/videobuf2-core.h | 9 ++++
>>> include/media/videobuf2-v4l2.h | 11 ++++
>>> include/uapi/linux/videodev2.h | 2 +
>>> 10 files changed, 128 insertions(+)
>>> create mode 100644
>>> Documentation/userspace-api/media/v4l/vidioc-delete-
>>> buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>> vidioc-dbg-g-chip-info
>>> vidioc-dbg-g-register
>>> vidioc-decoder-cmd
>>> + vidioc-delete-buf
>>> vidioc-dqevent
>>> vidioc-dv-timings-cap
>>> vidioc-encoder-cmd
>>> diff --git
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later ..
>>> +c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> + File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl
>>> +to delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>> +set appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> + File I/O is in progress.
>>> +
>>> +EINVAL
>>> + The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb) }
>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index) {
>>> + struct vb2_buffer *vb;
>>> +
>>> + vb = vb2_get_buffer(q, index);
>>> + if (!vb) {
>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> + dprintk(q, 1, "can't delete non dequeued buffer index %d\n",
>index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->planes[0].mem_priv)
>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> + /* Free MMAP buffers or release USERPTR buffers */
>>> + if (q->memory == VB2_MEMORY_MMAP)
>>> + __vb2_buf_mem_free(vb);
>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>> + __vb2_buf_dmabuf_put(vb);
>>> + else
>>> + __vb2_buf_userptr_put(vb);
>>> +
>>> + vb2_queue_remove_buffer(q, vb);
>>> + kfree(vb);
>> The num_buffers is not changed, Is that on purpose?
>> num_buffers doesn't mean the number of vb2 buffers, but only decides the
>max index of allocated vb2 buffer?
>> Once the index is deleted, then it won't be used anymore, unless free the
>vb2 queue? is it a hole in vb2 queue?
>
>Yes num_buffers isn't change and still represent the highest index of allocated
>buffer.
>DELETE_BUF remove the buffer from the list and free the memory.
>That could a create a hole in vb2 queue, that why the first patch of this series
>change all places where vb2 queue array was used by calls to helpers function
>and also check the return value.
>num_buffers is used as the lowest possible value when finding the free
>indexes for buffers (see vb2_queue_add_buffer()) this way I can guaranty that
>the indexes a continuous which is a requirement for create_bufs.
>I choose this solution because XArray API doesn't offer way to find continuous
>free range.
>It doesn't seem impossible to add it but this series is already big enough from
>my point of view.
>
Thank you for your detailed explanation.
>Regards,
>Benjamin
>
>>
>> And you can use xa_for_each() instead the for loop to iterate over the
>present buffers in vb2 queue.
>>
>>> +
>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>> + return 0;
>>> +}
>>> +
>>> /*
>>> * vb2_start_streaming() - Attempt to start streaming.
>>> * @q: videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev, } EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b) {
>>> + return vb2_core_delete_buf(q, b->index); }
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>> *create) {
>>> unsigned requested_planes = 1; diff --git
>>> a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-
>>> core/v4l2-dev.c index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>> video_device
>>> *vdev)
>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>vidioc_prepare_buf);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>> vidioc_streamoff);
>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> + vidioc_delete_buf);
>>> }
>>>
>>> if (is_vid || is_vbi || is_meta) { diff --git
>>> a/drivers/media/v4l2-core/v4l2- ioctl.c
>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b); }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> + struct file *file, void *fh, void *arg) {
>>> + struct v4l2_buffer *b = arg;
>>> + int ret = check_fmt(file, b->type);
>>> +
>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b); }
>>> +
>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>> struct file *file, void *fh, void
>>> *arg) { @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>v4l2_ioctls[] = {
>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl,
>>> id)),
>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>> + v4l_print_buffer, INFO_FL_QUEUE),
>>> };
>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>> index
>>> edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>> * @vidioc_prepare_buf: pointer to the function that implements
>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>> * @vidioc_overlay: pointer to the function that implements
>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>> * @vidioc_g_fbuf: pointer to the function that implements @@ -422,6
>>> +424,8 @@ struct v4l2_ioctl_ops {
>>> struct v4l2_create_buffers *b);
>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>> struct v4l2_buffer *b);
>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> + struct v4l2_buffer *b);
>>>
>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
>>> int (*vidioc_g_fbuf)(struct file *file, void *fh, diff --git
>>> a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>>> index
>>> 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>> */
>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index: id number of the buffer.
>>> + *
>>> + * Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>> /**
>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>> *
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h index 88a7a565170e..3beeb4c735f0
>>> 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>> */
>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>> struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b: buffer structure passed from userspace to
>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> +returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>> /**
>>> * vb2_qbuf() - Queue a buffer from userspace diff --git
>>> a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>> index
>>> aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers { #define
>>> VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>>>
>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>> /* Reminder: when adding new ioctls please add support for them to
>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> --
>>> 2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-27 7:30 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-27 7:30 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski
On 6/22/23 21:13, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> .../userspace-api/media/v4l/user-func.rst | 1 +
> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> include/media/v4l2-ioctl.h | 4 ++
> include/media/videobuf2-core.h | 9 ++++
> include/media/videobuf2-v4l2.h | 11 ++++
> include/uapi/linux/videodev2.h | 2 +
> 10 files changed, 128 insertions(+)
> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>
> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst
> index 15ff0bf7bbe6..8c74016e12fd 100644
> --- a/Documentation/userspace-api/media/v4l/user-func.rst
> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> @@ -17,6 +17,7 @@ Function Reference
> vidioc-dbg-g-chip-info
> vidioc-dbg-g-register
> vidioc-decoder-cmd
> + vidioc-delete-buf
> vidioc-dqevent
> vidioc-dv-timings-cap
> vidioc-encoder-cmd
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> new file mode 100644
> index 000000000000..0e7ce58f91bc
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> @@ -0,0 +1,51 @@
> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> +.. c:namespace:: V4L
> +
> +.. _VIDIOC_DELETE_BUF:
> +
> +************************
> +ioctl VIDIOC_DELETE_BUF
> +************************
> +
> +Name
> +====
> +
> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> +
> +Synopsis
> +========
> +
> +.. c:macro:: VIDIOC_DELETE_BUF
> +
> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> +
> +Arguments
> +=========
> +
> +``fd``
> + File descriptor returned by :c:func:`open()`.
> +
> +``argp``
> + Pointer to struct :c:type:`v4l2_buffer`.
> +
> +Description
> +===========
> +
> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
> +delete a buffer from a queue.
> +
> +The struct :c:type:`v4l2_buffer` structure is specified in
> +:ref:`buffer`.
> +
> +Return Value
> +============
> +
> +On success 0 is returned, on error -1 and the ``errno`` variable is set
> +appropriately. The generic error codes are described at the
> +:ref:`Generic Error Codes <gen-errors>` chapter.
> +
> +EBUSY
> + File I/O is in progress.
> +
> +EINVAL
> + The buffer ``index`` doesn't exist in the queue.
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 899783f67580..aa546c972c3d 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
> }
> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>
> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> +{
> + struct vb2_buffer *vb;
> +
> + vb = vb2_get_buffer(q, index);
> + if (!vb) {
> + dprintk(q, 1, "invalid buffer index %d\n", index);
> + return -EINVAL;
> + }
> +
> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> + dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
> + return -EINVAL;
> + }
> +
I know the driver could implement its own
v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is used
by the hardware as a future reference frame.
But I think we need a flag to let the user know which buffer is still
used by the hardware.
Alternative ref case is safe, we only know it's existing when it is
dequeued in current V4L2 buffer mechanism.
While the Golden reference frame, such long term reference frame could
last much longer.
> + if (vb->planes[0].mem_priv)
> + call_void_vb_qop(vb, buf_cleanup, vb);
> +
> + /* Free MMAP buffers or release USERPTR buffers */
> + if (q->memory == VB2_MEMORY_MMAP)
> + __vb2_buf_mem_free(vb);
> + else if (q->memory == VB2_MEMORY_DMABUF)
> + __vb2_buf_dmabuf_put(vb);
> + else
> + __vb2_buf_userptr_put(vb);
> +
> + vb2_queue_remove_buffer(q, vb);
> + kfree(vb);
> +
> + dprintk(q, 2, "buffer %d deleted\n", index);
> + return 0;
> +}
> +
> /*
> * vb2_start_streaming() - Attempt to start streaming.
> * @q: videobuf2 queue
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index 724135d41f7f..cea666c17b41 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> }
> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>
> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> +{
> + return vb2_core_delete_buf(q, b->index);
> +}
> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> +
> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
> {
> unsigned requested_planes = 1;
> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
> index f81279492682..80ace2e1e932 100644
> --- a/drivers/media/v4l2-core/v4l2-dev.c
> +++ b/drivers/media/v4l2-core/v4l2-dev.c
> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device *vdev)
> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF, vidioc_delete_buf);
> }
>
> if (is_vid || is_vbi || is_meta) {
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index a858acea6547..1c737279d3ef 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct v4l2_ioctl_ops *ops,
> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
> }
>
> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> + struct file *file, void *fh, void *arg)
> +{
> + struct v4l2_buffer *b = arg;
> + int ret = check_fmt(file, b->type);
> +
> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> +}
> +
> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> struct file *file, void *fh, void *arg)
> {
> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0),
> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer, INFO_FL_QUEUE),
> };
> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>
> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> index edb733f21604..2f232ed884c7 100644
> --- a/include/media/v4l2-ioctl.h
> +++ b/include/media/v4l2-ioctl.h
> @@ -163,6 +163,8 @@ struct v4l2_fh;
> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> * @vidioc_prepare_buf: pointer to the function that implements
> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> + * @vidioc_delete_buf: pointer to the function that implements
> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> * @vidioc_overlay: pointer to the function that implements
> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> * @vidioc_g_fbuf: pointer to the function that implements
> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
> struct v4l2_create_buffers *b);
> int (*vidioc_prepare_buf)(struct file *file, void *fh,
> struct v4l2_buffer *b);
> + int (*vidioc_delete_buf)(struct file *file, void *fh,
> + struct v4l2_buffer *b);
>
> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
> int (*vidioc_g_fbuf)(struct file *file, void *fh,
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 080b783d608d..0f9e68f76b77 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
> */
> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>
> +/**
> + * vb2_core_delete_buf() -
> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> + * @index: id number of the buffer.
> + *
> + * Return: returns zero on success; an error code otherwise.
> + */
> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> +
> /**
> * vb2_core_qbuf() - Queue a buffer from userspace
> *
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 88a7a565170e..3beeb4c735f0 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
> */
> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> struct v4l2_buffer *b);
> +/**
> + * vb2_delete_buf() - Delete the buffer from the queue
> + *
> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> + * @b: buffer structure passed from userspace to
> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> + *
> + * The return values from this function are intended to be directly returned
> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> + */
> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>
> /**
> * vb2_qbuf() - Queue a buffer from userspace
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index aee75eb9e686..31bba1915642 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>
> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
> +
>
> /* Reminder: when adding new ioctls please add support for them to
> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> --
> 2.39.2
>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-27 7:30 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-27 7:30 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski
On 6/22/23 21:13, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> .../userspace-api/media/v4l/user-func.rst | 1 +
> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> include/media/v4l2-ioctl.h | 4 ++
> include/media/videobuf2-core.h | 9 ++++
> include/media/videobuf2-v4l2.h | 11 ++++
> include/uapi/linux/videodev2.h | 2 +
> 10 files changed, 128 insertions(+)
> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>
> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst
> index 15ff0bf7bbe6..8c74016e12fd 100644
> --- a/Documentation/userspace-api/media/v4l/user-func.rst
> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> @@ -17,6 +17,7 @@ Function Reference
> vidioc-dbg-g-chip-info
> vidioc-dbg-g-register
> vidioc-decoder-cmd
> + vidioc-delete-buf
> vidioc-dqevent
> vidioc-dv-timings-cap
> vidioc-encoder-cmd
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> new file mode 100644
> index 000000000000..0e7ce58f91bc
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> @@ -0,0 +1,51 @@
> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> +.. c:namespace:: V4L
> +
> +.. _VIDIOC_DELETE_BUF:
> +
> +************************
> +ioctl VIDIOC_DELETE_BUF
> +************************
> +
> +Name
> +====
> +
> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> +
> +Synopsis
> +========
> +
> +.. c:macro:: VIDIOC_DELETE_BUF
> +
> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> +
> +Arguments
> +=========
> +
> +``fd``
> + File descriptor returned by :c:func:`open()`.
> +
> +``argp``
> + Pointer to struct :c:type:`v4l2_buffer`.
> +
> +Description
> +===========
> +
> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
> +delete a buffer from a queue.
> +
> +The struct :c:type:`v4l2_buffer` structure is specified in
> +:ref:`buffer`.
> +
> +Return Value
> +============
> +
> +On success 0 is returned, on error -1 and the ``errno`` variable is set
> +appropriately. The generic error codes are described at the
> +:ref:`Generic Error Codes <gen-errors>` chapter.
> +
> +EBUSY
> + File I/O is in progress.
> +
> +EINVAL
> + The buffer ``index`` doesn't exist in the queue.
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 899783f67580..aa546c972c3d 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
> }
> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>
> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> +{
> + struct vb2_buffer *vb;
> +
> + vb = vb2_get_buffer(q, index);
> + if (!vb) {
> + dprintk(q, 1, "invalid buffer index %d\n", index);
> + return -EINVAL;
> + }
> +
> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> + dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
> + return -EINVAL;
> + }
> +
I know the driver could implement its own
v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is used
by the hardware as a future reference frame.
But I think we need a flag to let the user know which buffer is still
used by the hardware.
Alternative ref case is safe, we only know it's existing when it is
dequeued in current V4L2 buffer mechanism.
While the Golden reference frame, such long term reference frame could
last much longer.
> + if (vb->planes[0].mem_priv)
> + call_void_vb_qop(vb, buf_cleanup, vb);
> +
> + /* Free MMAP buffers or release USERPTR buffers */
> + if (q->memory == VB2_MEMORY_MMAP)
> + __vb2_buf_mem_free(vb);
> + else if (q->memory == VB2_MEMORY_DMABUF)
> + __vb2_buf_dmabuf_put(vb);
> + else
> + __vb2_buf_userptr_put(vb);
> +
> + vb2_queue_remove_buffer(q, vb);
> + kfree(vb);
> +
> + dprintk(q, 2, "buffer %d deleted\n", index);
> + return 0;
> +}
> +
> /*
> * vb2_start_streaming() - Attempt to start streaming.
> * @q: videobuf2 queue
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index 724135d41f7f..cea666c17b41 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> }
> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>
> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> +{
> + return vb2_core_delete_buf(q, b->index);
> +}
> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> +
> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
> {
> unsigned requested_planes = 1;
> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
> index f81279492682..80ace2e1e932 100644
> --- a/drivers/media/v4l2-core/v4l2-dev.c
> +++ b/drivers/media/v4l2-core/v4l2-dev.c
> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device *vdev)
> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF, vidioc_delete_buf);
> }
>
> if (is_vid || is_vbi || is_meta) {
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index a858acea6547..1c737279d3ef 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct v4l2_ioctl_ops *ops,
> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
> }
>
> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> + struct file *file, void *fh, void *arg)
> +{
> + struct v4l2_buffer *b = arg;
> + int ret = check_fmt(file, b->type);
> +
> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> +}
> +
> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> struct file *file, void *fh, void *arg)
> {
> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0),
> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer, INFO_FL_QUEUE),
> };
> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>
> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> index edb733f21604..2f232ed884c7 100644
> --- a/include/media/v4l2-ioctl.h
> +++ b/include/media/v4l2-ioctl.h
> @@ -163,6 +163,8 @@ struct v4l2_fh;
> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> * @vidioc_prepare_buf: pointer to the function that implements
> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> + * @vidioc_delete_buf: pointer to the function that implements
> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> * @vidioc_overlay: pointer to the function that implements
> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> * @vidioc_g_fbuf: pointer to the function that implements
> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
> struct v4l2_create_buffers *b);
> int (*vidioc_prepare_buf)(struct file *file, void *fh,
> struct v4l2_buffer *b);
> + int (*vidioc_delete_buf)(struct file *file, void *fh,
> + struct v4l2_buffer *b);
>
> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
> int (*vidioc_g_fbuf)(struct file *file, void *fh,
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 080b783d608d..0f9e68f76b77 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
> */
> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>
> +/**
> + * vb2_core_delete_buf() -
> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> + * @index: id number of the buffer.
> + *
> + * Return: returns zero on success; an error code otherwise.
> + */
> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> +
> /**
> * vb2_core_qbuf() - Queue a buffer from userspace
> *
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 88a7a565170e..3beeb4c735f0 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
> */
> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> struct v4l2_buffer *b);
> +/**
> + * vb2_delete_buf() - Delete the buffer from the queue
> + *
> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> + * @b: buffer structure passed from userspace to
> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> + *
> + * The return values from this function are intended to be directly returned
> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> + */
> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>
> /**
> * vb2_qbuf() - Queue a buffer from userspace
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index aee75eb9e686..31bba1915642 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>
> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
> +
>
> /* Reminder: when adding new ioctls please add support for them to
> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> --
> 2.39.2
>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-27 7:30 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-27 7:30 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski
On 6/22/23 21:13, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> .../userspace-api/media/v4l/user-func.rst | 1 +
> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> include/media/v4l2-ioctl.h | 4 ++
> include/media/videobuf2-core.h | 9 ++++
> include/media/videobuf2-v4l2.h | 11 ++++
> include/uapi/linux/videodev2.h | 2 +
> 10 files changed, 128 insertions(+)
> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>
> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst b/Documentation/userspace-api/media/v4l/user-func.rst
> index 15ff0bf7bbe6..8c74016e12fd 100644
> --- a/Documentation/userspace-api/media/v4l/user-func.rst
> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> @@ -17,6 +17,7 @@ Function Reference
> vidioc-dbg-g-chip-info
> vidioc-dbg-g-register
> vidioc-decoder-cmd
> + vidioc-delete-buf
> vidioc-dqevent
> vidioc-dv-timings-cap
> vidioc-encoder-cmd
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> new file mode 100644
> index 000000000000..0e7ce58f91bc
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> @@ -0,0 +1,51 @@
> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> +.. c:namespace:: V4L
> +
> +.. _VIDIOC_DELETE_BUF:
> +
> +************************
> +ioctl VIDIOC_DELETE_BUF
> +************************
> +
> +Name
> +====
> +
> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> +
> +Synopsis
> +========
> +
> +.. c:macro:: VIDIOC_DELETE_BUF
> +
> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> +
> +Arguments
> +=========
> +
> +``fd``
> + File descriptor returned by :c:func:`open()`.
> +
> +``argp``
> + Pointer to struct :c:type:`v4l2_buffer`.
> +
> +Description
> +===========
> +
> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
> +delete a buffer from a queue.
> +
> +The struct :c:type:`v4l2_buffer` structure is specified in
> +:ref:`buffer`.
> +
> +Return Value
> +============
> +
> +On success 0 is returned, on error -1 and the ``errno`` variable is set
> +appropriately. The generic error codes are described at the
> +:ref:`Generic Error Codes <gen-errors>` chapter.
> +
> +EBUSY
> + File I/O is in progress.
> +
> +EINVAL
> + The buffer ``index`` doesn't exist in the queue.
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 899783f67580..aa546c972c3d 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
> }
> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>
> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> +{
> + struct vb2_buffer *vb;
> +
> + vb = vb2_get_buffer(q, index);
> + if (!vb) {
> + dprintk(q, 1, "invalid buffer index %d\n", index);
> + return -EINVAL;
> + }
> +
> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> + dprintk(q, 1, "can't delete non dequeued buffer index %d\n", index);
> + return -EINVAL;
> + }
> +
I know the driver could implement its own
v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is used
by the hardware as a future reference frame.
But I think we need a flag to let the user know which buffer is still
used by the hardware.
Alternative ref case is safe, we only know it's existing when it is
dequeued in current V4L2 buffer mechanism.
While the Golden reference frame, such long term reference frame could
last much longer.
> + if (vb->planes[0].mem_priv)
> + call_void_vb_qop(vb, buf_cleanup, vb);
> +
> + /* Free MMAP buffers or release USERPTR buffers */
> + if (q->memory == VB2_MEMORY_MMAP)
> + __vb2_buf_mem_free(vb);
> + else if (q->memory == VB2_MEMORY_DMABUF)
> + __vb2_buf_dmabuf_put(vb);
> + else
> + __vb2_buf_userptr_put(vb);
> +
> + vb2_queue_remove_buffer(q, vb);
> + kfree(vb);
> +
> + dprintk(q, 2, "buffer %d deleted\n", index);
> + return 0;
> +}
> +
> /*
> * vb2_start_streaming() - Attempt to start streaming.
> * @q: videobuf2 queue
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index 724135d41f7f..cea666c17b41 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> }
> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>
> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> +{
> + return vb2_core_delete_buf(q, b->index);
> +}
> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> +
> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
> {
> unsigned requested_planes = 1;
> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
> index f81279492682..80ace2e1e932 100644
> --- a/drivers/media/v4l2-core/v4l2-dev.c
> +++ b/drivers/media/v4l2-core/v4l2-dev.c
> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct video_device *vdev)
> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF, vidioc_prepare_buf);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF, vidioc_streamoff);
> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF, vidioc_delete_buf);
> }
>
> if (is_vid || is_vbi || is_meta) {
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index a858acea6547..1c737279d3ef 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct v4l2_ioctl_ops *ops,
> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
> }
>
> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> + struct file *file, void *fh, void *arg)
> +{
> + struct v4l2_buffer *b = arg;
> + int ret = check_fmt(file, b->type);
> +
> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> +}
> +
> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> struct file *file, void *fh, void *arg)
> {
> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0),
> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf, v4l_print_buffer, INFO_FL_QUEUE),
> };
> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>
> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> index edb733f21604..2f232ed884c7 100644
> --- a/include/media/v4l2-ioctl.h
> +++ b/include/media/v4l2-ioctl.h
> @@ -163,6 +163,8 @@ struct v4l2_fh;
> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> * @vidioc_prepare_buf: pointer to the function that implements
> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> + * @vidioc_delete_buf: pointer to the function that implements
> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> * @vidioc_overlay: pointer to the function that implements
> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> * @vidioc_g_fbuf: pointer to the function that implements
> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
> struct v4l2_create_buffers *b);
> int (*vidioc_prepare_buf)(struct file *file, void *fh,
> struct v4l2_buffer *b);
> + int (*vidioc_delete_buf)(struct file *file, void *fh,
> + struct v4l2_buffer *b);
>
> int (*vidioc_overlay)(struct file *file, void *fh, unsigned int i);
> int (*vidioc_g_fbuf)(struct file *file, void *fh,
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 080b783d608d..0f9e68f76b77 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
> */
> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>
> +/**
> + * vb2_core_delete_buf() -
> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> + * @index: id number of the buffer.
> + *
> + * Return: returns zero on success; an error code otherwise.
> + */
> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> +
> /**
> * vb2_core_qbuf() - Queue a buffer from userspace
> *
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 88a7a565170e..3beeb4c735f0 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
> */
> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> struct v4l2_buffer *b);
> +/**
> + * vb2_delete_buf() - Delete the buffer from the queue
> + *
> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> + * @b: buffer structure passed from userspace to
> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> + *
> + * The return values from this function are intended to be directly returned
> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> + */
> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>
> /**
> * vb2_qbuf() - Queue a buffer from userspace
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index aee75eb9e686..31bba1915642 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct v4l2_dbg_chip_info)
>
> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
> +
>
> /* Reminder: when adding new ioctls please add support for them to
> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> --
> 2.39.2
>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-27 7:30 ` Hsia-Jun Li
(?)
@ 2023-06-27 8:43 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-27 8:43 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski
Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>
> On 6/22/23 21:13, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> .../userspace-api/media/v4l/user-func.rst | 1 +
>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>> include/media/v4l2-ioctl.h | 4 ++
>> include/media/videobuf2-core.h | 9 ++++
>> include/media/videobuf2-v4l2.h | 11 ++++
>> include/uapi/linux/videodev2.h | 2 +
>> 10 files changed, 128 insertions(+)
>> create mode 100644
>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>
>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>> b/Documentation/userspace-api/media/v4l/user-func.rst
>> index 15ff0bf7bbe6..8c74016e12fd 100644
>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>> @@ -17,6 +17,7 @@ Function Reference
>> vidioc-dbg-g-chip-info
>> vidioc-dbg-g-register
>> vidioc-decoder-cmd
>> + vidioc-delete-buf
>> vidioc-dqevent
>> vidioc-dv-timings-cap
>> vidioc-encoder-cmd
>> diff --git
>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> new file mode 100644
>> index 000000000000..0e7ce58f91bc
>> --- /dev/null
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> @@ -0,0 +1,51 @@
>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>> +.. c:namespace:: V4L
>> +
>> +.. _VIDIOC_DELETE_BUF:
>> +
>> +************************
>> +ioctl VIDIOC_DELETE_BUF
>> +************************
>> +
>> +Name
>> +====
>> +
>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>> +
>> +Synopsis
>> +========
>> +
>> +.. c:macro:: VIDIOC_DELETE_BUF
>> +
>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>> +
>> +Arguments
>> +=========
>> +
>> +``fd``
>> + File descriptor returned by :c:func:`open()`.
>> +
>> +``argp``
>> + Pointer to struct :c:type:`v4l2_buffer`.
>> +
>> +Description
>> +===========
>> +
>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>> +delete a buffer from a queue.
>> +
>> +The struct :c:type:`v4l2_buffer` structure is specified in
>> +:ref:`buffer`.
>> +
>> +Return Value
>> +============
>> +
>> +On success 0 is returned, on error -1 and the ``errno`` variable is set
>> +appropriately. The generic error codes are described at the
>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>> +
>> +EBUSY
>> + File I/O is in progress.
>> +
>> +EINVAL
>> + The buffer ``index`` doesn't exist in the queue.
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index 899783f67580..aa546c972c3d 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>> unsigned int index, void *pb)
>> }
>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>> +{
>> + struct vb2_buffer *vb;
>> +
>> + vb = vb2_get_buffer(q, index);
>> + if (!vb) {
>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>> + return -EINVAL;
>> + }
>> +
>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>> + dprintk(q, 1, "can't delete non dequeued buffer index
>> %d\n", index);
>> + return -EINVAL;
>> + }
>> +
> I know the driver could implement its own
> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
> used by the hardware as a future reference frame.
> But I think we need a flag to let the user know which buffer is still
> used by the hardware.
> Alternative ref case is safe, we only know it's existing when it is
> dequeued in current V4L2 buffer mechanism.
> While the Golden reference frame, such long term reference frame could
> last much longer.
It is up to userland stack to know frames life time, it got the information for that.
>> + if (vb->planes[0].mem_priv)
>> + call_void_vb_qop(vb, buf_cleanup, vb);
>> +
>> + /* Free MMAP buffers or release USERPTR buffers */
>> + if (q->memory == VB2_MEMORY_MMAP)
>> + __vb2_buf_mem_free(vb);
>> + else if (q->memory == VB2_MEMORY_DMABUF)
>> + __vb2_buf_dmabuf_put(vb);
>> + else
>> + __vb2_buf_userptr_put(vb);
>> +
>> + vb2_queue_remove_buffer(q, vb);
>> + kfree(vb);
>> +
>> + dprintk(q, 2, "buffer %d deleted\n", index);
>> + return 0;
>> +}
>> +
>> /*
>> * vb2_start_streaming() - Attempt to start streaming.
>> * @q: videobuf2 queue
>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> index 724135d41f7f..cea666c17b41 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>> media_device *mdev,
>> }
>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>> +{
>> + return vb2_core_delete_buf(q, b->index);
>> +}
>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>> +
>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>> *create)
>> {
>> unsigned requested_planes = 1;
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>> b/drivers/media/v4l2-core/v4l2-dev.c
>> index f81279492682..80ace2e1e932 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>> video_device *vdev)
>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>> vidioc_prepare_buf);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>> vidioc_streamoff);
>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>> vidioc_delete_buf);
>> }
>>
>> if (is_vid || is_vbi || is_meta) {
>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index a858acea6547..1c737279d3ef 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>> v4l2_ioctl_ops *ops,
>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>> }
>>
>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>> + struct file *file, void *fh, void *arg)
>> +{
>> + struct v4l2_buffer *b = arg;
>> + int ret = check_fmt(file, b->type);
>> +
>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>> +}
>> +
>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>> struct file *file, void *fh, void *arg)
>> {
>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>> v4l2_ioctls[] = {
>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>> v4l_print_freq_band, 0),
>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>> v4l_print_buffer, INFO_FL_QUEUE),
>> };
>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>
>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>> index edb733f21604..2f232ed884c7 100644
>> --- a/include/media/v4l2-ioctl.h
>> +++ b/include/media/v4l2-ioctl.h
>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>> * @vidioc_prepare_buf: pointer to the function that implements
>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>> + * @vidioc_delete_buf: pointer to the function that implements
>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>> * @vidioc_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>> * @vidioc_g_fbuf: pointer to the function that implements
>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>> struct v4l2_create_buffers *b);
>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>> struct v4l2_buffer *b);
>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>> + struct v4l2_buffer *b);
>>
>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>> int i);
>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>> diff --git a/include/media/videobuf2-core.h
>> b/include/media/videobuf2-core.h
>> index 080b783d608d..0f9e68f76b77 100644
>> --- a/include/media/videobuf2-core.h
>> +++ b/include/media/videobuf2-core.h
>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>> enum vb2_memory memory,
>> */
>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>> void *pb);
>>
>> +/**
>> + * vb2_core_delete_buf() -
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @index: id number of the buffer.
>> + *
>> + * Return: returns zero on success; an error code otherwise.
>> + */
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>> +
>> /**
>> * vb2_core_qbuf() - Queue a buffer from userspace
>> *
>> diff --git a/include/media/videobuf2-v4l2.h
>> b/include/media/videobuf2-v4l2.h
>> index 88a7a565170e..3beeb4c735f0 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>> v4l2_create_buffers *create);
>> */
>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>> struct v4l2_buffer *b);
>> +/**
>> + * vb2_delete_buf() - Delete the buffer from the queue
>> + *
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @b: buffer structure passed from userspace to
>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>> + *
>> + * The return values from this function are intended to be directly
>> returned
>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>> + */
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>
>> /**
>> * vb2_qbuf() - Queue a buffer from userspace
>> diff --git a/include/uapi/linux/videodev2.h
>> b/include/uapi/linux/videodev2.h
>> index aee75eb9e686..31bba1915642 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>> v4l2_dbg_chip_info)
>>
>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>> v4l2_query_ext_ctrl)
>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>> +
>>
>> /* Reminder: when adding new ioctls please add support for them to
>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>> --
>> 2.39.2
>>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-27 8:43 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-27 8:43 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski
Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>
> On 6/22/23 21:13, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> .../userspace-api/media/v4l/user-func.rst | 1 +
>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>> include/media/v4l2-ioctl.h | 4 ++
>> include/media/videobuf2-core.h | 9 ++++
>> include/media/videobuf2-v4l2.h | 11 ++++
>> include/uapi/linux/videodev2.h | 2 +
>> 10 files changed, 128 insertions(+)
>> create mode 100644
>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>
>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>> b/Documentation/userspace-api/media/v4l/user-func.rst
>> index 15ff0bf7bbe6..8c74016e12fd 100644
>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>> @@ -17,6 +17,7 @@ Function Reference
>> vidioc-dbg-g-chip-info
>> vidioc-dbg-g-register
>> vidioc-decoder-cmd
>> + vidioc-delete-buf
>> vidioc-dqevent
>> vidioc-dv-timings-cap
>> vidioc-encoder-cmd
>> diff --git
>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> new file mode 100644
>> index 000000000000..0e7ce58f91bc
>> --- /dev/null
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> @@ -0,0 +1,51 @@
>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>> +.. c:namespace:: V4L
>> +
>> +.. _VIDIOC_DELETE_BUF:
>> +
>> +************************
>> +ioctl VIDIOC_DELETE_BUF
>> +************************
>> +
>> +Name
>> +====
>> +
>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>> +
>> +Synopsis
>> +========
>> +
>> +.. c:macro:: VIDIOC_DELETE_BUF
>> +
>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>> +
>> +Arguments
>> +=========
>> +
>> +``fd``
>> + File descriptor returned by :c:func:`open()`.
>> +
>> +``argp``
>> + Pointer to struct :c:type:`v4l2_buffer`.
>> +
>> +Description
>> +===========
>> +
>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>> +delete a buffer from a queue.
>> +
>> +The struct :c:type:`v4l2_buffer` structure is specified in
>> +:ref:`buffer`.
>> +
>> +Return Value
>> +============
>> +
>> +On success 0 is returned, on error -1 and the ``errno`` variable is set
>> +appropriately. The generic error codes are described at the
>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>> +
>> +EBUSY
>> + File I/O is in progress.
>> +
>> +EINVAL
>> + The buffer ``index`` doesn't exist in the queue.
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index 899783f67580..aa546c972c3d 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>> unsigned int index, void *pb)
>> }
>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>> +{
>> + struct vb2_buffer *vb;
>> +
>> + vb = vb2_get_buffer(q, index);
>> + if (!vb) {
>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>> + return -EINVAL;
>> + }
>> +
>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>> + dprintk(q, 1, "can't delete non dequeued buffer index
>> %d\n", index);
>> + return -EINVAL;
>> + }
>> +
> I know the driver could implement its own
> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
> used by the hardware as a future reference frame.
> But I think we need a flag to let the user know which buffer is still
> used by the hardware.
> Alternative ref case is safe, we only know it's existing when it is
> dequeued in current V4L2 buffer mechanism.
> While the Golden reference frame, such long term reference frame could
> last much longer.
It is up to userland stack to know frames life time, it got the information for that.
>> + if (vb->planes[0].mem_priv)
>> + call_void_vb_qop(vb, buf_cleanup, vb);
>> +
>> + /* Free MMAP buffers or release USERPTR buffers */
>> + if (q->memory == VB2_MEMORY_MMAP)
>> + __vb2_buf_mem_free(vb);
>> + else if (q->memory == VB2_MEMORY_DMABUF)
>> + __vb2_buf_dmabuf_put(vb);
>> + else
>> + __vb2_buf_userptr_put(vb);
>> +
>> + vb2_queue_remove_buffer(q, vb);
>> + kfree(vb);
>> +
>> + dprintk(q, 2, "buffer %d deleted\n", index);
>> + return 0;
>> +}
>> +
>> /*
>> * vb2_start_streaming() - Attempt to start streaming.
>> * @q: videobuf2 queue
>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> index 724135d41f7f..cea666c17b41 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>> media_device *mdev,
>> }
>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>> +{
>> + return vb2_core_delete_buf(q, b->index);
>> +}
>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>> +
>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>> *create)
>> {
>> unsigned requested_planes = 1;
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>> b/drivers/media/v4l2-core/v4l2-dev.c
>> index f81279492682..80ace2e1e932 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>> video_device *vdev)
>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>> vidioc_prepare_buf);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>> vidioc_streamoff);
>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>> vidioc_delete_buf);
>> }
>>
>> if (is_vid || is_vbi || is_meta) {
>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index a858acea6547..1c737279d3ef 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>> v4l2_ioctl_ops *ops,
>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>> }
>>
>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>> + struct file *file, void *fh, void *arg)
>> +{
>> + struct v4l2_buffer *b = arg;
>> + int ret = check_fmt(file, b->type);
>> +
>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>> +}
>> +
>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>> struct file *file, void *fh, void *arg)
>> {
>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>> v4l2_ioctls[] = {
>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>> v4l_print_freq_band, 0),
>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>> v4l_print_buffer, INFO_FL_QUEUE),
>> };
>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>
>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>> index edb733f21604..2f232ed884c7 100644
>> --- a/include/media/v4l2-ioctl.h
>> +++ b/include/media/v4l2-ioctl.h
>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>> * @vidioc_prepare_buf: pointer to the function that implements
>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>> + * @vidioc_delete_buf: pointer to the function that implements
>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>> * @vidioc_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>> * @vidioc_g_fbuf: pointer to the function that implements
>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>> struct v4l2_create_buffers *b);
>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>> struct v4l2_buffer *b);
>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>> + struct v4l2_buffer *b);
>>
>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>> int i);
>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>> diff --git a/include/media/videobuf2-core.h
>> b/include/media/videobuf2-core.h
>> index 080b783d608d..0f9e68f76b77 100644
>> --- a/include/media/videobuf2-core.h
>> +++ b/include/media/videobuf2-core.h
>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>> enum vb2_memory memory,
>> */
>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>> void *pb);
>>
>> +/**
>> + * vb2_core_delete_buf() -
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @index: id number of the buffer.
>> + *
>> + * Return: returns zero on success; an error code otherwise.
>> + */
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>> +
>> /**
>> * vb2_core_qbuf() - Queue a buffer from userspace
>> *
>> diff --git a/include/media/videobuf2-v4l2.h
>> b/include/media/videobuf2-v4l2.h
>> index 88a7a565170e..3beeb4c735f0 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>> v4l2_create_buffers *create);
>> */
>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>> struct v4l2_buffer *b);
>> +/**
>> + * vb2_delete_buf() - Delete the buffer from the queue
>> + *
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @b: buffer structure passed from userspace to
>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>> + *
>> + * The return values from this function are intended to be directly
>> returned
>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>> + */
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>
>> /**
>> * vb2_qbuf() - Queue a buffer from userspace
>> diff --git a/include/uapi/linux/videodev2.h
>> b/include/uapi/linux/videodev2.h
>> index aee75eb9e686..31bba1915642 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>> v4l2_dbg_chip_info)
>>
>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>> v4l2_query_ext_ctrl)
>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>> +
>>
>> /* Reminder: when adding new ioctls please add support for them to
>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>> --
>> 2.39.2
>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-27 8:43 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-27 8:43 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski
Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>
> On 6/22/23 21:13, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> .../userspace-api/media/v4l/user-func.rst | 1 +
>> .../media/v4l/vidioc-delete-buf.rst | 51 +++++++++++++++++++
>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>> include/media/v4l2-ioctl.h | 4 ++
>> include/media/videobuf2-core.h | 9 ++++
>> include/media/videobuf2-v4l2.h | 11 ++++
>> include/uapi/linux/videodev2.h | 2 +
>> 10 files changed, 128 insertions(+)
>> create mode 100644
>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>
>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>> b/Documentation/userspace-api/media/v4l/user-func.rst
>> index 15ff0bf7bbe6..8c74016e12fd 100644
>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>> @@ -17,6 +17,7 @@ Function Reference
>> vidioc-dbg-g-chip-info
>> vidioc-dbg-g-register
>> vidioc-decoder-cmd
>> + vidioc-delete-buf
>> vidioc-dqevent
>> vidioc-dv-timings-cap
>> vidioc-encoder-cmd
>> diff --git
>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> new file mode 100644
>> index 000000000000..0e7ce58f91bc
>> --- /dev/null
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>> @@ -0,0 +1,51 @@
>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>> +.. c:namespace:: V4L
>> +
>> +.. _VIDIOC_DELETE_BUF:
>> +
>> +************************
>> +ioctl VIDIOC_DELETE_BUF
>> +************************
>> +
>> +Name
>> +====
>> +
>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>> +
>> +Synopsis
>> +========
>> +
>> +.. c:macro:: VIDIOC_DELETE_BUF
>> +
>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>> +
>> +Arguments
>> +=========
>> +
>> +``fd``
>> + File descriptor returned by :c:func:`open()`.
>> +
>> +``argp``
>> + Pointer to struct :c:type:`v4l2_buffer`.
>> +
>> +Description
>> +===========
>> +
>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>> +delete a buffer from a queue.
>> +
>> +The struct :c:type:`v4l2_buffer` structure is specified in
>> +:ref:`buffer`.
>> +
>> +Return Value
>> +============
>> +
>> +On success 0 is returned, on error -1 and the ``errno`` variable is set
>> +appropriately. The generic error codes are described at the
>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>> +
>> +EBUSY
>> + File I/O is in progress.
>> +
>> +EINVAL
>> + The buffer ``index`` doesn't exist in the queue.
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>> b/drivers/media/common/videobuf2/videobuf2-core.c
>> index 899783f67580..aa546c972c3d 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>> unsigned int index, void *pb)
>> }
>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>> +{
>> + struct vb2_buffer *vb;
>> +
>> + vb = vb2_get_buffer(q, index);
>> + if (!vb) {
>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>> + return -EINVAL;
>> + }
>> +
>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>> + dprintk(q, 1, "can't delete non dequeued buffer index
>> %d\n", index);
>> + return -EINVAL;
>> + }
>> +
> I know the driver could implement its own
> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
> used by the hardware as a future reference frame.
> But I think we need a flag to let the user know which buffer is still
> used by the hardware.
> Alternative ref case is safe, we only know it's existing when it is
> dequeued in current V4L2 buffer mechanism.
> While the Golden reference frame, such long term reference frame could
> last much longer.
It is up to userland stack to know frames life time, it got the information for that.
>> + if (vb->planes[0].mem_priv)
>> + call_void_vb_qop(vb, buf_cleanup, vb);
>> +
>> + /* Free MMAP buffers or release USERPTR buffers */
>> + if (q->memory == VB2_MEMORY_MMAP)
>> + __vb2_buf_mem_free(vb);
>> + else if (q->memory == VB2_MEMORY_DMABUF)
>> + __vb2_buf_dmabuf_put(vb);
>> + else
>> + __vb2_buf_userptr_put(vb);
>> +
>> + vb2_queue_remove_buffer(q, vb);
>> + kfree(vb);
>> +
>> + dprintk(q, 2, "buffer %d deleted\n", index);
>> + return 0;
>> +}
>> +
>> /*
>> * vb2_start_streaming() - Attempt to start streaming.
>> * @q: videobuf2 queue
>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> index 724135d41f7f..cea666c17b41 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>> media_device *mdev,
>> }
>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>> +{
>> + return vb2_core_delete_buf(q, b->index);
>> +}
>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>> +
>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>> *create)
>> {
>> unsigned requested_planes = 1;
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>> b/drivers/media/v4l2-core/v4l2-dev.c
>> index f81279492682..80ace2e1e932 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>> video_device *vdev)
>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>> vidioc_prepare_buf);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON, vidioc_streamon);
>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>> vidioc_streamoff);
>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>> vidioc_delete_buf);
>> }
>>
>> if (is_vid || is_vbi || is_meta) {
>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index a858acea6547..1c737279d3ef 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>> v4l2_ioctl_ops *ops,
>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>> }
>>
>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>> + struct file *file, void *fh, void *arg)
>> +{
>> + struct v4l2_buffer *b = arg;
>> + int ret = check_fmt(file, b->type);
>> +
>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>> +}
>> +
>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>> struct file *file, void *fh, void *arg)
>> {
>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>> v4l2_ioctls[] = {
>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>> v4l_print_freq_band, 0),
>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>> v4l_print_buffer, INFO_FL_QUEUE),
>> };
>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>
>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>> index edb733f21604..2f232ed884c7 100644
>> --- a/include/media/v4l2-ioctl.h
>> +++ b/include/media/v4l2-ioctl.h
>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>> * @vidioc_prepare_buf: pointer to the function that implements
>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>> + * @vidioc_delete_buf: pointer to the function that implements
>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>> * @vidioc_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>> * @vidioc_g_fbuf: pointer to the function that implements
>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>> struct v4l2_create_buffers *b);
>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>> struct v4l2_buffer *b);
>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>> + struct v4l2_buffer *b);
>>
>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>> int i);
>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>> diff --git a/include/media/videobuf2-core.h
>> b/include/media/videobuf2-core.h
>> index 080b783d608d..0f9e68f76b77 100644
>> --- a/include/media/videobuf2-core.h
>> +++ b/include/media/videobuf2-core.h
>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>> enum vb2_memory memory,
>> */
>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>> void *pb);
>>
>> +/**
>> + * vb2_core_delete_buf() -
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @index: id number of the buffer.
>> + *
>> + * Return: returns zero on success; an error code otherwise.
>> + */
>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>> +
>> /**
>> * vb2_core_qbuf() - Queue a buffer from userspace
>> *
>> diff --git a/include/media/videobuf2-v4l2.h
>> b/include/media/videobuf2-v4l2.h
>> index 88a7a565170e..3beeb4c735f0 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>> v4l2_create_buffers *create);
>> */
>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>> struct v4l2_buffer *b);
>> +/**
>> + * vb2_delete_buf() - Delete the buffer from the queue
>> + *
>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>> + * @b: buffer structure passed from userspace to
>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>> + *
>> + * The return values from this function are intended to be directly
>> returned
>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>> + */
>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>
>> /**
>> * vb2_qbuf() - Queue a buffer from userspace
>> diff --git a/include/uapi/linux/videodev2.h
>> b/include/uapi/linux/videodev2.h
>> index aee75eb9e686..31bba1915642 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>> v4l2_dbg_chip_info)
>>
>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>> v4l2_query_ext_ctrl)
>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>> +
>>
>> /* Reminder: when adding new ioctls please add support for them to
>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>> --
>> 2.39.2
>>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-27 8:43 ` Benjamin Gaignard
(?)
@ 2023-06-27 8:47 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-27 8:47 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski
On 6/27/23 16:43, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>
>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>> +++++++++++++++++++
>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>> include/media/v4l2-ioctl.h | 4 ++
>>> include/media/videobuf2-core.h | 9 ++++
>>> include/media/videobuf2-v4l2.h | 11 ++++
>>> include/uapi/linux/videodev2.h | 2 +
>>> 10 files changed, 128 insertions(+)
>>> create mode 100644
>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>> vidioc-dbg-g-chip-info
>>> vidioc-dbg-g-register
>>> vidioc-decoder-cmd
>>> + vidioc-delete-buf
>>> vidioc-dqevent
>>> vidioc-dv-timings-cap
>>> vidioc-encoder-cmd
>>> diff --git
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>> +.. c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> + File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>>> +delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>> set
>>> +appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> + File I/O is in progress.
>>> +
>>> +EINVAL
>>> + The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb)
>>> }
>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>> +{
>>> + struct vb2_buffer *vb;
>>> +
>>> + vb = vb2_get_buffer(q, index);
>>> + if (!vb) {
>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>> %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>> I know the driver could implement its own
>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>> used by the hardware as a future reference frame.
>> But I think we need a flag to let the user know which buffer is still
>> used by the hardware.
>> Alternative ref case is safe, we only know it's existing when it is
>> dequeued in current V4L2 buffer mechanism.
>> While the Golden reference frame, such long term reference frame could
>> last much longer.
>
> It is up to userland stack to know frames life time, it got the
> information for that.
That is true for the stateless codec driver.
While application for stateful decoder could never do that. It also
breaks what the document said:
"The backing memory of |CAPTURE| buffers that are used as reference
frames by the stream may be read by the hardware even after they are
dequeued. Consequently, the client should avoid writing into this memory
while the |CAPTURE| queue is streaming. Failure to observe this may
result in corruption of decoded frames."
>
>>> + if (vb->planes[0].mem_priv)
>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> + /* Free MMAP buffers or release USERPTR buffers */
>>> + if (q->memory == VB2_MEMORY_MMAP)
>>> + __vb2_buf_mem_free(vb);
>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>> + __vb2_buf_dmabuf_put(vb);
>>> + else
>>> + __vb2_buf_userptr_put(vb);
>>> +
>>> + vb2_queue_remove_buffer(q, vb);
>>> + kfree(vb);
>>> +
>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>> + return 0;
>>> +}
>>> +
>>> /*
>>> * vb2_start_streaming() - Attempt to start streaming.
>>> * @q: videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev,
>>> }
>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>> +{
>>> + return vb2_core_delete_buf(q, b->index);
>>> +}
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>> *create)
>>> {
>>> unsigned requested_planes = 1;
>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>> index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>> video_device *vdev)
>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>> vidioc_prepare_buf);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>> vidioc_streamon);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>> vidioc_streamoff);
>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> vidioc_delete_buf);
>>> }
>>>
>>> if (is_vid || is_vbi || is_meta) {
>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>> }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> + struct file *file, void *fh, void *arg)
>>> +{
>>> + struct v4l2_buffer *b = arg;
>>> + int ret = check_fmt(file, b->type);
>>> +
>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>> +}
>>> +
>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>> struct file *file, void *fh, void
>>> *arg)
>>> {
>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>> v4l2_ioctls[] = {
>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>> v4l_print_buffer, INFO_FL_QUEUE),
>>> };
>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>> index edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>> * @vidioc_prepare_buf: pointer to the function that implements
>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>> * @vidioc_overlay: pointer to the function that implements
>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>> * @vidioc_g_fbuf: pointer to the function that implements
>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>> struct v4l2_create_buffers *b);
>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>> struct v4l2_buffer *b);
>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> + struct v4l2_buffer *b);
>>>
>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>> int i);
>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>> diff --git a/include/media/videobuf2-core.h
>>> b/include/media/videobuf2-core.h
>>> index 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>> */
>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index: id number of the buffer.
>>> + *
>>> + * Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>> /**
>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>> *
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h
>>> index 88a7a565170e..3beeb4c735f0 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>> */
>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>> struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b: buffer structure passed from userspace to
>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>> /**
>>> * vb2_qbuf() - Queue a buffer from userspace
>>> diff --git a/include/uapi/linux/videodev2.h
>>> b/include/uapi/linux/videodev2.h
>>> index aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>> v4l2_dbg_chip_info)
>>>
>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>> /* Reminder: when adding new ioctls please add support for them to
>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> --
>>> 2.39.2
>>>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-27 8:47 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-27 8:47 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski
On 6/27/23 16:43, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>
>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>> +++++++++++++++++++
>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>> include/media/v4l2-ioctl.h | 4 ++
>>> include/media/videobuf2-core.h | 9 ++++
>>> include/media/videobuf2-v4l2.h | 11 ++++
>>> include/uapi/linux/videodev2.h | 2 +
>>> 10 files changed, 128 insertions(+)
>>> create mode 100644
>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>> vidioc-dbg-g-chip-info
>>> vidioc-dbg-g-register
>>> vidioc-decoder-cmd
>>> + vidioc-delete-buf
>>> vidioc-dqevent
>>> vidioc-dv-timings-cap
>>> vidioc-encoder-cmd
>>> diff --git
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>> +.. c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> + File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>>> +delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>> set
>>> +appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> + File I/O is in progress.
>>> +
>>> +EINVAL
>>> + The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb)
>>> }
>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>> +{
>>> + struct vb2_buffer *vb;
>>> +
>>> + vb = vb2_get_buffer(q, index);
>>> + if (!vb) {
>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>> %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>> I know the driver could implement its own
>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>> used by the hardware as a future reference frame.
>> But I think we need a flag to let the user know which buffer is still
>> used by the hardware.
>> Alternative ref case is safe, we only know it's existing when it is
>> dequeued in current V4L2 buffer mechanism.
>> While the Golden reference frame, such long term reference frame could
>> last much longer.
>
> It is up to userland stack to know frames life time, it got the
> information for that.
That is true for the stateless codec driver.
While application for stateful decoder could never do that. It also
breaks what the document said:
"The backing memory of |CAPTURE| buffers that are used as reference
frames by the stream may be read by the hardware even after they are
dequeued. Consequently, the client should avoid writing into this memory
while the |CAPTURE| queue is streaming. Failure to observe this may
result in corruption of decoded frames."
>
>>> + if (vb->planes[0].mem_priv)
>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> + /* Free MMAP buffers or release USERPTR buffers */
>>> + if (q->memory == VB2_MEMORY_MMAP)
>>> + __vb2_buf_mem_free(vb);
>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>> + __vb2_buf_dmabuf_put(vb);
>>> + else
>>> + __vb2_buf_userptr_put(vb);
>>> +
>>> + vb2_queue_remove_buffer(q, vb);
>>> + kfree(vb);
>>> +
>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>> + return 0;
>>> +}
>>> +
>>> /*
>>> * vb2_start_streaming() - Attempt to start streaming.
>>> * @q: videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev,
>>> }
>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>> +{
>>> + return vb2_core_delete_buf(q, b->index);
>>> +}
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>> *create)
>>> {
>>> unsigned requested_planes = 1;
>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>> index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>> video_device *vdev)
>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>> vidioc_prepare_buf);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>> vidioc_streamon);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>> vidioc_streamoff);
>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> vidioc_delete_buf);
>>> }
>>>
>>> if (is_vid || is_vbi || is_meta) {
>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>> }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> + struct file *file, void *fh, void *arg)
>>> +{
>>> + struct v4l2_buffer *b = arg;
>>> + int ret = check_fmt(file, b->type);
>>> +
>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>> +}
>>> +
>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>> struct file *file, void *fh, void
>>> *arg)
>>> {
>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>> v4l2_ioctls[] = {
>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>> v4l_print_buffer, INFO_FL_QUEUE),
>>> };
>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>> index edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>> * @vidioc_prepare_buf: pointer to the function that implements
>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>> * @vidioc_overlay: pointer to the function that implements
>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>> * @vidioc_g_fbuf: pointer to the function that implements
>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>> struct v4l2_create_buffers *b);
>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>> struct v4l2_buffer *b);
>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> + struct v4l2_buffer *b);
>>>
>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>> int i);
>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>> diff --git a/include/media/videobuf2-core.h
>>> b/include/media/videobuf2-core.h
>>> index 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>> */
>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index: id number of the buffer.
>>> + *
>>> + * Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>> /**
>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>> *
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h
>>> index 88a7a565170e..3beeb4c735f0 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>> */
>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>> struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b: buffer structure passed from userspace to
>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>> /**
>>> * vb2_qbuf() - Queue a buffer from userspace
>>> diff --git a/include/uapi/linux/videodev2.h
>>> b/include/uapi/linux/videodev2.h
>>> index aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>> v4l2_dbg_chip_info)
>>>
>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>> /* Reminder: when adding new ioctls please add support for them to
>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> --
>>> 2.39.2
>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-27 8:47 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-27 8:47 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski
On 6/27/23 16:43, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>
>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>> +++++++++++++++++++
>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>> include/media/v4l2-ioctl.h | 4 ++
>>> include/media/videobuf2-core.h | 9 ++++
>>> include/media/videobuf2-v4l2.h | 11 ++++
>>> include/uapi/linux/videodev2.h | 2 +
>>> 10 files changed, 128 insertions(+)
>>> create mode 100644
>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>> @@ -17,6 +17,7 @@ Function Reference
>>> vidioc-dbg-g-chip-info
>>> vidioc-dbg-g-register
>>> vidioc-decoder-cmd
>>> + vidioc-delete-buf
>>> vidioc-dqevent
>>> vidioc-dv-timings-cap
>>> vidioc-encoder-cmd
>>> diff --git
>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> new file mode 100644
>>> index 000000000000..0e7ce58f91bc
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>> @@ -0,0 +1,51 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>> +.. c:namespace:: V4L
>>> +
>>> +.. _VIDIOC_DELETE_BUF:
>>> +
>>> +************************
>>> +ioctl VIDIOC_DELETE_BUF
>>> +************************
>>> +
>>> +Name
>>> +====
>>> +
>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>> +
>>> +Synopsis
>>> +========
>>> +
>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>> +
>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>> +
>>> +Arguments
>>> +=========
>>> +
>>> +``fd``
>>> + File descriptor returned by :c:func:`open()`.
>>> +
>>> +``argp``
>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>> +
>>> +Description
>>> +===========
>>> +
>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF` ioctl to
>>> +delete a buffer from a queue.
>>> +
>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>> +:ref:`buffer`.
>>> +
>>> +Return Value
>>> +============
>>> +
>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>> set
>>> +appropriately. The generic error codes are described at the
>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>> +
>>> +EBUSY
>>> + File I/O is in progress.
>>> +
>>> +EINVAL
>>> + The buffer ``index`` doesn't exist in the queue.
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>> index 899783f67580..aa546c972c3d 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>> unsigned int index, void *pb)
>>> }
>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>> +{
>>> + struct vb2_buffer *vb;
>>> +
>>> + vb = vb2_get_buffer(q, index);
>>> + if (!vb) {
>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>> %d\n", index);
>>> + return -EINVAL;
>>> + }
>>> +
>> I know the driver could implement its own
>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>> used by the hardware as a future reference frame.
>> But I think we need a flag to let the user know which buffer is still
>> used by the hardware.
>> Alternative ref case is safe, we only know it's existing when it is
>> dequeued in current V4L2 buffer mechanism.
>> While the Golden reference frame, such long term reference frame could
>> last much longer.
>
> It is up to userland stack to know frames life time, it got the
> information for that.
That is true for the stateless codec driver.
While application for stateful decoder could never do that. It also
breaks what the document said:
"The backing memory of |CAPTURE| buffers that are used as reference
frames by the stream may be read by the hardware even after they are
dequeued. Consequently, the client should avoid writing into this memory
while the |CAPTURE| queue is streaming. Failure to observe this may
result in corruption of decoded frames."
>
>>> + if (vb->planes[0].mem_priv)
>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>> +
>>> + /* Free MMAP buffers or release USERPTR buffers */
>>> + if (q->memory == VB2_MEMORY_MMAP)
>>> + __vb2_buf_mem_free(vb);
>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>> + __vb2_buf_dmabuf_put(vb);
>>> + else
>>> + __vb2_buf_userptr_put(vb);
>>> +
>>> + vb2_queue_remove_buffer(q, vb);
>>> + kfree(vb);
>>> +
>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>> + return 0;
>>> +}
>>> +
>>> /*
>>> * vb2_start_streaming() - Attempt to start streaming.
>>> * @q: videobuf2 queue
>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> index 724135d41f7f..cea666c17b41 100644
>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>> media_device *mdev,
>>> }
>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>> +{
>>> + return vb2_core_delete_buf(q, b->index);
>>> +}
>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>> +
>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>> *create)
>>> {
>>> unsigned requested_planes = 1;
>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>> index f81279492682..80ace2e1e932 100644
>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>> video_device *vdev)
>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>> vidioc_prepare_buf);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>> vidioc_streamon);
>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>> vidioc_streamoff);
>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>> vidioc_delete_buf);
>>> }
>>>
>>> if (is_vid || is_vbi || is_meta) {
>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index a858acea6547..1c737279d3ef 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>> v4l2_ioctl_ops *ops,
>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>> }
>>>
>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>> + struct file *file, void *fh, void *arg)
>>> +{
>>> + struct v4l2_buffer *b = arg;
>>> + int ret = check_fmt(file, b->type);
>>> +
>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>> +}
>>> +
>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>> struct file *file, void *fh, void
>>> *arg)
>>> {
>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>> v4l2_ioctls[] = {
>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>> v4l_print_freq_band, 0),
>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>> v4l_print_buffer, INFO_FL_QUEUE),
>>> };
>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>
>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>> index edb733f21604..2f232ed884c7 100644
>>> --- a/include/media/v4l2-ioctl.h
>>> +++ b/include/media/v4l2-ioctl.h
>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>> * @vidioc_prepare_buf: pointer to the function that implements
>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>> + * @vidioc_delete_buf: pointer to the function that implements
>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>> * @vidioc_overlay: pointer to the function that implements
>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>> * @vidioc_g_fbuf: pointer to the function that implements
>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>> struct v4l2_create_buffers *b);
>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>> struct v4l2_buffer *b);
>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>> + struct v4l2_buffer *b);
>>>
>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>> int i);
>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>> diff --git a/include/media/videobuf2-core.h
>>> b/include/media/videobuf2-core.h
>>> index 080b783d608d..0f9e68f76b77 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>> enum vb2_memory memory,
>>> */
>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>> void *pb);
>>>
>>> +/**
>>> + * vb2_core_delete_buf() -
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @index: id number of the buffer.
>>> + *
>>> + * Return: returns zero on success; an error code otherwise.
>>> + */
>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>> +
>>> /**
>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>> *
>>> diff --git a/include/media/videobuf2-v4l2.h
>>> b/include/media/videobuf2-v4l2.h
>>> index 88a7a565170e..3beeb4c735f0 100644
>>> --- a/include/media/videobuf2-v4l2.h
>>> +++ b/include/media/videobuf2-v4l2.h
>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>> v4l2_create_buffers *create);
>>> */
>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>> struct v4l2_buffer *b);
>>> +/**
>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>> + *
>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>> + * @b: buffer structure passed from userspace to
>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>> + *
>>> + * The return values from this function are intended to be directly
>>> returned
>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>> + */
>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>
>>> /**
>>> * vb2_qbuf() - Queue a buffer from userspace
>>> diff --git a/include/uapi/linux/videodev2.h
>>> b/include/uapi/linux/videodev2.h
>>> index aee75eb9e686..31bba1915642 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>> v4l2_dbg_chip_info)
>>>
>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>> v4l2_query_ext_ctrl)
>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>> +
>>>
>>> /* Reminder: when adding new ioctls please add support for them to
>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>> --
>>> 2.39.2
>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-27 8:47 ` Hsia-Jun Li
(?)
@ 2023-06-30 9:43 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-30 9:43 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
On 6/27/23 16:47, Hsia-Jun Li wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> On 6/27/23 16:43, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>
>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> ---
>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>> +++++++++++++++++++
>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>> include/media/v4l2-ioctl.h | 4 ++
>>>> include/media/videobuf2-core.h | 9 ++++
>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>> include/uapi/linux/videodev2.h | 2 +
>>>> 10 files changed, 128 insertions(+)
>>>> create mode 100644
>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> @@ -17,6 +17,7 @@ Function Reference
>>>> vidioc-dbg-g-chip-info
>>>> vidioc-dbg-g-register
>>>> vidioc-decoder-cmd
>>>> + vidioc-delete-buf
>>>> vidioc-dqevent
>>>> vidioc-dv-timings-cap
>>>> vidioc-encoder-cmd
>>>> diff --git
>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> new file mode 100644
>>>> index 000000000000..0e7ce58f91bc
>>>> --- /dev/null
>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> @@ -0,0 +1,51 @@
>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>> +.. c:namespace:: V4L
>>>> +
>>>> +.. _VIDIOC_DELETE_BUF:
>>>> +
>>>> +************************
>>>> +ioctl VIDIOC_DELETE_BUF
>>>> +************************
>>>> +
>>>> +Name
>>>> +====
>>>> +
>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>> +
>>>> +Synopsis
>>>> +========
>>>> +
>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>> +
>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>> +
>>>> +Arguments
>>>> +=========
>>>> +
>>>> +``fd``
>>>> + File descriptor returned by :c:func:`open()`.
>>>> +
>>>> +``argp``
>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>> +
>>>> +Description
>>>> +===========
>>>> +
>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>> ioctl to
>>>> +delete a buffer from a queue.
>>>> +
>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>> +:ref:`buffer`.
>>>> +
>>>> +Return Value
>>>> +============
>>>> +
>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>> set
>>>> +appropriately. The generic error codes are described at the
>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>> +
>>>> +EBUSY
>>>> + File I/O is in progress.
>>>> +
>>>> +EINVAL
>>>> + The buffer ``index`` doesn't exist in the queue.
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index 899783f67580..aa546c972c3d 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>> unsigned int index, void *pb)
>>>> }
>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>> +{
>>>> + struct vb2_buffer *vb;
>>>> +
>>>> + vb = vb2_get_buffer(q, index);
>>>> + if (!vb) {
>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>> %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>> I know the driver could implement its own
>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>> used by the hardware as a future reference frame.
>>> But I think we need a flag to let the user know which buffer is still
>>> used by the hardware.
>>> Alternative ref case is safe, we only know it's existing when it is
>>> dequeued in current V4L2 buffer mechanism.
>>> While the Golden reference frame, such long term reference frame could
>>> last much longer.
>>
>> It is up to userland stack to know frames life time, it got the
>> information for that.
>
> That is true for the stateless codec driver.
>
> While application for stateful decoder could never do that. It also
> breaks what the document said:
>
> "The backing memory of |CAPTURE| buffers that are used as reference
> frames by the stream may be read by the hardware even after they are
> dequeued. Consequently, the client should avoid writing into this memory
> while the |CAPTURE| queue is streaming. Failure to observe this may
> result in corruption of decoded frames."
>
>>
>>>> + if (vb->planes[0].mem_priv)
>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>> +
>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>> + __vb2_buf_mem_free(vb);
Here is another problem for the existing application, the mmap() from
the mmap offset or exportbuffer fd would not create a reference to
buffer in this step(while the exportbuffer would create one itself).
When you delete a buffer, you may not release it from its virtual memory
space, leaving a corrupted virtual memory space. Also this behavior is
right, because mmap(2) says:
"After the mmap() call has returned, the file descriptor, fd, can be
closed immediately without invalidating the map‐ping."
>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>> + __vb2_buf_dmabuf_put(vb);
>>>> + else
>>>> + __vb2_buf_userptr_put(vb);
>>>> +
>>>> + vb2_queue_remove_buffer(q, vb);
>>>> + kfree(vb);
>>>> +
>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>> + return 0;
>>>> +}
>>>> +
>>>> /*
>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>> * @q: videobuf2 queue
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> index 724135d41f7f..cea666c17b41 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>> media_device *mdev,
>>>> }
>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>> +{
>>>> + return vb2_core_delete_buf(q, b->index);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>> +
>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>> *create)
>>>> {
>>>> unsigned requested_planes = 1;
>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>> index f81279492682..80ace2e1e932 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>> video_device *vdev)
>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>> vidioc_prepare_buf);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>> vidioc_streamon);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>> vidioc_streamoff);
>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>> vidioc_delete_buf);
>>>> }
>>>>
>>>> if (is_vid || is_vbi || is_meta) {
>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index a858acea6547..1c737279d3ef 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>> v4l2_ioctl_ops *ops,
>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>> }
>>>>
>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>> + struct file *file, void *fh, void *arg)
>>>> +{
>>>> + struct v4l2_buffer *b = arg;
>>>> + int ret = check_fmt(file, b->type);
>>>> +
>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>> +}
>>>> +
>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>> struct file *file, void *fh, void
>>>> *arg)
>>>> {
>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>> v4l2_ioctls[] = {
>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>> v4l_print_freq_band, 0),
>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>> };
>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>
>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>> index edb733f21604..2f232ed884c7 100644
>>>> --- a/include/media/v4l2-ioctl.h
>>>> +++ b/include/media/v4l2-ioctl.h
>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>> * @vidioc_overlay: pointer to the function that implements
>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>> struct v4l2_create_buffers *b);
>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>> struct v4l2_buffer *b);
>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>> + struct v4l2_buffer *b);
>>>>
>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>> int i);
>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>> diff --git a/include/media/videobuf2-core.h
>>>> b/include/media/videobuf2-core.h
>>>> index 080b783d608d..0f9e68f76b77 100644
>>>> --- a/include/media/videobuf2-core.h
>>>> +++ b/include/media/videobuf2-core.h
>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>> enum vb2_memory memory,
>>>> */
>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>> void *pb);
>>>>
>>>> +/**
>>>> + * vb2_core_delete_buf() -
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @index: id number of the buffer.
>>>> + *
>>>> + * Return: returns zero on success; an error code otherwise.
>>>> + */
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>> +
>>>> /**
>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>> *
>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>> b/include/media/videobuf2-v4l2.h
>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>> --- a/include/media/videobuf2-v4l2.h
>>>> +++ b/include/media/videobuf2-v4l2.h
>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>> v4l2_create_buffers *create);
>>>> */
>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>> struct v4l2_buffer *b);
>>>> +/**
>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>> + *
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @b: buffer structure passed from userspace to
>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>> + *
>>>> + * The return values from this function are intended to be directly
>>>> returned
>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>> + */
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>
>>>> /**
>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>> diff --git a/include/uapi/linux/videodev2.h
>>>> b/include/uapi/linux/videodev2.h
>>>> index aee75eb9e686..31bba1915642 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>> v4l2_dbg_chip_info)
>>>>
>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>> v4l2_query_ext_ctrl)
>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>> +
>>>>
>>>> /* Reminder: when adding new ioctls please add support for them to
>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>> --
>>>> 2.39.2
>>>>
> --
> Hsia-Jun(Randy) Li
>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-30 9:43 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-30 9:43 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
On 6/27/23 16:47, Hsia-Jun Li wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> On 6/27/23 16:43, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>
>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> ---
>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>> +++++++++++++++++++
>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>> include/media/v4l2-ioctl.h | 4 ++
>>>> include/media/videobuf2-core.h | 9 ++++
>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>> include/uapi/linux/videodev2.h | 2 +
>>>> 10 files changed, 128 insertions(+)
>>>> create mode 100644
>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> @@ -17,6 +17,7 @@ Function Reference
>>>> vidioc-dbg-g-chip-info
>>>> vidioc-dbg-g-register
>>>> vidioc-decoder-cmd
>>>> + vidioc-delete-buf
>>>> vidioc-dqevent
>>>> vidioc-dv-timings-cap
>>>> vidioc-encoder-cmd
>>>> diff --git
>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> new file mode 100644
>>>> index 000000000000..0e7ce58f91bc
>>>> --- /dev/null
>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> @@ -0,0 +1,51 @@
>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>> +.. c:namespace:: V4L
>>>> +
>>>> +.. _VIDIOC_DELETE_BUF:
>>>> +
>>>> +************************
>>>> +ioctl VIDIOC_DELETE_BUF
>>>> +************************
>>>> +
>>>> +Name
>>>> +====
>>>> +
>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>> +
>>>> +Synopsis
>>>> +========
>>>> +
>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>> +
>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>> +
>>>> +Arguments
>>>> +=========
>>>> +
>>>> +``fd``
>>>> + File descriptor returned by :c:func:`open()`.
>>>> +
>>>> +``argp``
>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>> +
>>>> +Description
>>>> +===========
>>>> +
>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>> ioctl to
>>>> +delete a buffer from a queue.
>>>> +
>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>> +:ref:`buffer`.
>>>> +
>>>> +Return Value
>>>> +============
>>>> +
>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>> set
>>>> +appropriately. The generic error codes are described at the
>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>> +
>>>> +EBUSY
>>>> + File I/O is in progress.
>>>> +
>>>> +EINVAL
>>>> + The buffer ``index`` doesn't exist in the queue.
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index 899783f67580..aa546c972c3d 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>> unsigned int index, void *pb)
>>>> }
>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>> +{
>>>> + struct vb2_buffer *vb;
>>>> +
>>>> + vb = vb2_get_buffer(q, index);
>>>> + if (!vb) {
>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>> %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>> I know the driver could implement its own
>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>> used by the hardware as a future reference frame.
>>> But I think we need a flag to let the user know which buffer is still
>>> used by the hardware.
>>> Alternative ref case is safe, we only know it's existing when it is
>>> dequeued in current V4L2 buffer mechanism.
>>> While the Golden reference frame, such long term reference frame could
>>> last much longer.
>>
>> It is up to userland stack to know frames life time, it got the
>> information for that.
>
> That is true for the stateless codec driver.
>
> While application for stateful decoder could never do that. It also
> breaks what the document said:
>
> "The backing memory of |CAPTURE| buffers that are used as reference
> frames by the stream may be read by the hardware even after they are
> dequeued. Consequently, the client should avoid writing into this memory
> while the |CAPTURE| queue is streaming. Failure to observe this may
> result in corruption of decoded frames."
>
>>
>>>> + if (vb->planes[0].mem_priv)
>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>> +
>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>> + __vb2_buf_mem_free(vb);
Here is another problem for the existing application, the mmap() from
the mmap offset or exportbuffer fd would not create a reference to
buffer in this step(while the exportbuffer would create one itself).
When you delete a buffer, you may not release it from its virtual memory
space, leaving a corrupted virtual memory space. Also this behavior is
right, because mmap(2) says:
"After the mmap() call has returned, the file descriptor, fd, can be
closed immediately without invalidating the map‐ping."
>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>> + __vb2_buf_dmabuf_put(vb);
>>>> + else
>>>> + __vb2_buf_userptr_put(vb);
>>>> +
>>>> + vb2_queue_remove_buffer(q, vb);
>>>> + kfree(vb);
>>>> +
>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>> + return 0;
>>>> +}
>>>> +
>>>> /*
>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>> * @q: videobuf2 queue
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> index 724135d41f7f..cea666c17b41 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>> media_device *mdev,
>>>> }
>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>> +{
>>>> + return vb2_core_delete_buf(q, b->index);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>> +
>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>> *create)
>>>> {
>>>> unsigned requested_planes = 1;
>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>> index f81279492682..80ace2e1e932 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>> video_device *vdev)
>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>> vidioc_prepare_buf);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>> vidioc_streamon);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>> vidioc_streamoff);
>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>> vidioc_delete_buf);
>>>> }
>>>>
>>>> if (is_vid || is_vbi || is_meta) {
>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index a858acea6547..1c737279d3ef 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>> v4l2_ioctl_ops *ops,
>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>> }
>>>>
>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>> + struct file *file, void *fh, void *arg)
>>>> +{
>>>> + struct v4l2_buffer *b = arg;
>>>> + int ret = check_fmt(file, b->type);
>>>> +
>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>> +}
>>>> +
>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>> struct file *file, void *fh, void
>>>> *arg)
>>>> {
>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>> v4l2_ioctls[] = {
>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>> v4l_print_freq_band, 0),
>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>> };
>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>
>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>> index edb733f21604..2f232ed884c7 100644
>>>> --- a/include/media/v4l2-ioctl.h
>>>> +++ b/include/media/v4l2-ioctl.h
>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>> * @vidioc_overlay: pointer to the function that implements
>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>> struct v4l2_create_buffers *b);
>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>> struct v4l2_buffer *b);
>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>> + struct v4l2_buffer *b);
>>>>
>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>> int i);
>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>> diff --git a/include/media/videobuf2-core.h
>>>> b/include/media/videobuf2-core.h
>>>> index 080b783d608d..0f9e68f76b77 100644
>>>> --- a/include/media/videobuf2-core.h
>>>> +++ b/include/media/videobuf2-core.h
>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>> enum vb2_memory memory,
>>>> */
>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>> void *pb);
>>>>
>>>> +/**
>>>> + * vb2_core_delete_buf() -
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @index: id number of the buffer.
>>>> + *
>>>> + * Return: returns zero on success; an error code otherwise.
>>>> + */
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>> +
>>>> /**
>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>> *
>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>> b/include/media/videobuf2-v4l2.h
>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>> --- a/include/media/videobuf2-v4l2.h
>>>> +++ b/include/media/videobuf2-v4l2.h
>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>> v4l2_create_buffers *create);
>>>> */
>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>> struct v4l2_buffer *b);
>>>> +/**
>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>> + *
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @b: buffer structure passed from userspace to
>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>> + *
>>>> + * The return values from this function are intended to be directly
>>>> returned
>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>> + */
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>
>>>> /**
>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>> diff --git a/include/uapi/linux/videodev2.h
>>>> b/include/uapi/linux/videodev2.h
>>>> index aee75eb9e686..31bba1915642 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>> v4l2_dbg_chip_info)
>>>>
>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>> v4l2_query_ext_ctrl)
>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>> +
>>>>
>>>> /* Reminder: when adding new ioctls please add support for them to
>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>> --
>>>> 2.39.2
>>>>
> --
> Hsia-Jun(Randy) Li
>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-06-30 9:43 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-30 9:43 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
On 6/27/23 16:47, Hsia-Jun Li wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> On 6/27/23 16:43, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>
>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> ---
>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>> +++++++++++++++++++
>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>> include/media/v4l2-ioctl.h | 4 ++
>>>> include/media/videobuf2-core.h | 9 ++++
>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>> include/uapi/linux/videodev2.h | 2 +
>>>> 10 files changed, 128 insertions(+)
>>>> create mode 100644
>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>> @@ -17,6 +17,7 @@ Function Reference
>>>> vidioc-dbg-g-chip-info
>>>> vidioc-dbg-g-register
>>>> vidioc-decoder-cmd
>>>> + vidioc-delete-buf
>>>> vidioc-dqevent
>>>> vidioc-dv-timings-cap
>>>> vidioc-encoder-cmd
>>>> diff --git
>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> new file mode 100644
>>>> index 000000000000..0e7ce58f91bc
>>>> --- /dev/null
>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>> @@ -0,0 +1,51 @@
>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>> +.. c:namespace:: V4L
>>>> +
>>>> +.. _VIDIOC_DELETE_BUF:
>>>> +
>>>> +************************
>>>> +ioctl VIDIOC_DELETE_BUF
>>>> +************************
>>>> +
>>>> +Name
>>>> +====
>>>> +
>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>> +
>>>> +Synopsis
>>>> +========
>>>> +
>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>> +
>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>> +
>>>> +Arguments
>>>> +=========
>>>> +
>>>> +``fd``
>>>> + File descriptor returned by :c:func:`open()`.
>>>> +
>>>> +``argp``
>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>> +
>>>> +Description
>>>> +===========
>>>> +
>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>> ioctl to
>>>> +delete a buffer from a queue.
>>>> +
>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>> +:ref:`buffer`.
>>>> +
>>>> +Return Value
>>>> +============
>>>> +
>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>> set
>>>> +appropriately. The generic error codes are described at the
>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>> +
>>>> +EBUSY
>>>> + File I/O is in progress.
>>>> +
>>>> +EINVAL
>>>> + The buffer ``index`` doesn't exist in the queue.
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> index 899783f67580..aa546c972c3d 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>> unsigned int index, void *pb)
>>>> }
>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>> +{
>>>> + struct vb2_buffer *vb;
>>>> +
>>>> + vb = vb2_get_buffer(q, index);
>>>> + if (!vb) {
>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>> %d\n", index);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>> I know the driver could implement its own
>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>> used by the hardware as a future reference frame.
>>> But I think we need a flag to let the user know which buffer is still
>>> used by the hardware.
>>> Alternative ref case is safe, we only know it's existing when it is
>>> dequeued in current V4L2 buffer mechanism.
>>> While the Golden reference frame, such long term reference frame could
>>> last much longer.
>>
>> It is up to userland stack to know frames life time, it got the
>> information for that.
>
> That is true for the stateless codec driver.
>
> While application for stateful decoder could never do that. It also
> breaks what the document said:
>
> "The backing memory of |CAPTURE| buffers that are used as reference
> frames by the stream may be read by the hardware even after they are
> dequeued. Consequently, the client should avoid writing into this memory
> while the |CAPTURE| queue is streaming. Failure to observe this may
> result in corruption of decoded frames."
>
>>
>>>> + if (vb->planes[0].mem_priv)
>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>> +
>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>> + __vb2_buf_mem_free(vb);
Here is another problem for the existing application, the mmap() from
the mmap offset or exportbuffer fd would not create a reference to
buffer in this step(while the exportbuffer would create one itself).
When you delete a buffer, you may not release it from its virtual memory
space, leaving a corrupted virtual memory space. Also this behavior is
right, because mmap(2) says:
"After the mmap() call has returned, the file descriptor, fd, can be
closed immediately without invalidating the map‐ping."
>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>> + __vb2_buf_dmabuf_put(vb);
>>>> + else
>>>> + __vb2_buf_userptr_put(vb);
>>>> +
>>>> + vb2_queue_remove_buffer(q, vb);
>>>> + kfree(vb);
>>>> +
>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>> + return 0;
>>>> +}
>>>> +
>>>> /*
>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>> * @q: videobuf2 queue
>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> index 724135d41f7f..cea666c17b41 100644
>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>> media_device *mdev,
>>>> }
>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>> +{
>>>> + return vb2_core_delete_buf(q, b->index);
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>> +
>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>> *create)
>>>> {
>>>> unsigned requested_planes = 1;
>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>> index f81279492682..80ace2e1e932 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>> video_device *vdev)
>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>> vidioc_prepare_buf);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>> vidioc_streamon);
>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>> vidioc_streamoff);
>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>> vidioc_delete_buf);
>>>> }
>>>>
>>>> if (is_vid || is_vbi || is_meta) {
>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index a858acea6547..1c737279d3ef 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>> v4l2_ioctl_ops *ops,
>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>> }
>>>>
>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>> + struct file *file, void *fh, void *arg)
>>>> +{
>>>> + struct v4l2_buffer *b = arg;
>>>> + int ret = check_fmt(file, b->type);
>>>> +
>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>> +}
>>>> +
>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>> struct file *file, void *fh, void
>>>> *arg)
>>>> {
>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>> v4l2_ioctls[] = {
>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>> v4l_print_freq_band, 0),
>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>> };
>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>
>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>> index edb733f21604..2f232ed884c7 100644
>>>> --- a/include/media/v4l2-ioctl.h
>>>> +++ b/include/media/v4l2-ioctl.h
>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>> * @vidioc_overlay: pointer to the function that implements
>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>> struct v4l2_create_buffers *b);
>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>> struct v4l2_buffer *b);
>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>> + struct v4l2_buffer *b);
>>>>
>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>> int i);
>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>> diff --git a/include/media/videobuf2-core.h
>>>> b/include/media/videobuf2-core.h
>>>> index 080b783d608d..0f9e68f76b77 100644
>>>> --- a/include/media/videobuf2-core.h
>>>> +++ b/include/media/videobuf2-core.h
>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>> enum vb2_memory memory,
>>>> */
>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>> void *pb);
>>>>
>>>> +/**
>>>> + * vb2_core_delete_buf() -
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @index: id number of the buffer.
>>>> + *
>>>> + * Return: returns zero on success; an error code otherwise.
>>>> + */
>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>> +
>>>> /**
>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>> *
>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>> b/include/media/videobuf2-v4l2.h
>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>> --- a/include/media/videobuf2-v4l2.h
>>>> +++ b/include/media/videobuf2-v4l2.h
>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>> v4l2_create_buffers *create);
>>>> */
>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>> struct v4l2_buffer *b);
>>>> +/**
>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>> + *
>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>> + * @b: buffer structure passed from userspace to
>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>> + *
>>>> + * The return values from this function are intended to be directly
>>>> returned
>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>> + */
>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>
>>>> /**
>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>> diff --git a/include/uapi/linux/videodev2.h
>>>> b/include/uapi/linux/videodev2.h
>>>> index aee75eb9e686..31bba1915642 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>> v4l2_dbg_chip_info)
>>>>
>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>> v4l2_query_ext_ctrl)
>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>> +
>>>>
>>>> /* Reminder: when adding new ioctls please add support for them to
>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>> --
>>>> 2.39.2
>>>>
> --
> Hsia-Jun(Randy) Li
>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-30 9:43 ` Hsia-Jun Li
(?)
@ 2023-07-03 8:12 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 8:12 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>
> On 6/27/23 16:47, Hsia-Jun Li wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>
>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>
>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>> ---
>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>> +++++++++++++++++++
>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>> 10 files changed, 128 insertions(+)
>>>>> create mode 100644
>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>
>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>> vidioc-dbg-g-chip-info
>>>>> vidioc-dbg-g-register
>>>>> vidioc-decoder-cmd
>>>>> + vidioc-delete-buf
>>>>> vidioc-dqevent
>>>>> vidioc-dv-timings-cap
>>>>> vidioc-encoder-cmd
>>>>> diff --git
>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> new file mode 100644
>>>>> index 000000000000..0e7ce58f91bc
>>>>> --- /dev/null
>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> @@ -0,0 +1,51 @@
>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>> +.. c:namespace:: V4L
>>>>> +
>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>> +
>>>>> +************************
>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>> +************************
>>>>> +
>>>>> +Name
>>>>> +====
>>>>> +
>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>> +
>>>>> +Synopsis
>>>>> +========
>>>>> +
>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>> +
>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>> +
>>>>> +Arguments
>>>>> +=========
>>>>> +
>>>>> +``fd``
>>>>> + File descriptor returned by :c:func:`open()`.
>>>>> +
>>>>> +``argp``
>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>> +
>>>>> +Description
>>>>> +===========
>>>>> +
>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>> ioctl to
>>>>> +delete a buffer from a queue.
>>>>> +
>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>> +:ref:`buffer`.
>>>>> +
>>>>> +Return Value
>>>>> +============
>>>>> +
>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>> set
>>>>> +appropriately. The generic error codes are described at the
>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>> +
>>>>> +EBUSY
>>>>> + File I/O is in progress.
>>>>> +
>>>>> +EINVAL
>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> index 899783f67580..aa546c972c3d 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>> unsigned int index, void *pb)
>>>>> }
>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>
>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>> +{
>>>>> + struct vb2_buffer *vb;
>>>>> +
>>>>> + vb = vb2_get_buffer(q, index);
>>>>> + if (!vb) {
>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>> + return -EINVAL;
>>>>> + }
>>>>> +
>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>>> %d\n", index);
>>>>> + return -EINVAL;
>>>>> + }
>>>>> +
>>>> I know the driver could implement its own
>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>> used by the hardware as a future reference frame.
>>>> But I think we need a flag to let the user know which buffer is still
>>>> used by the hardware.
>>>> Alternative ref case is safe, we only know it's existing when it is
>>>> dequeued in current V4L2 buffer mechanism.
>>>> While the Golden reference frame, such long term reference frame could
>>>> last much longer.
>>>
>>> It is up to userland stack to know frames life time, it got the
>>> information for that.
>>
>> That is true for the stateless codec driver.
>>
>> While application for stateful decoder could never do that. It also
>> breaks what the document said:
>>
>> "The backing memory of |CAPTURE| buffers that are used as reference
>> frames by the stream may be read by the hardware even after they are
>> dequeued. Consequently, the client should avoid writing into this memory
>> while the |CAPTURE| queue is streaming. Failure to observe this may
>> result in corruption of decoded frames."
>>
>>>
>>>>> + if (vb->planes[0].mem_priv)
>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>> +
>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>> + __vb2_buf_mem_free(vb);
>
> Here is another problem for the existing application, the mmap() from
> the mmap offset or exportbuffer fd would not create a reference to
> buffer in this step(while the exportbuffer would create one itself).
>
> When you delete a buffer, you may not release it from its virtual
> memory space, leaving a corrupted virtual memory space. Also this
> behavior is right, because mmap(2) says:
>
> "After the mmap() call has returned, the file descriptor, fd, can
> be closed immediately without invalidating the map‐ping."
Existing applications do not call DELETE_BUF ioctl and when call it they will be aware that the buffer is removed.
I have done it in GStreamer:
https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer/-/commit/fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9
Regards,
Benjamin
>
>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>> + else
>>>>> + __vb2_buf_userptr_put(vb);
>>>>> +
>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>> + kfree(vb);
>>>>> +
>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> /*
>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>> * @q: videobuf2 queue
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>> media_device *mdev,
>>>>> }
>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>
>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>> +{
>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>> +}
>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>> +
>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>> *create)
>>>>> {
>>>>> unsigned requested_planes = 1;
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>> index f81279492682..80ace2e1e932 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>> video_device *vdev)
>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>> vidioc_prepare_buf);
>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>> vidioc_streamon);
>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>> vidioc_streamoff);
>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>> vidioc_delete_buf);
>>>>> }
>>>>>
>>>>> if (is_vid || is_vbi || is_meta) {
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> index a858acea6547..1c737279d3ef 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>> v4l2_ioctl_ops *ops,
>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>> }
>>>>>
>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>> + struct file *file, void *fh, void *arg)
>>>>> +{
>>>>> + struct v4l2_buffer *b = arg;
>>>>> + int ret = check_fmt(file, b->type);
>>>>> +
>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>> +}
>>>>> +
>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>> struct file *file, void *fh, void
>>>>> *arg)
>>>>> {
>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>> v4l2_ioctls[] = {
>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>> v4l_print_freq_band, 0),
>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>> };
>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>
>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>> index edb733f21604..2f232ed884c7 100644
>>>>> --- a/include/media/v4l2-ioctl.h
>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>> struct v4l2_create_buffers *b);
>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>> struct v4l2_buffer *b);
>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>> + struct v4l2_buffer *b);
>>>>>
>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>> int i);
>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>> diff --git a/include/media/videobuf2-core.h
>>>>> b/include/media/videobuf2-core.h
>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>> --- a/include/media/videobuf2-core.h
>>>>> +++ b/include/media/videobuf2-core.h
>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>> enum vb2_memory memory,
>>>>> */
>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>> void *pb);
>>>>>
>>>>> +/**
>>>>> + * vb2_core_delete_buf() -
>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>> + * @index: id number of the buffer.
>>>>> + *
>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>> + */
>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>> +
>>>>> /**
>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>> *
>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>> b/include/media/videobuf2-v4l2.h
>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>> v4l2_create_buffers *create);
>>>>> */
>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>> struct v4l2_buffer *b);
>>>>> +/**
>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>> + *
>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>> + * @b: buffer structure passed from userspace to
>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>> + *
>>>>> + * The return values from this function are intended to be directly
>>>>> returned
>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>> + */
>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>
>>>>> /**
>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>> b/include/uapi/linux/videodev2.h
>>>>> index aee75eb9e686..31bba1915642 100644
>>>>> --- a/include/uapi/linux/videodev2.h
>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>> v4l2_dbg_chip_info)
>>>>>
>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>> v4l2_query_ext_ctrl)
>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>> +
>>>>>
>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>> --
>>>>> 2.39.2
>>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 8:12 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 8:12 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>
> On 6/27/23 16:47, Hsia-Jun Li wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>
>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>
>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>> ---
>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>> +++++++++++++++++++
>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>> 10 files changed, 128 insertions(+)
>>>>> create mode 100644
>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>
>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>> vidioc-dbg-g-chip-info
>>>>> vidioc-dbg-g-register
>>>>> vidioc-decoder-cmd
>>>>> + vidioc-delete-buf
>>>>> vidioc-dqevent
>>>>> vidioc-dv-timings-cap
>>>>> vidioc-encoder-cmd
>>>>> diff --git
>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> new file mode 100644
>>>>> index 000000000000..0e7ce58f91bc
>>>>> --- /dev/null
>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> @@ -0,0 +1,51 @@
>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>> +.. c:namespace:: V4L
>>>>> +
>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>> +
>>>>> +************************
>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>> +************************
>>>>> +
>>>>> +Name
>>>>> +====
>>>>> +
>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>> +
>>>>> +Synopsis
>>>>> +========
>>>>> +
>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>> +
>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>> +
>>>>> +Arguments
>>>>> +=========
>>>>> +
>>>>> +``fd``
>>>>> + File descriptor returned by :c:func:`open()`.
>>>>> +
>>>>> +``argp``
>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>> +
>>>>> +Description
>>>>> +===========
>>>>> +
>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>> ioctl to
>>>>> +delete a buffer from a queue.
>>>>> +
>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>> +:ref:`buffer`.
>>>>> +
>>>>> +Return Value
>>>>> +============
>>>>> +
>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>> set
>>>>> +appropriately. The generic error codes are described at the
>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>> +
>>>>> +EBUSY
>>>>> + File I/O is in progress.
>>>>> +
>>>>> +EINVAL
>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> index 899783f67580..aa546c972c3d 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>> unsigned int index, void *pb)
>>>>> }
>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>
>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>> +{
>>>>> + struct vb2_buffer *vb;
>>>>> +
>>>>> + vb = vb2_get_buffer(q, index);
>>>>> + if (!vb) {
>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>> + return -EINVAL;
>>>>> + }
>>>>> +
>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>>> %d\n", index);
>>>>> + return -EINVAL;
>>>>> + }
>>>>> +
>>>> I know the driver could implement its own
>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>> used by the hardware as a future reference frame.
>>>> But I think we need a flag to let the user know which buffer is still
>>>> used by the hardware.
>>>> Alternative ref case is safe, we only know it's existing when it is
>>>> dequeued in current V4L2 buffer mechanism.
>>>> While the Golden reference frame, such long term reference frame could
>>>> last much longer.
>>>
>>> It is up to userland stack to know frames life time, it got the
>>> information for that.
>>
>> That is true for the stateless codec driver.
>>
>> While application for stateful decoder could never do that. It also
>> breaks what the document said:
>>
>> "The backing memory of |CAPTURE| buffers that are used as reference
>> frames by the stream may be read by the hardware even after they are
>> dequeued. Consequently, the client should avoid writing into this memory
>> while the |CAPTURE| queue is streaming. Failure to observe this may
>> result in corruption of decoded frames."
>>
>>>
>>>>> + if (vb->planes[0].mem_priv)
>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>> +
>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>> + __vb2_buf_mem_free(vb);
>
> Here is another problem for the existing application, the mmap() from
> the mmap offset or exportbuffer fd would not create a reference to
> buffer in this step(while the exportbuffer would create one itself).
>
> When you delete a buffer, you may not release it from its virtual
> memory space, leaving a corrupted virtual memory space. Also this
> behavior is right, because mmap(2) says:
>
> "After the mmap() call has returned, the file descriptor, fd, can
> be closed immediately without invalidating the map‐ping."
Existing applications do not call DELETE_BUF ioctl and when call it they will be aware that the buffer is removed.
I have done it in GStreamer:
https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer/-/commit/fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9
Regards,
Benjamin
>
>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>> + else
>>>>> + __vb2_buf_userptr_put(vb);
>>>>> +
>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>> + kfree(vb);
>>>>> +
>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> /*
>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>> * @q: videobuf2 queue
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>> media_device *mdev,
>>>>> }
>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>
>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>> +{
>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>> +}
>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>> +
>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>> *create)
>>>>> {
>>>>> unsigned requested_planes = 1;
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>> index f81279492682..80ace2e1e932 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>> video_device *vdev)
>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>> vidioc_prepare_buf);
>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>> vidioc_streamon);
>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>> vidioc_streamoff);
>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>> vidioc_delete_buf);
>>>>> }
>>>>>
>>>>> if (is_vid || is_vbi || is_meta) {
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> index a858acea6547..1c737279d3ef 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>> v4l2_ioctl_ops *ops,
>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>> }
>>>>>
>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>> + struct file *file, void *fh, void *arg)
>>>>> +{
>>>>> + struct v4l2_buffer *b = arg;
>>>>> + int ret = check_fmt(file, b->type);
>>>>> +
>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>> +}
>>>>> +
>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>> struct file *file, void *fh, void
>>>>> *arg)
>>>>> {
>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>> v4l2_ioctls[] = {
>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>> v4l_print_freq_band, 0),
>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>> };
>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>
>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>> index edb733f21604..2f232ed884c7 100644
>>>>> --- a/include/media/v4l2-ioctl.h
>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>> struct v4l2_create_buffers *b);
>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>> struct v4l2_buffer *b);
>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>> + struct v4l2_buffer *b);
>>>>>
>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>> int i);
>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>> diff --git a/include/media/videobuf2-core.h
>>>>> b/include/media/videobuf2-core.h
>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>> --- a/include/media/videobuf2-core.h
>>>>> +++ b/include/media/videobuf2-core.h
>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>> enum vb2_memory memory,
>>>>> */
>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>> void *pb);
>>>>>
>>>>> +/**
>>>>> + * vb2_core_delete_buf() -
>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>> + * @index: id number of the buffer.
>>>>> + *
>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>> + */
>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>> +
>>>>> /**
>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>> *
>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>> b/include/media/videobuf2-v4l2.h
>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>> v4l2_create_buffers *create);
>>>>> */
>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>> struct v4l2_buffer *b);
>>>>> +/**
>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>> + *
>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>> + * @b: buffer structure passed from userspace to
>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>> + *
>>>>> + * The return values from this function are intended to be directly
>>>>> returned
>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>> + */
>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>
>>>>> /**
>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>> b/include/uapi/linux/videodev2.h
>>>>> index aee75eb9e686..31bba1915642 100644
>>>>> --- a/include/uapi/linux/videodev2.h
>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>> v4l2_dbg_chip_info)
>>>>>
>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>> v4l2_query_ext_ctrl)
>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>> +
>>>>>
>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>> --
>>>>> 2.39.2
>>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 8:12 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 8:12 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>
> On 6/27/23 16:47, Hsia-Jun Li wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>
>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>
>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>> ---
>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>> +++++++++++++++++++
>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>> 10 files changed, 128 insertions(+)
>>>>> create mode 100644
>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>
>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>> vidioc-dbg-g-chip-info
>>>>> vidioc-dbg-g-register
>>>>> vidioc-decoder-cmd
>>>>> + vidioc-delete-buf
>>>>> vidioc-dqevent
>>>>> vidioc-dv-timings-cap
>>>>> vidioc-encoder-cmd
>>>>> diff --git
>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> new file mode 100644
>>>>> index 000000000000..0e7ce58f91bc
>>>>> --- /dev/null
>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>> @@ -0,0 +1,51 @@
>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>> +.. c:namespace:: V4L
>>>>> +
>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>> +
>>>>> +************************
>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>> +************************
>>>>> +
>>>>> +Name
>>>>> +====
>>>>> +
>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>> +
>>>>> +Synopsis
>>>>> +========
>>>>> +
>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>> +
>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>> +
>>>>> +Arguments
>>>>> +=========
>>>>> +
>>>>> +``fd``
>>>>> + File descriptor returned by :c:func:`open()`.
>>>>> +
>>>>> +``argp``
>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>> +
>>>>> +Description
>>>>> +===========
>>>>> +
>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>> ioctl to
>>>>> +delete a buffer from a queue.
>>>>> +
>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>> +:ref:`buffer`.
>>>>> +
>>>>> +Return Value
>>>>> +============
>>>>> +
>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>> set
>>>>> +appropriately. The generic error codes are described at the
>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>> +
>>>>> +EBUSY
>>>>> + File I/O is in progress.
>>>>> +
>>>>> +EINVAL
>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> index 899783f67580..aa546c972c3d 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>> unsigned int index, void *pb)
>>>>> }
>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>
>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>> +{
>>>>> + struct vb2_buffer *vb;
>>>>> +
>>>>> + vb = vb2_get_buffer(q, index);
>>>>> + if (!vb) {
>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>> + return -EINVAL;
>>>>> + }
>>>>> +
>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>>> %d\n", index);
>>>>> + return -EINVAL;
>>>>> + }
>>>>> +
>>>> I know the driver could implement its own
>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>> used by the hardware as a future reference frame.
>>>> But I think we need a flag to let the user know which buffer is still
>>>> used by the hardware.
>>>> Alternative ref case is safe, we only know it's existing when it is
>>>> dequeued in current V4L2 buffer mechanism.
>>>> While the Golden reference frame, such long term reference frame could
>>>> last much longer.
>>>
>>> It is up to userland stack to know frames life time, it got the
>>> information for that.
>>
>> That is true for the stateless codec driver.
>>
>> While application for stateful decoder could never do that. It also
>> breaks what the document said:
>>
>> "The backing memory of |CAPTURE| buffers that are used as reference
>> frames by the stream may be read by the hardware even after they are
>> dequeued. Consequently, the client should avoid writing into this memory
>> while the |CAPTURE| queue is streaming. Failure to observe this may
>> result in corruption of decoded frames."
>>
>>>
>>>>> + if (vb->planes[0].mem_priv)
>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>> +
>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>> + __vb2_buf_mem_free(vb);
>
> Here is another problem for the existing application, the mmap() from
> the mmap offset or exportbuffer fd would not create a reference to
> buffer in this step(while the exportbuffer would create one itself).
>
> When you delete a buffer, you may not release it from its virtual
> memory space, leaving a corrupted virtual memory space. Also this
> behavior is right, because mmap(2) says:
>
> "After the mmap() call has returned, the file descriptor, fd, can
> be closed immediately without invalidating the map‐ping."
Existing applications do not call DELETE_BUF ioctl and when call it they will be aware that the buffer is removed.
I have done it in GStreamer:
https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer/-/commit/fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9
Regards,
Benjamin
>
>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>> + else
>>>>> + __vb2_buf_userptr_put(vb);
>>>>> +
>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>> + kfree(vb);
>>>>> +
>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> /*
>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>> * @q: videobuf2 queue
>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>> media_device *mdev,
>>>>> }
>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>
>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>> +{
>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>> +}
>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>> +
>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>> *create)
>>>>> {
>>>>> unsigned requested_planes = 1;
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>> index f81279492682..80ace2e1e932 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>> video_device *vdev)
>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>> vidioc_prepare_buf);
>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>> vidioc_streamon);
>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>> vidioc_streamoff);
>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>> vidioc_delete_buf);
>>>>> }
>>>>>
>>>>> if (is_vid || is_vbi || is_meta) {
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> index a858acea6547..1c737279d3ef 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>> v4l2_ioctl_ops *ops,
>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>> }
>>>>>
>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>> + struct file *file, void *fh, void *arg)
>>>>> +{
>>>>> + struct v4l2_buffer *b = arg;
>>>>> + int ret = check_fmt(file, b->type);
>>>>> +
>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>> +}
>>>>> +
>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>> struct file *file, void *fh, void
>>>>> *arg)
>>>>> {
>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>> v4l2_ioctls[] = {
>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>> v4l_print_freq_band, 0),
>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>> };
>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>
>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>> index edb733f21604..2f232ed884c7 100644
>>>>> --- a/include/media/v4l2-ioctl.h
>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>> struct v4l2_create_buffers *b);
>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>> struct v4l2_buffer *b);
>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>> + struct v4l2_buffer *b);
>>>>>
>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>> int i);
>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>> diff --git a/include/media/videobuf2-core.h
>>>>> b/include/media/videobuf2-core.h
>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>> --- a/include/media/videobuf2-core.h
>>>>> +++ b/include/media/videobuf2-core.h
>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>> enum vb2_memory memory,
>>>>> */
>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>> void *pb);
>>>>>
>>>>> +/**
>>>>> + * vb2_core_delete_buf() -
>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>> + * @index: id number of the buffer.
>>>>> + *
>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>> + */
>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>> +
>>>>> /**
>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>> *
>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>> b/include/media/videobuf2-v4l2.h
>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>> v4l2_create_buffers *create);
>>>>> */
>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>> struct v4l2_buffer *b);
>>>>> +/**
>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>> + *
>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>> + * @b: buffer structure passed from userspace to
>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>> + *
>>>>> + * The return values from this function are intended to be directly
>>>>> returned
>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>> + */
>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>
>>>>> /**
>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>> b/include/uapi/linux/videodev2.h
>>>>> index aee75eb9e686..31bba1915642 100644
>>>>> --- a/include/uapi/linux/videodev2.h
>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>> v4l2_dbg_chip_info)
>>>>>
>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>> v4l2_query_ext_ctrl)
>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>> +
>>>>>
>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>> --
>>>>> 2.39.2
>>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-07-03 8:12 ` Benjamin Gaignard
(?)
@ 2023-07-03 8:19 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 8:19 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
On 7/3/23 16:12, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>
>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>
>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>> ---
>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>> +++++++++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>> 10 files changed, 128 insertions(+)
>>>>>> create mode 100644
>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>
>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>> vidioc-dbg-g-chip-info
>>>>>> vidioc-dbg-g-register
>>>>>> vidioc-decoder-cmd
>>>>>> + vidioc-delete-buf
>>>>>> vidioc-dqevent
>>>>>> vidioc-dv-timings-cap
>>>>>> vidioc-encoder-cmd
>>>>>> diff --git
>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> new file mode 100644
>>>>>> index 000000000000..0e7ce58f91bc
>>>>>> --- /dev/null
>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> @@ -0,0 +1,51 @@
>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>> +.. c:namespace:: V4L
>>>>>> +
>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>> +
>>>>>> +************************
>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>> +************************
>>>>>> +
>>>>>> +Name
>>>>>> +====
>>>>>> +
>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>> +
>>>>>> +Synopsis
>>>>>> +========
>>>>>> +
>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>> +
>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>> +
>>>>>> +Arguments
>>>>>> +=========
>>>>>> +
>>>>>> +``fd``
>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>> +
>>>>>> +``argp``
>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>> +
>>>>>> +Description
>>>>>> +===========
>>>>>> +
>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>> ioctl to
>>>>>> +delete a buffer from a queue.
>>>>>> +
>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>> +:ref:`buffer`.
>>>>>> +
>>>>>> +Return Value
>>>>>> +============
>>>>>> +
>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>> set
>>>>>> +appropriately. The generic error codes are described at the
>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>> +
>>>>>> +EBUSY
>>>>>> + File I/O is in progress.
>>>>>> +
>>>>>> +EINVAL
>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>> unsigned int index, void *pb)
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>> +{
>>>>>> + struct vb2_buffer *vb;
>>>>>> +
>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>> + if (!vb) {
>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>> index
>>>>>> %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>> I know the driver could implement its own
>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>> used by the hardware as a future reference frame.
>>>>> But I think we need a flag to let the user know which buffer is still
>>>>> used by the hardware.
>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>> dequeued in current V4L2 buffer mechanism.
>>>>> While the Golden reference frame, such long term reference frame
>>>>> could
>>>>> last much longer.
>>>>
>>>> It is up to userland stack to know frames life time, it got the
>>>> information for that.
>>>
>>> That is true for the stateless codec driver.
>>>
>>> While application for stateful decoder could never do that. It also
>>> breaks what the document said:
>>>
>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>> frames by the stream may be read by the hardware even after they are
>>> dequeued. Consequently, the client should avoid writing into this
>>> memory
>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>> result in corruption of decoded frames."
>>>
>>>>
>>>>>> + if (vb->planes[0].mem_priv)
>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>> +
>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>> + __vb2_buf_mem_free(vb);
>>
>> Here is another problem for the existing application, the mmap() from
>> the mmap offset or exportbuffer fd would not create a reference to
>> buffer in this step(while the exportbuffer would create one itself).
>>
>> When you delete a buffer, you may not release it from its virtual
>> memory space, leaving a corrupted virtual memory space. Also this
>> behavior is right, because mmap(2) says:
>>
>> "After the mmap() call has returned, the file descriptor, fd, can
>> be closed immediately without invalidating the map‐ping."
>
> Existing applications do not call DELETE_BUF ioctl and when call it
> they will be aware that the buffer is removed.
> I have done it in GStreamer:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>
I have read that.
There is not a VP8 parser in Gstreamer, while a parser would not work
when deal with the secure video(userspace can't access the data context
at all).
Besides, this adds extra work for the application for a stateful codec
driver. The application need to parser the bitstream and track the dpb.
I don't mind if you could fix the nonfiction mechanism for those
non-display frame and internal reference state.
That could be requirement for codec firmware that its driver could
support this DELETE_BUF ioctl() feature.
>
> Regards,
> Benjamin
>
>>
>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>> + else
>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>> +
>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>> + kfree(vb);
>>>>>> +
>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>> + return 0;
>>>>>> +}
>>>>>> +
>>>>>> /*
>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>> * @q: videobuf2 queue
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>> media_device *mdev,
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>> +{
>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>> +}
>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>> +
>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>> v4l2_create_buffers
>>>>>> *create)
>>>>>> {
>>>>>> unsigned requested_planes = 1;
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>> video_device *vdev)
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>> vidioc_prepare_buf);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>> vidioc_streamon);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>> vidioc_streamoff);
>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>> vidioc_delete_buf);
>>>>>> }
>>>>>>
>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>> v4l2_ioctl_ops *ops,
>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>> }
>>>>>>
>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>> + struct file *file, void *fh, void *arg)
>>>>>> +{
>>>>>> + struct v4l2_buffer *b = arg;
>>>>>> + int ret = check_fmt(file, b->type);
>>>>>> +
>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>> +}
>>>>>> +
>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>> struct file *file, void *fh, void
>>>>>> *arg)
>>>>>> {
>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>> v4l2_ioctls[] = {
>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>> v4l_print_freq_band, 0),
>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>> };
>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>
>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>> struct v4l2_create_buffers *b);
>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>> struct v4l2_buffer *b);
>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>> + struct v4l2_buffer *b);
>>>>>>
>>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>> int i);
>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>> b/include/media/videobuf2-core.h
>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>> --- a/include/media/videobuf2-core.h
>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>> enum vb2_memory memory,
>>>>>> */
>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>> void *pb);
>>>>>>
>>>>>> +/**
>>>>>> + * vb2_core_delete_buf() -
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @index: id number of the buffer.
>>>>>> + *
>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>> + */
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>> +
>>>>>> /**
>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>> *
>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>> v4l2_create_buffers *create);
>>>>>> */
>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>> *mdev,
>>>>>> struct v4l2_buffer *b);
>>>>>> +/**
>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>> + *
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @b: buffer structure passed from userspace to
>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>> + *
>>>>>> + * The return values from this function are intended to be directly
>>>>>> returned
>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>> + */
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>
>>>>>> /**
>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>> b/include/uapi/linux/videodev2.h
>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>> v4l2_dbg_chip_info)
>>>>>>
>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>> v4l2_query_ext_ctrl)
>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>> +
>>>>>>
>>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>> --
>>>>>> 2.39.2
>>>>>>
>>> --
>>> Hsia-Jun(Randy) Li
>>>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 8:19 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 8:19 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
On 7/3/23 16:12, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>
>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>
>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>> ---
>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>> +++++++++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>> 10 files changed, 128 insertions(+)
>>>>>> create mode 100644
>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>
>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>> vidioc-dbg-g-chip-info
>>>>>> vidioc-dbg-g-register
>>>>>> vidioc-decoder-cmd
>>>>>> + vidioc-delete-buf
>>>>>> vidioc-dqevent
>>>>>> vidioc-dv-timings-cap
>>>>>> vidioc-encoder-cmd
>>>>>> diff --git
>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> new file mode 100644
>>>>>> index 000000000000..0e7ce58f91bc
>>>>>> --- /dev/null
>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> @@ -0,0 +1,51 @@
>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>> +.. c:namespace:: V4L
>>>>>> +
>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>> +
>>>>>> +************************
>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>> +************************
>>>>>> +
>>>>>> +Name
>>>>>> +====
>>>>>> +
>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>> +
>>>>>> +Synopsis
>>>>>> +========
>>>>>> +
>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>> +
>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>> +
>>>>>> +Arguments
>>>>>> +=========
>>>>>> +
>>>>>> +``fd``
>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>> +
>>>>>> +``argp``
>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>> +
>>>>>> +Description
>>>>>> +===========
>>>>>> +
>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>> ioctl to
>>>>>> +delete a buffer from a queue.
>>>>>> +
>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>> +:ref:`buffer`.
>>>>>> +
>>>>>> +Return Value
>>>>>> +============
>>>>>> +
>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>> set
>>>>>> +appropriately. The generic error codes are described at the
>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>> +
>>>>>> +EBUSY
>>>>>> + File I/O is in progress.
>>>>>> +
>>>>>> +EINVAL
>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>> unsigned int index, void *pb)
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>> +{
>>>>>> + struct vb2_buffer *vb;
>>>>>> +
>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>> + if (!vb) {
>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>> index
>>>>>> %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>> I know the driver could implement its own
>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>> used by the hardware as a future reference frame.
>>>>> But I think we need a flag to let the user know which buffer is still
>>>>> used by the hardware.
>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>> dequeued in current V4L2 buffer mechanism.
>>>>> While the Golden reference frame, such long term reference frame
>>>>> could
>>>>> last much longer.
>>>>
>>>> It is up to userland stack to know frames life time, it got the
>>>> information for that.
>>>
>>> That is true for the stateless codec driver.
>>>
>>> While application for stateful decoder could never do that. It also
>>> breaks what the document said:
>>>
>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>> frames by the stream may be read by the hardware even after they are
>>> dequeued. Consequently, the client should avoid writing into this
>>> memory
>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>> result in corruption of decoded frames."
>>>
>>>>
>>>>>> + if (vb->planes[0].mem_priv)
>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>> +
>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>> + __vb2_buf_mem_free(vb);
>>
>> Here is another problem for the existing application, the mmap() from
>> the mmap offset or exportbuffer fd would not create a reference to
>> buffer in this step(while the exportbuffer would create one itself).
>>
>> When you delete a buffer, you may not release it from its virtual
>> memory space, leaving a corrupted virtual memory space. Also this
>> behavior is right, because mmap(2) says:
>>
>> "After the mmap() call has returned, the file descriptor, fd, can
>> be closed immediately without invalidating the map‐ping."
>
> Existing applications do not call DELETE_BUF ioctl and when call it
> they will be aware that the buffer is removed.
> I have done it in GStreamer:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>
I have read that.
There is not a VP8 parser in Gstreamer, while a parser would not work
when deal with the secure video(userspace can't access the data context
at all).
Besides, this adds extra work for the application for a stateful codec
driver. The application need to parser the bitstream and track the dpb.
I don't mind if you could fix the nonfiction mechanism for those
non-display frame and internal reference state.
That could be requirement for codec firmware that its driver could
support this DELETE_BUF ioctl() feature.
>
> Regards,
> Benjamin
>
>>
>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>> + else
>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>> +
>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>> + kfree(vb);
>>>>>> +
>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>> + return 0;
>>>>>> +}
>>>>>> +
>>>>>> /*
>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>> * @q: videobuf2 queue
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>> media_device *mdev,
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>> +{
>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>> +}
>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>> +
>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>> v4l2_create_buffers
>>>>>> *create)
>>>>>> {
>>>>>> unsigned requested_planes = 1;
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>> video_device *vdev)
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>> vidioc_prepare_buf);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>> vidioc_streamon);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>> vidioc_streamoff);
>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>> vidioc_delete_buf);
>>>>>> }
>>>>>>
>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>> v4l2_ioctl_ops *ops,
>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>> }
>>>>>>
>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>> + struct file *file, void *fh, void *arg)
>>>>>> +{
>>>>>> + struct v4l2_buffer *b = arg;
>>>>>> + int ret = check_fmt(file, b->type);
>>>>>> +
>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>> +}
>>>>>> +
>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>> struct file *file, void *fh, void
>>>>>> *arg)
>>>>>> {
>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>> v4l2_ioctls[] = {
>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>> v4l_print_freq_band, 0),
>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>> };
>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>
>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>> struct v4l2_create_buffers *b);
>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>> struct v4l2_buffer *b);
>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>> + struct v4l2_buffer *b);
>>>>>>
>>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>> int i);
>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>> b/include/media/videobuf2-core.h
>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>> --- a/include/media/videobuf2-core.h
>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>> enum vb2_memory memory,
>>>>>> */
>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>> void *pb);
>>>>>>
>>>>>> +/**
>>>>>> + * vb2_core_delete_buf() -
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @index: id number of the buffer.
>>>>>> + *
>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>> + */
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>> +
>>>>>> /**
>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>> *
>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>> v4l2_create_buffers *create);
>>>>>> */
>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>> *mdev,
>>>>>> struct v4l2_buffer *b);
>>>>>> +/**
>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>> + *
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @b: buffer structure passed from userspace to
>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>> + *
>>>>>> + * The return values from this function are intended to be directly
>>>>>> returned
>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>> + */
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>
>>>>>> /**
>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>> b/include/uapi/linux/videodev2.h
>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>> v4l2_dbg_chip_info)
>>>>>>
>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>> v4l2_query_ext_ctrl)
>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>> +
>>>>>>
>>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>> --
>>>>>> 2.39.2
>>>>>>
>>> --
>>> Hsia-Jun(Randy) Li
>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 8:19 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 8:19 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
On 7/3/23 16:12, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>
>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>
>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>> ---
>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>> +++++++++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>> 10 files changed, 128 insertions(+)
>>>>>> create mode 100644
>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>
>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>> vidioc-dbg-g-chip-info
>>>>>> vidioc-dbg-g-register
>>>>>> vidioc-decoder-cmd
>>>>>> + vidioc-delete-buf
>>>>>> vidioc-dqevent
>>>>>> vidioc-dv-timings-cap
>>>>>> vidioc-encoder-cmd
>>>>>> diff --git
>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> new file mode 100644
>>>>>> index 000000000000..0e7ce58f91bc
>>>>>> --- /dev/null
>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> @@ -0,0 +1,51 @@
>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>> +.. c:namespace:: V4L
>>>>>> +
>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>> +
>>>>>> +************************
>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>> +************************
>>>>>> +
>>>>>> +Name
>>>>>> +====
>>>>>> +
>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>> +
>>>>>> +Synopsis
>>>>>> +========
>>>>>> +
>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>> +
>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>> +
>>>>>> +Arguments
>>>>>> +=========
>>>>>> +
>>>>>> +``fd``
>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>> +
>>>>>> +``argp``
>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>> +
>>>>>> +Description
>>>>>> +===========
>>>>>> +
>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>> ioctl to
>>>>>> +delete a buffer from a queue.
>>>>>> +
>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>> +:ref:`buffer`.
>>>>>> +
>>>>>> +Return Value
>>>>>> +============
>>>>>> +
>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>> set
>>>>>> +appropriately. The generic error codes are described at the
>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>> +
>>>>>> +EBUSY
>>>>>> + File I/O is in progress.
>>>>>> +
>>>>>> +EINVAL
>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>> unsigned int index, void *pb)
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>> +{
>>>>>> + struct vb2_buffer *vb;
>>>>>> +
>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>> + if (!vb) {
>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>> index
>>>>>> %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>> I know the driver could implement its own
>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>> used by the hardware as a future reference frame.
>>>>> But I think we need a flag to let the user know which buffer is still
>>>>> used by the hardware.
>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>> dequeued in current V4L2 buffer mechanism.
>>>>> While the Golden reference frame, such long term reference frame
>>>>> could
>>>>> last much longer.
>>>>
>>>> It is up to userland stack to know frames life time, it got the
>>>> information for that.
>>>
>>> That is true for the stateless codec driver.
>>>
>>> While application for stateful decoder could never do that. It also
>>> breaks what the document said:
>>>
>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>> frames by the stream may be read by the hardware even after they are
>>> dequeued. Consequently, the client should avoid writing into this
>>> memory
>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>> result in corruption of decoded frames."
>>>
>>>>
>>>>>> + if (vb->planes[0].mem_priv)
>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>> +
>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>> + __vb2_buf_mem_free(vb);
>>
>> Here is another problem for the existing application, the mmap() from
>> the mmap offset or exportbuffer fd would not create a reference to
>> buffer in this step(while the exportbuffer would create one itself).
>>
>> When you delete a buffer, you may not release it from its virtual
>> memory space, leaving a corrupted virtual memory space. Also this
>> behavior is right, because mmap(2) says:
>>
>> "After the mmap() call has returned, the file descriptor, fd, can
>> be closed immediately without invalidating the map‐ping."
>
> Existing applications do not call DELETE_BUF ioctl and when call it
> they will be aware that the buffer is removed.
> I have done it in GStreamer:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>
I have read that.
There is not a VP8 parser in Gstreamer, while a parser would not work
when deal with the secure video(userspace can't access the data context
at all).
Besides, this adds extra work for the application for a stateful codec
driver. The application need to parser the bitstream and track the dpb.
I don't mind if you could fix the nonfiction mechanism for those
non-display frame and internal reference state.
That could be requirement for codec firmware that its driver could
support this DELETE_BUF ioctl() feature.
>
> Regards,
> Benjamin
>
>>
>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>> + else
>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>> +
>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>> + kfree(vb);
>>>>>> +
>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>> + return 0;
>>>>>> +}
>>>>>> +
>>>>>> /*
>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>> * @q: videobuf2 queue
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>> media_device *mdev,
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>> +{
>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>> +}
>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>> +
>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>> v4l2_create_buffers
>>>>>> *create)
>>>>>> {
>>>>>> unsigned requested_planes = 1;
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>> video_device *vdev)
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>> vidioc_prepare_buf);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>> vidioc_streamon);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>> vidioc_streamoff);
>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>> vidioc_delete_buf);
>>>>>> }
>>>>>>
>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>> v4l2_ioctl_ops *ops,
>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>> }
>>>>>>
>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>> + struct file *file, void *fh, void *arg)
>>>>>> +{
>>>>>> + struct v4l2_buffer *b = arg;
>>>>>> + int ret = check_fmt(file, b->type);
>>>>>> +
>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>> +}
>>>>>> +
>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>> struct file *file, void *fh, void
>>>>>> *arg)
>>>>>> {
>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>> v4l2_ioctls[] = {
>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>> v4l_print_freq_band, 0),
>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>> };
>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>
>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>> struct v4l2_create_buffers *b);
>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>> struct v4l2_buffer *b);
>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>> + struct v4l2_buffer *b);
>>>>>>
>>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>> int i);
>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>> b/include/media/videobuf2-core.h
>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>> --- a/include/media/videobuf2-core.h
>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>> enum vb2_memory memory,
>>>>>> */
>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>> void *pb);
>>>>>>
>>>>>> +/**
>>>>>> + * vb2_core_delete_buf() -
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @index: id number of the buffer.
>>>>>> + *
>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>> + */
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>> +
>>>>>> /**
>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>> *
>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>> v4l2_create_buffers *create);
>>>>>> */
>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>> *mdev,
>>>>>> struct v4l2_buffer *b);
>>>>>> +/**
>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>> + *
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @b: buffer structure passed from userspace to
>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>> + *
>>>>>> + * The return values from this function are intended to be directly
>>>>>> returned
>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>> + */
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>
>>>>>> /**
>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>> b/include/uapi/linux/videodev2.h
>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>> v4l2_dbg_chip_info)
>>>>>>
>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>> v4l2_query_ext_ctrl)
>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>> +
>>>>>>
>>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>> --
>>>>>> 2.39.2
>>>>>>
>>> --
>>> Hsia-Jun(Randy) Li
>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-07-03 8:19 ` Hsia-Jun Li
(?)
@ 2023-07-03 8:52 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 8:52 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>
> On 7/3/23 16:12, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>
>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>
>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>> ---
>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>> +++++++++++++++++++
>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>> create mode 100644
>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>
>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>> vidioc-dbg-g-chip-info
>>>>>>> vidioc-dbg-g-register
>>>>>>> vidioc-decoder-cmd
>>>>>>> + vidioc-delete-buf
>>>>>>> vidioc-dqevent
>>>>>>> vidioc-dv-timings-cap
>>>>>>> vidioc-encoder-cmd
>>>>>>> diff --git
>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> new file mode 100644
>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>> --- /dev/null
>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> @@ -0,0 +1,51 @@
>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>> +.. c:namespace:: V4L
>>>>>>> +
>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>> +
>>>>>>> +************************
>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>> +************************
>>>>>>> +
>>>>>>> +Name
>>>>>>> +====
>>>>>>> +
>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>> +
>>>>>>> +Synopsis
>>>>>>> +========
>>>>>>> +
>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>> +
>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>>> +
>>>>>>> +Arguments
>>>>>>> +=========
>>>>>>> +
>>>>>>> +``fd``
>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>> +
>>>>>>> +``argp``
>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>> +
>>>>>>> +Description
>>>>>>> +===========
>>>>>>> +
>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>> ioctl to
>>>>>>> +delete a buffer from a queue.
>>>>>>> +
>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>> +:ref:`buffer`.
>>>>>>> +
>>>>>>> +Return Value
>>>>>>> +============
>>>>>>> +
>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>> variable is
>>>>>>> set
>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>> +
>>>>>>> +EBUSY
>>>>>>> + File I/O is in progress.
>>>>>>> +
>>>>>>> +EINVAL
>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>> *q,
>>>>>>> unsigned int index, void *pb)
>>>>>>> }
>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>
>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>> +{
>>>>>>> + struct vb2_buffer *vb;
>>>>>>> +
>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>> + if (!vb) {
>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>> + return -EINVAL;
>>>>>>> + }
>>>>>>> +
>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>> index
>>>>>>> %d\n", index);
>>>>>>> + return -EINVAL;
>>>>>>> + }
>>>>>>> +
>>>>>> I know the driver could implement its own
>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>> used by the hardware as a future reference frame.
>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>> still
>>>>>> used by the hardware.
>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>> While the Golden reference frame, such long term reference frame
>>>>>> could
>>>>>> last much longer.
>>>>>
>>>>> It is up to userland stack to know frames life time, it got the
>>>>> information for that.
>>>>
>>>> That is true for the stateless codec driver.
>>>>
>>>> While application for stateful decoder could never do that. It also
>>>> breaks what the document said:
>>>>
>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>> frames by the stream may be read by the hardware even after they are
>>>> dequeued. Consequently, the client should avoid writing into this
>>>> memory
>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>> result in corruption of decoded frames."
>>>>
>>>>>
>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>> +
>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>> + __vb2_buf_mem_free(vb);
>>>
>>> Here is another problem for the existing application, the mmap() from
>>> the mmap offset or exportbuffer fd would not create a reference to
>>> buffer in this step(while the exportbuffer would create one itself).
>>>
>>> When you delete a buffer, you may not release it from its virtual
>>> memory space, leaving a corrupted virtual memory space. Also this
>>> behavior is right, because mmap(2) says:
>>>
>>> "After the mmap() call has returned, the file descriptor, fd, can
>>> be closed immediately without invalidating the map‐ping."
>>
>> Existing applications do not call DELETE_BUF ioctl and when call it
>> they will be aware that the buffer is removed.
>> I have done it in GStreamer:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>
>
> I have read that.
>
> There is not a VP8 parser in Gstreamer, while a parser would not work
> when deal with the secure video(userspace can't access the data
> context at all).
>
> Besides, this adds extra work for the application for a stateful codec
> driver. The application need to parser the bitstream and track the dpb.
>
> I don't mind if you could fix the nonfiction mechanism for those
> non-display frame and internal reference state.
>
> That could be requirement for codec firmware that its driver could
> support this DELETE_BUF ioctl() feature.
Sorry I don't see the link with my patches here...
I have work on non-secure VP9 on stateless codec.
DELETE_BUF ioctl is optional and the main goal is to offer a way to applications
to save memory if they know when they could delete buffers without risk.
>
>>
>> Regards,
>> Benjamin
>>
>>>
>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>> + else
>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>> +
>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>> + kfree(vb);
>>>>>>> +
>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>> + return 0;
>>>>>>> +}
>>>>>>> +
>>>>>>> /*
>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>> * @q: videobuf2 queue
>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>> struct
>>>>>>> media_device *mdev,
>>>>>>> }
>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>
>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>> +{
>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>> +}
>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>> +
>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>> v4l2_create_buffers
>>>>>>> *create)
>>>>>>> {
>>>>>>> unsigned requested_planes = 1;
>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>> video_device *vdev)
>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>> vidioc_prepare_buf);
>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>> vidioc_streamon);
>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>> vidioc_streamoff);
>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>> vidioc_delete_buf);
>>>>>>> }
>>>>>>>
>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>> }
>>>>>>>
>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>> +{
>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>> +
>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>> +}
>>>>>>> +
>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>> struct file *file, void *fh, void
>>>>>>> *arg)
>>>>>>> {
>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>> v4l2_ioctls[] = {
>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>> v4l_print_freq_band, 0),
>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>> };
>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>
>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>> struct v4l2_create_buffers *b);
>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>> struct v4l2_buffer *b);
>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>> + struct v4l2_buffer *b);
>>>>>>>
>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>> unsigned
>>>>>>> int i);
>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>> b/include/media/videobuf2-core.h
>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>> enum vb2_memory memory,
>>>>>>> */
>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>>> void *pb);
>>>>>>>
>>>>>>> +/**
>>>>>>> + * vb2_core_delete_buf() -
>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>> + * @index: id number of the buffer.
>>>>>>> + *
>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>> + */
>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>> +
>>>>>>> /**
>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>> *
>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>> struct
>>>>>>> v4l2_create_buffers *create);
>>>>>>> */
>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>> *mdev,
>>>>>>> struct v4l2_buffer *b);
>>>>>>> +/**
>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>> + *
>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>> + *
>>>>>>> + * The return values from this function are intended to be
>>>>>>> directly
>>>>>>> returned
>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>> + */
>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>
>>>>>>> /**
>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>> v4l2_dbg_chip_info)
>>>>>>>
>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>> v4l2_query_ext_ctrl)
>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>>> +
>>>>>>>
>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>> them to
>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>> --
>>>>>>> 2.39.2
>>>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 8:52 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 8:52 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>
> On 7/3/23 16:12, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>
>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>
>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>> ---
>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>> +++++++++++++++++++
>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>> create mode 100644
>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>
>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>> vidioc-dbg-g-chip-info
>>>>>>> vidioc-dbg-g-register
>>>>>>> vidioc-decoder-cmd
>>>>>>> + vidioc-delete-buf
>>>>>>> vidioc-dqevent
>>>>>>> vidioc-dv-timings-cap
>>>>>>> vidioc-encoder-cmd
>>>>>>> diff --git
>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> new file mode 100644
>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>> --- /dev/null
>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> @@ -0,0 +1,51 @@
>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>> +.. c:namespace:: V4L
>>>>>>> +
>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>> +
>>>>>>> +************************
>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>> +************************
>>>>>>> +
>>>>>>> +Name
>>>>>>> +====
>>>>>>> +
>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>> +
>>>>>>> +Synopsis
>>>>>>> +========
>>>>>>> +
>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>> +
>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>>> +
>>>>>>> +Arguments
>>>>>>> +=========
>>>>>>> +
>>>>>>> +``fd``
>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>> +
>>>>>>> +``argp``
>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>> +
>>>>>>> +Description
>>>>>>> +===========
>>>>>>> +
>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>> ioctl to
>>>>>>> +delete a buffer from a queue.
>>>>>>> +
>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>> +:ref:`buffer`.
>>>>>>> +
>>>>>>> +Return Value
>>>>>>> +============
>>>>>>> +
>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>> variable is
>>>>>>> set
>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>> +
>>>>>>> +EBUSY
>>>>>>> + File I/O is in progress.
>>>>>>> +
>>>>>>> +EINVAL
>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>> *q,
>>>>>>> unsigned int index, void *pb)
>>>>>>> }
>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>
>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>> +{
>>>>>>> + struct vb2_buffer *vb;
>>>>>>> +
>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>> + if (!vb) {
>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>> + return -EINVAL;
>>>>>>> + }
>>>>>>> +
>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>> index
>>>>>>> %d\n", index);
>>>>>>> + return -EINVAL;
>>>>>>> + }
>>>>>>> +
>>>>>> I know the driver could implement its own
>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>> used by the hardware as a future reference frame.
>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>> still
>>>>>> used by the hardware.
>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>> While the Golden reference frame, such long term reference frame
>>>>>> could
>>>>>> last much longer.
>>>>>
>>>>> It is up to userland stack to know frames life time, it got the
>>>>> information for that.
>>>>
>>>> That is true for the stateless codec driver.
>>>>
>>>> While application for stateful decoder could never do that. It also
>>>> breaks what the document said:
>>>>
>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>> frames by the stream may be read by the hardware even after they are
>>>> dequeued. Consequently, the client should avoid writing into this
>>>> memory
>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>> result in corruption of decoded frames."
>>>>
>>>>>
>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>> +
>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>> + __vb2_buf_mem_free(vb);
>>>
>>> Here is another problem for the existing application, the mmap() from
>>> the mmap offset or exportbuffer fd would not create a reference to
>>> buffer in this step(while the exportbuffer would create one itself).
>>>
>>> When you delete a buffer, you may not release it from its virtual
>>> memory space, leaving a corrupted virtual memory space. Also this
>>> behavior is right, because mmap(2) says:
>>>
>>> "After the mmap() call has returned, the file descriptor, fd, can
>>> be closed immediately without invalidating the map‐ping."
>>
>> Existing applications do not call DELETE_BUF ioctl and when call it
>> they will be aware that the buffer is removed.
>> I have done it in GStreamer:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>
>
> I have read that.
>
> There is not a VP8 parser in Gstreamer, while a parser would not work
> when deal with the secure video(userspace can't access the data
> context at all).
>
> Besides, this adds extra work for the application for a stateful codec
> driver. The application need to parser the bitstream and track the dpb.
>
> I don't mind if you could fix the nonfiction mechanism for those
> non-display frame and internal reference state.
>
> That could be requirement for codec firmware that its driver could
> support this DELETE_BUF ioctl() feature.
Sorry I don't see the link with my patches here...
I have work on non-secure VP9 on stateless codec.
DELETE_BUF ioctl is optional and the main goal is to offer a way to applications
to save memory if they know when they could delete buffers without risk.
>
>>
>> Regards,
>> Benjamin
>>
>>>
>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>> + else
>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>> +
>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>> + kfree(vb);
>>>>>>> +
>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>> + return 0;
>>>>>>> +}
>>>>>>> +
>>>>>>> /*
>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>> * @q: videobuf2 queue
>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>> struct
>>>>>>> media_device *mdev,
>>>>>>> }
>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>
>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>> +{
>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>> +}
>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>> +
>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>> v4l2_create_buffers
>>>>>>> *create)
>>>>>>> {
>>>>>>> unsigned requested_planes = 1;
>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>> video_device *vdev)
>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>> vidioc_prepare_buf);
>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>> vidioc_streamon);
>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>> vidioc_streamoff);
>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>> vidioc_delete_buf);
>>>>>>> }
>>>>>>>
>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>> }
>>>>>>>
>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>> +{
>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>> +
>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>> +}
>>>>>>> +
>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>> struct file *file, void *fh, void
>>>>>>> *arg)
>>>>>>> {
>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>> v4l2_ioctls[] = {
>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>> v4l_print_freq_band, 0),
>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>> };
>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>
>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>> struct v4l2_create_buffers *b);
>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>> struct v4l2_buffer *b);
>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>> + struct v4l2_buffer *b);
>>>>>>>
>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>> unsigned
>>>>>>> int i);
>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>> b/include/media/videobuf2-core.h
>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>> enum vb2_memory memory,
>>>>>>> */
>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>>> void *pb);
>>>>>>>
>>>>>>> +/**
>>>>>>> + * vb2_core_delete_buf() -
>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>> + * @index: id number of the buffer.
>>>>>>> + *
>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>> + */
>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>> +
>>>>>>> /**
>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>> *
>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>> struct
>>>>>>> v4l2_create_buffers *create);
>>>>>>> */
>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>> *mdev,
>>>>>>> struct v4l2_buffer *b);
>>>>>>> +/**
>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>> + *
>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>> + *
>>>>>>> + * The return values from this function are intended to be
>>>>>>> directly
>>>>>>> returned
>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>> + */
>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>
>>>>>>> /**
>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>> v4l2_dbg_chip_info)
>>>>>>>
>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>> v4l2_query_ext_ctrl)
>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>>> +
>>>>>>>
>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>> them to
>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>> --
>>>>>>> 2.39.2
>>>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 8:52 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 8:52 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>
> On 7/3/23 16:12, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>
>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>
>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>> ---
>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>> +++++++++++++++++++
>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>> create mode 100644
>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>
>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>> vidioc-dbg-g-chip-info
>>>>>>> vidioc-dbg-g-register
>>>>>>> vidioc-decoder-cmd
>>>>>>> + vidioc-delete-buf
>>>>>>> vidioc-dqevent
>>>>>>> vidioc-dv-timings-cap
>>>>>>> vidioc-encoder-cmd
>>>>>>> diff --git
>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> new file mode 100644
>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>> --- /dev/null
>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>> @@ -0,0 +1,51 @@
>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>> +.. c:namespace:: V4L
>>>>>>> +
>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>> +
>>>>>>> +************************
>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>> +************************
>>>>>>> +
>>>>>>> +Name
>>>>>>> +====
>>>>>>> +
>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>> +
>>>>>>> +Synopsis
>>>>>>> +========
>>>>>>> +
>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>> +
>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>>> +
>>>>>>> +Arguments
>>>>>>> +=========
>>>>>>> +
>>>>>>> +``fd``
>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>> +
>>>>>>> +``argp``
>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>> +
>>>>>>> +Description
>>>>>>> +===========
>>>>>>> +
>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>> ioctl to
>>>>>>> +delete a buffer from a queue.
>>>>>>> +
>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>> +:ref:`buffer`.
>>>>>>> +
>>>>>>> +Return Value
>>>>>>> +============
>>>>>>> +
>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>> variable is
>>>>>>> set
>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>> +
>>>>>>> +EBUSY
>>>>>>> + File I/O is in progress.
>>>>>>> +
>>>>>>> +EINVAL
>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>> *q,
>>>>>>> unsigned int index, void *pb)
>>>>>>> }
>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>
>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>> +{
>>>>>>> + struct vb2_buffer *vb;
>>>>>>> +
>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>> + if (!vb) {
>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>> + return -EINVAL;
>>>>>>> + }
>>>>>>> +
>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>> index
>>>>>>> %d\n", index);
>>>>>>> + return -EINVAL;
>>>>>>> + }
>>>>>>> +
>>>>>> I know the driver could implement its own
>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>> used by the hardware as a future reference frame.
>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>> still
>>>>>> used by the hardware.
>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>> While the Golden reference frame, such long term reference frame
>>>>>> could
>>>>>> last much longer.
>>>>>
>>>>> It is up to userland stack to know frames life time, it got the
>>>>> information for that.
>>>>
>>>> That is true for the stateless codec driver.
>>>>
>>>> While application for stateful decoder could never do that. It also
>>>> breaks what the document said:
>>>>
>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>> frames by the stream may be read by the hardware even after they are
>>>> dequeued. Consequently, the client should avoid writing into this
>>>> memory
>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>> result in corruption of decoded frames."
>>>>
>>>>>
>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>> +
>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>> + __vb2_buf_mem_free(vb);
>>>
>>> Here is another problem for the existing application, the mmap() from
>>> the mmap offset or exportbuffer fd would not create a reference to
>>> buffer in this step(while the exportbuffer would create one itself).
>>>
>>> When you delete a buffer, you may not release it from its virtual
>>> memory space, leaving a corrupted virtual memory space. Also this
>>> behavior is right, because mmap(2) says:
>>>
>>> "After the mmap() call has returned, the file descriptor, fd, can
>>> be closed immediately without invalidating the map‐ping."
>>
>> Existing applications do not call DELETE_BUF ioctl and when call it
>> they will be aware that the buffer is removed.
>> I have done it in GStreamer:
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>
>
> I have read that.
>
> There is not a VP8 parser in Gstreamer, while a parser would not work
> when deal with the secure video(userspace can't access the data
> context at all).
>
> Besides, this adds extra work for the application for a stateful codec
> driver. The application need to parser the bitstream and track the dpb.
>
> I don't mind if you could fix the nonfiction mechanism for those
> non-display frame and internal reference state.
>
> That could be requirement for codec firmware that its driver could
> support this DELETE_BUF ioctl() feature.
Sorry I don't see the link with my patches here...
I have work on non-secure VP9 on stateless codec.
DELETE_BUF ioctl is optional and the main goal is to offer a way to applications
to save memory if they know when they could delete buffers without risk.
>
>>
>> Regards,
>> Benjamin
>>
>>>
>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>> + else
>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>> +
>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>> + kfree(vb);
>>>>>>> +
>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>> + return 0;
>>>>>>> +}
>>>>>>> +
>>>>>>> /*
>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>> * @q: videobuf2 queue
>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>> struct
>>>>>>> media_device *mdev,
>>>>>>> }
>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>
>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>> +{
>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>> +}
>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>> +
>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>> v4l2_create_buffers
>>>>>>> *create)
>>>>>>> {
>>>>>>> unsigned requested_planes = 1;
>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>> video_device *vdev)
>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>> vidioc_prepare_buf);
>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>> vidioc_streamon);
>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>> vidioc_streamoff);
>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>> vidioc_delete_buf);
>>>>>>> }
>>>>>>>
>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>> }
>>>>>>>
>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>> +{
>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>> +
>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>> +}
>>>>>>> +
>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>> struct file *file, void *fh, void
>>>>>>> *arg)
>>>>>>> {
>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>> v4l2_ioctls[] = {
>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>> v4l_print_freq_band, 0),
>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>> };
>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>
>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>> struct v4l2_create_buffers *b);
>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>> struct v4l2_buffer *b);
>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>> + struct v4l2_buffer *b);
>>>>>>>
>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>> unsigned
>>>>>>> int i);
>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>> b/include/media/videobuf2-core.h
>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>> enum vb2_memory memory,
>>>>>>> */
>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>>> void *pb);
>>>>>>>
>>>>>>> +/**
>>>>>>> + * vb2_core_delete_buf() -
>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>> + * @index: id number of the buffer.
>>>>>>> + *
>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>> + */
>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>> +
>>>>>>> /**
>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>> *
>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>> struct
>>>>>>> v4l2_create_buffers *create);
>>>>>>> */
>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>> *mdev,
>>>>>>> struct v4l2_buffer *b);
>>>>>>> +/**
>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>> + *
>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>> + *
>>>>>>> + * The return values from this function are intended to be
>>>>>>> directly
>>>>>>> returned
>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>> + */
>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>
>>>>>>> /**
>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>> v4l2_dbg_chip_info)
>>>>>>>
>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>> v4l2_query_ext_ctrl)
>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>>> +
>>>>>>>
>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>> them to
>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>> --
>>>>>>> 2.39.2
>>>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-07-03 8:52 ` Benjamin Gaignard
(?)
@ 2023-07-03 9:20 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 9:20 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
On 7/3/23 16:52, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>
>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>
>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>
>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>
>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>> ---
>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>> +++++++++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>> create mode 100644
>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>
>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>> vidioc-dbg-g-register
>>>>>>>> vidioc-decoder-cmd
>>>>>>>> + vidioc-delete-buf
>>>>>>>> vidioc-dqevent
>>>>>>>> vidioc-dv-timings-cap
>>>>>>>> vidioc-encoder-cmd
>>>>>>>> diff --git
>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> new file mode 100644
>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>> --- /dev/null
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>> +.. c:namespace:: V4L
>>>>>>>> +
>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>> +
>>>>>>>> +************************
>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>> +************************
>>>>>>>> +
>>>>>>>> +Name
>>>>>>>> +====
>>>>>>>> +
>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>> +
>>>>>>>> +Synopsis
>>>>>>>> +========
>>>>>>>> +
>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>> +
>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>> *argp)``
>>>>>>>> +
>>>>>>>> +Arguments
>>>>>>>> +=========
>>>>>>>> +
>>>>>>>> +``fd``
>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>> +
>>>>>>>> +``argp``
>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>> +
>>>>>>>> +Description
>>>>>>>> +===========
>>>>>>>> +
>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>> ioctl to
>>>>>>>> +delete a buffer from a queue.
>>>>>>>> +
>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>> +:ref:`buffer`.
>>>>>>>> +
>>>>>>>> +Return Value
>>>>>>>> +============
>>>>>>>> +
>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>> variable is
>>>>>>>> set
>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>> +
>>>>>>>> +EBUSY
>>>>>>>> + File I/O is in progress.
>>>>>>>> +
>>>>>>>> +EINVAL
>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>> *q,
>>>>>>>> unsigned int index, void *pb)
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>> +{
>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>> +
>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>> + if (!vb) {
>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>> index
>>>>>>>> %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>> I know the driver could implement its own
>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>> used by the hardware as a future reference frame.
>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>> still
>>>>>>> used by the hardware.
>>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>> could
>>>>>>> last much longer.
>>>>>>
>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>> information for that.
>>>>>
>>>>> That is true for the stateless codec driver.
>>>>>
>>>>> While application for stateful decoder could never do that. It also
>>>>> breaks what the document said:
>>>>>
>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>> frames by the stream may be read by the hardware even after they are
>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>> memory
>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>> result in corruption of decoded frames."
>>>>>
>>>>>>
>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>> +
>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>
>>>> Here is another problem for the existing application, the mmap() from
>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>
>>>> When you delete a buffer, you may not release it from its virtual
>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>> behavior is right, because mmap(2) says:
>>>>
>>>> "After the mmap() call has returned, the file descriptor, fd, can
>>>> be closed immediately without invalidating the map‐ping."
>>>
>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>> they will be aware that the buffer is removed.
>>> I have done it in GStreamer:
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>
>>>
>>
>> I have read that.
>>
>> There is not a VP8 parser in Gstreamer, while a parser would not work
>> when deal with the secure video(userspace can't access the data
>> context at all).
>>
>> Besides, this adds extra work for the application for a stateful codec
>> driver. The application need to parser the bitstream and track the dpb.
>>
>> I don't mind if you could fix the nonfiction mechanism for those
>> non-display frame and internal reference state.
>>
>> That could be requirement for codec firmware that its driver could
>> support this DELETE_BUF ioctl() feature.
>
> Sorry I don't see the link with my patches here...
> I have work on non-secure VP9 on stateless codec.
> DELETE_BUF ioctl is optional and the main goal is to offer a way to
> applications
> to save memory if they know when they could delete buffers without risk.
I try to explain why I think this design in not "complete". One problem
resolved, more problems would occur.
For non-secure video, those applications have worked:
- It would break what stateful means here, application need to
acquire(parse) the information that driver should offer.
Or it would break the decoding model.
- Your Gstreamer sample code or this design won't work for AV1.
For all the future possible secure video:
- This feature could never be used from the current design.
>
>>
>>>
>>> Regards,
>>> Benjamin
>>>
>>>>
>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>> + else
>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>> +
>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>> + kfree(vb);
>>>>>>>> +
>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>> + return 0;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> /*
>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>> * @q: videobuf2 queue
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>> struct
>>>>>>>> media_device *mdev,
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>> +{
>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>> +}
>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>> +
>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>> v4l2_create_buffers
>>>>>>>> *create)
>>>>>>>> {
>>>>>>>> unsigned requested_planes = 1;
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>> video_device *vdev)
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>> vidioc_prepare_buf);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>> vidioc_streamon);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>> vidioc_streamoff);
>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>> vidioc_delete_buf);
>>>>>>>> }
>>>>>>>>
>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>> }
>>>>>>>>
>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>>> +{
>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>> +
>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>> struct file *file, void *fh, void
>>>>>>>> *arg)
>>>>>>>> {
>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>> v4l2_ioctls[] = {
>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>> match)),
>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>> };
>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>
>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>> struct v4l2_create_buffers *b);
>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>> unsigned
>>>>>>>> int i);
>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>>> enum vb2_memory memory,
>>>>>>>> */
>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>> index,
>>>>>>>> void *pb);
>>>>>>>>
>>>>>>>> +/**
>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @index: id number of the buffer.
>>>>>>>> + *
>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>> + */
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>>> +
>>>>>>>> /**
>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>> *
>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>> struct
>>>>>>>> v4l2_create_buffers *create);
>>>>>>>> */
>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>> *mdev,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> +/**
>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>> + *
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>> + *
>>>>>>>> + * The return values from this function are intended to be
>>>>>>>> directly
>>>>>>>> returned
>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>> + */
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> /**
>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>
>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>> v4l2_buffer)
>>>>>>>> +
>>>>>>>>
>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>> them to
>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>> --
>>>>>>>> 2.39.2
>>>>>>>>
>>>>> --
>>>>> Hsia-Jun(Randy) Li
>>>>>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 9:20 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 9:20 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
On 7/3/23 16:52, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>
>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>
>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>
>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>
>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>> ---
>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>> +++++++++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>> create mode 100644
>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>
>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>> vidioc-dbg-g-register
>>>>>>>> vidioc-decoder-cmd
>>>>>>>> + vidioc-delete-buf
>>>>>>>> vidioc-dqevent
>>>>>>>> vidioc-dv-timings-cap
>>>>>>>> vidioc-encoder-cmd
>>>>>>>> diff --git
>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> new file mode 100644
>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>> --- /dev/null
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>> +.. c:namespace:: V4L
>>>>>>>> +
>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>> +
>>>>>>>> +************************
>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>> +************************
>>>>>>>> +
>>>>>>>> +Name
>>>>>>>> +====
>>>>>>>> +
>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>> +
>>>>>>>> +Synopsis
>>>>>>>> +========
>>>>>>>> +
>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>> +
>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>> *argp)``
>>>>>>>> +
>>>>>>>> +Arguments
>>>>>>>> +=========
>>>>>>>> +
>>>>>>>> +``fd``
>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>> +
>>>>>>>> +``argp``
>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>> +
>>>>>>>> +Description
>>>>>>>> +===========
>>>>>>>> +
>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>> ioctl to
>>>>>>>> +delete a buffer from a queue.
>>>>>>>> +
>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>> +:ref:`buffer`.
>>>>>>>> +
>>>>>>>> +Return Value
>>>>>>>> +============
>>>>>>>> +
>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>> variable is
>>>>>>>> set
>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>> +
>>>>>>>> +EBUSY
>>>>>>>> + File I/O is in progress.
>>>>>>>> +
>>>>>>>> +EINVAL
>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>> *q,
>>>>>>>> unsigned int index, void *pb)
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>> +{
>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>> +
>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>> + if (!vb) {
>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>> index
>>>>>>>> %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>> I know the driver could implement its own
>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>> used by the hardware as a future reference frame.
>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>> still
>>>>>>> used by the hardware.
>>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>> could
>>>>>>> last much longer.
>>>>>>
>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>> information for that.
>>>>>
>>>>> That is true for the stateless codec driver.
>>>>>
>>>>> While application for stateful decoder could never do that. It also
>>>>> breaks what the document said:
>>>>>
>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>> frames by the stream may be read by the hardware even after they are
>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>> memory
>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>> result in corruption of decoded frames."
>>>>>
>>>>>>
>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>> +
>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>
>>>> Here is another problem for the existing application, the mmap() from
>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>
>>>> When you delete a buffer, you may not release it from its virtual
>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>> behavior is right, because mmap(2) says:
>>>>
>>>> "After the mmap() call has returned, the file descriptor, fd, can
>>>> be closed immediately without invalidating the map‐ping."
>>>
>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>> they will be aware that the buffer is removed.
>>> I have done it in GStreamer:
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>
>>>
>>
>> I have read that.
>>
>> There is not a VP8 parser in Gstreamer, while a parser would not work
>> when deal with the secure video(userspace can't access the data
>> context at all).
>>
>> Besides, this adds extra work for the application for a stateful codec
>> driver. The application need to parser the bitstream and track the dpb.
>>
>> I don't mind if you could fix the nonfiction mechanism for those
>> non-display frame and internal reference state.
>>
>> That could be requirement for codec firmware that its driver could
>> support this DELETE_BUF ioctl() feature.
>
> Sorry I don't see the link with my patches here...
> I have work on non-secure VP9 on stateless codec.
> DELETE_BUF ioctl is optional and the main goal is to offer a way to
> applications
> to save memory if they know when they could delete buffers without risk.
I try to explain why I think this design in not "complete". One problem
resolved, more problems would occur.
For non-secure video, those applications have worked:
- It would break what stateful means here, application need to
acquire(parse) the information that driver should offer.
Or it would break the decoding model.
- Your Gstreamer sample code or this design won't work for AV1.
For all the future possible secure video:
- This feature could never be used from the current design.
>
>>
>>>
>>> Regards,
>>> Benjamin
>>>
>>>>
>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>> + else
>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>> +
>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>> + kfree(vb);
>>>>>>>> +
>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>> + return 0;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> /*
>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>> * @q: videobuf2 queue
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>> struct
>>>>>>>> media_device *mdev,
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>> +{
>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>> +}
>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>> +
>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>> v4l2_create_buffers
>>>>>>>> *create)
>>>>>>>> {
>>>>>>>> unsigned requested_planes = 1;
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>> video_device *vdev)
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>> vidioc_prepare_buf);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>> vidioc_streamon);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>> vidioc_streamoff);
>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>> vidioc_delete_buf);
>>>>>>>> }
>>>>>>>>
>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>> }
>>>>>>>>
>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>>> +{
>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>> +
>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>> struct file *file, void *fh, void
>>>>>>>> *arg)
>>>>>>>> {
>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>> v4l2_ioctls[] = {
>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>> match)),
>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>> };
>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>
>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>> struct v4l2_create_buffers *b);
>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>> unsigned
>>>>>>>> int i);
>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>>> enum vb2_memory memory,
>>>>>>>> */
>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>> index,
>>>>>>>> void *pb);
>>>>>>>>
>>>>>>>> +/**
>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @index: id number of the buffer.
>>>>>>>> + *
>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>> + */
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>>> +
>>>>>>>> /**
>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>> *
>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>> struct
>>>>>>>> v4l2_create_buffers *create);
>>>>>>>> */
>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>> *mdev,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> +/**
>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>> + *
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>> + *
>>>>>>>> + * The return values from this function are intended to be
>>>>>>>> directly
>>>>>>>> returned
>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>> + */
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> /**
>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>
>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>> v4l2_buffer)
>>>>>>>> +
>>>>>>>>
>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>> them to
>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>> --
>>>>>>>> 2.39.2
>>>>>>>>
>>>>> --
>>>>> Hsia-Jun(Randy) Li
>>>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 9:20 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 9:20 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
On 7/3/23 16:52, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>
>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>
>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>
>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>
>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>> ---
>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>> +++++++++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>> create mode 100644
>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>
>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>> vidioc-dbg-g-register
>>>>>>>> vidioc-decoder-cmd
>>>>>>>> + vidioc-delete-buf
>>>>>>>> vidioc-dqevent
>>>>>>>> vidioc-dv-timings-cap
>>>>>>>> vidioc-encoder-cmd
>>>>>>>> diff --git
>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> new file mode 100644
>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>> --- /dev/null
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>> +.. c:namespace:: V4L
>>>>>>>> +
>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>> +
>>>>>>>> +************************
>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>> +************************
>>>>>>>> +
>>>>>>>> +Name
>>>>>>>> +====
>>>>>>>> +
>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>> +
>>>>>>>> +Synopsis
>>>>>>>> +========
>>>>>>>> +
>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>> +
>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>> *argp)``
>>>>>>>> +
>>>>>>>> +Arguments
>>>>>>>> +=========
>>>>>>>> +
>>>>>>>> +``fd``
>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>> +
>>>>>>>> +``argp``
>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>> +
>>>>>>>> +Description
>>>>>>>> +===========
>>>>>>>> +
>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>> ioctl to
>>>>>>>> +delete a buffer from a queue.
>>>>>>>> +
>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>> +:ref:`buffer`.
>>>>>>>> +
>>>>>>>> +Return Value
>>>>>>>> +============
>>>>>>>> +
>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>> variable is
>>>>>>>> set
>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>> +
>>>>>>>> +EBUSY
>>>>>>>> + File I/O is in progress.
>>>>>>>> +
>>>>>>>> +EINVAL
>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>> *q,
>>>>>>>> unsigned int index, void *pb)
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>> +{
>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>> +
>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>> + if (!vb) {
>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>> index
>>>>>>>> %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>> I know the driver could implement its own
>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>> used by the hardware as a future reference frame.
>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>> still
>>>>>>> used by the hardware.
>>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>> could
>>>>>>> last much longer.
>>>>>>
>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>> information for that.
>>>>>
>>>>> That is true for the stateless codec driver.
>>>>>
>>>>> While application for stateful decoder could never do that. It also
>>>>> breaks what the document said:
>>>>>
>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>> frames by the stream may be read by the hardware even after they are
>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>> memory
>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>> result in corruption of decoded frames."
>>>>>
>>>>>>
>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>> +
>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>
>>>> Here is another problem for the existing application, the mmap() from
>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>
>>>> When you delete a buffer, you may not release it from its virtual
>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>> behavior is right, because mmap(2) says:
>>>>
>>>> "After the mmap() call has returned, the file descriptor, fd, can
>>>> be closed immediately without invalidating the map‐ping."
>>>
>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>> they will be aware that the buffer is removed.
>>> I have done it in GStreamer:
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>
>>>
>>
>> I have read that.
>>
>> There is not a VP8 parser in Gstreamer, while a parser would not work
>> when deal with the secure video(userspace can't access the data
>> context at all).
>>
>> Besides, this adds extra work for the application for a stateful codec
>> driver. The application need to parser the bitstream and track the dpb.
>>
>> I don't mind if you could fix the nonfiction mechanism for those
>> non-display frame and internal reference state.
>>
>> That could be requirement for codec firmware that its driver could
>> support this DELETE_BUF ioctl() feature.
>
> Sorry I don't see the link with my patches here...
> I have work on non-secure VP9 on stateless codec.
> DELETE_BUF ioctl is optional and the main goal is to offer a way to
> applications
> to save memory if they know when they could delete buffers without risk.
I try to explain why I think this design in not "complete". One problem
resolved, more problems would occur.
For non-secure video, those applications have worked:
- It would break what stateful means here, application need to
acquire(parse) the information that driver should offer.
Or it would break the decoding model.
- Your Gstreamer sample code or this design won't work for AV1.
For all the future possible secure video:
- This feature could never be used from the current design.
>
>>
>>>
>>> Regards,
>>> Benjamin
>>>
>>>>
>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>> + else
>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>> +
>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>> + kfree(vb);
>>>>>>>> +
>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>> + return 0;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> /*
>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>> * @q: videobuf2 queue
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>> struct
>>>>>>>> media_device *mdev,
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>> +{
>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>> +}
>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>> +
>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>> v4l2_create_buffers
>>>>>>>> *create)
>>>>>>>> {
>>>>>>>> unsigned requested_planes = 1;
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>> video_device *vdev)
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>> vidioc_prepare_buf);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>> vidioc_streamon);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>> vidioc_streamoff);
>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>> vidioc_delete_buf);
>>>>>>>> }
>>>>>>>>
>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>> }
>>>>>>>>
>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>>> +{
>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>> +
>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>> struct file *file, void *fh, void
>>>>>>>> *arg)
>>>>>>>> {
>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>> v4l2_ioctls[] = {
>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>> match)),
>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>> };
>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>
>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>> struct v4l2_create_buffers *b);
>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>> unsigned
>>>>>>>> int i);
>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>>> enum vb2_memory memory,
>>>>>>>> */
>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>> index,
>>>>>>>> void *pb);
>>>>>>>>
>>>>>>>> +/**
>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @index: id number of the buffer.
>>>>>>>> + *
>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>> + */
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>>> +
>>>>>>>> /**
>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>> *
>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>> struct
>>>>>>>> v4l2_create_buffers *create);
>>>>>>>> */
>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>> *mdev,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> +/**
>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>> + *
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>> + *
>>>>>>>> + * The return values from this function are intended to be
>>>>>>>> directly
>>>>>>>> returned
>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>> + */
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> /**
>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>
>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>> v4l2_buffer)
>>>>>>>> +
>>>>>>>>
>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>> them to
>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>> --
>>>>>>>> 2.39.2
>>>>>>>>
>>>>> --
>>>>> Hsia-Jun(Randy) Li
>>>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-07-03 9:20 ` Hsia-Jun Li
(?)
@ 2023-07-03 10:35 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 10:35 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>
> On 7/3/23 16:52, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>
>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>
>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>> ---
>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>> +++++++++++++++++++
>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>> create mode 100644
>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>
>>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>> + vidioc-delete-buf
>>>>>>>>> vidioc-dqevent
>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>> diff --git
>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> new file mode 100644
>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>> --- /dev/null
>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>> +
>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>> +
>>>>>>>>> +************************
>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>> +************************
>>>>>>>>> +
>>>>>>>>> +Name
>>>>>>>>> +====
>>>>>>>>> +
>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>> +
>>>>>>>>> +Synopsis
>>>>>>>>> +========
>>>>>>>>> +
>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>> +
>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>> *argp)``
>>>>>>>>> +
>>>>>>>>> +Arguments
>>>>>>>>> +=========
>>>>>>>>> +
>>>>>>>>> +``fd``
>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>> +
>>>>>>>>> +``argp``
>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>> +
>>>>>>>>> +Description
>>>>>>>>> +===========
>>>>>>>>> +
>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>> ioctl to
>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>> +
>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>> +:ref:`buffer`.
>>>>>>>>> +
>>>>>>>>> +Return Value
>>>>>>>>> +============
>>>>>>>>> +
>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>> variable is
>>>>>>>>> set
>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>> +
>>>>>>>>> +EBUSY
>>>>>>>>> + File I/O is in progress.
>>>>>>>>> +
>>>>>>>>> +EINVAL
>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>>> *q,
>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>> }
>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>
>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>>> +{
>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>> +
>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>> + if (!vb) {
>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>> index);
>>>>>>>>> + return -EINVAL;
>>>>>>>>> + }
>>>>>>>>> +
>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>> index
>>>>>>>>> %d\n", index);
>>>>>>>>> + return -EINVAL;
>>>>>>>>> + }
>>>>>>>>> +
>>>>>>>> I know the driver could implement its own
>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>> still
>>>>>>>> used by the hardware.
>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>> it is
>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>> could
>>>>>>>> last much longer.
>>>>>>>
>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>> information for that.
>>>>>>
>>>>>> That is true for the stateless codec driver.
>>>>>>
>>>>>> While application for stateful decoder could never do that. It also
>>>>>> breaks what the document said:
>>>>>>
>>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>>> frames by the stream may be read by the hardware even after they are
>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>> memory
>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>>> result in corruption of decoded frames."
>>>>>>
>>>>>>>
>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>> +
>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>
>>>>> Here is another problem for the existing application, the mmap() from
>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>>
>>>>> When you delete a buffer, you may not release it from its virtual
>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>> behavior is right, because mmap(2) says:
>>>>>
>>>>> "After the mmap() call has returned, the file descriptor, fd, can
>>>>> be closed immediately without invalidating the map‐ping."
>>>>
>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>> they will be aware that the buffer is removed.
>>>> I have done it in GStreamer:
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>
>>>>
>>>
>>> I have read that.
>>>
>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>> when deal with the secure video(userspace can't access the data
>>> context at all).
>>>
>>> Besides, this adds extra work for the application for a stateful codec
>>> driver. The application need to parser the bitstream and track the dpb.
>>>
>>> I don't mind if you could fix the nonfiction mechanism for those
>>> non-display frame and internal reference state.
>>>
>>> That could be requirement for codec firmware that its driver could
>>> support this DELETE_BUF ioctl() feature.
>>
>> Sorry I don't see the link with my patches here...
>> I have work on non-secure VP9 on stateless codec.
>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>> applications
>> to save memory if they know when they could delete buffers without risk.
>
> I try to explain why I think this design in not "complete". One
> problem resolved, more problems would occur.
>
>
> For non-secure video, those applications have worked:
>
> - It would break what stateful means here, application need to
> acquire(parse) the information that driver should offer.
>
> Or it would break the decoding model.
>
> - Your Gstreamer sample code or this design won't work for AV1.
why ?
>
> For all the future possible secure video:
>
> - This feature could never be used from the current design.
>
>
>>
>>>
>>>>
>>>> Regards,
>>>> Benjamin
>>>>
>>>>>
>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>> + else
>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>> +
>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>> + kfree(vb);
>>>>>>>>> +
>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>> + return 0;
>>>>>>>>> +}
>>>>>>>>> +
>>>>>>>>> /*
>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>> struct
>>>>>>>>> media_device *mdev,
>>>>>>>>> }
>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>
>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>> +{
>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>> +}
>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>> +
>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>> v4l2_create_buffers
>>>>>>>>> *create)
>>>>>>>>> {
>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>> video_device *vdev)
>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>> vidioc_streamon);
>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>> vidioc_streamoff);
>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>> vidioc_delete_buf);
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>>>> +{
>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>> +
>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>> +}
>>>>>>>>> +
>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>> struct file *file, void *fh,
>>>>>>>>> void
>>>>>>>>> *arg)
>>>>>>>>> {
>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>> match)),
>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>> };
>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>
>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>> *b);
>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>
>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>> unsigned
>>>>>>>>> int i);
>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>> *q,
>>>>>>>>> enum vb2_memory memory,
>>>>>>>>> */
>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>> index,
>>>>>>>>> void *pb);
>>>>>>>>>
>>>>>>>>> +/**
>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>> + *
>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>> + */
>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>> index);
>>>>>>>>> +
>>>>>>>>> /**
>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>> *
>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>> struct
>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>> */
>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>> *mdev,
>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>> +/**
>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>> + *
>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>> queue.
>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>> + *
>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>> directly
>>>>>>>>> returned
>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>> + */
>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>>
>>>>>>>>> /**
>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>
>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>> v4l2_buffer)
>>>>>>>>> +
>>>>>>>>>
>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>> them to
>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>> --
>>>>>>>>> 2.39.2
>>>>>>>>>
>>>>>> --
>>>>>> Hsia-Jun(Randy) Li
>>>>>>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 10:35 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 10:35 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>
> On 7/3/23 16:52, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>
>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>
>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>> ---
>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>> +++++++++++++++++++
>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>> create mode 100644
>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>
>>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>> + vidioc-delete-buf
>>>>>>>>> vidioc-dqevent
>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>> diff --git
>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> new file mode 100644
>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>> --- /dev/null
>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>> +
>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>> +
>>>>>>>>> +************************
>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>> +************************
>>>>>>>>> +
>>>>>>>>> +Name
>>>>>>>>> +====
>>>>>>>>> +
>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>> +
>>>>>>>>> +Synopsis
>>>>>>>>> +========
>>>>>>>>> +
>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>> +
>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>> *argp)``
>>>>>>>>> +
>>>>>>>>> +Arguments
>>>>>>>>> +=========
>>>>>>>>> +
>>>>>>>>> +``fd``
>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>> +
>>>>>>>>> +``argp``
>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>> +
>>>>>>>>> +Description
>>>>>>>>> +===========
>>>>>>>>> +
>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>> ioctl to
>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>> +
>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>> +:ref:`buffer`.
>>>>>>>>> +
>>>>>>>>> +Return Value
>>>>>>>>> +============
>>>>>>>>> +
>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>> variable is
>>>>>>>>> set
>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>> +
>>>>>>>>> +EBUSY
>>>>>>>>> + File I/O is in progress.
>>>>>>>>> +
>>>>>>>>> +EINVAL
>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>>> *q,
>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>> }
>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>
>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>>> +{
>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>> +
>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>> + if (!vb) {
>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>> index);
>>>>>>>>> + return -EINVAL;
>>>>>>>>> + }
>>>>>>>>> +
>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>> index
>>>>>>>>> %d\n", index);
>>>>>>>>> + return -EINVAL;
>>>>>>>>> + }
>>>>>>>>> +
>>>>>>>> I know the driver could implement its own
>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>> still
>>>>>>>> used by the hardware.
>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>> it is
>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>> could
>>>>>>>> last much longer.
>>>>>>>
>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>> information for that.
>>>>>>
>>>>>> That is true for the stateless codec driver.
>>>>>>
>>>>>> While application for stateful decoder could never do that. It also
>>>>>> breaks what the document said:
>>>>>>
>>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>>> frames by the stream may be read by the hardware even after they are
>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>> memory
>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>>> result in corruption of decoded frames."
>>>>>>
>>>>>>>
>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>> +
>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>
>>>>> Here is another problem for the existing application, the mmap() from
>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>>
>>>>> When you delete a buffer, you may not release it from its virtual
>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>> behavior is right, because mmap(2) says:
>>>>>
>>>>> "After the mmap() call has returned, the file descriptor, fd, can
>>>>> be closed immediately without invalidating the map‐ping."
>>>>
>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>> they will be aware that the buffer is removed.
>>>> I have done it in GStreamer:
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>
>>>>
>>>
>>> I have read that.
>>>
>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>> when deal with the secure video(userspace can't access the data
>>> context at all).
>>>
>>> Besides, this adds extra work for the application for a stateful codec
>>> driver. The application need to parser the bitstream and track the dpb.
>>>
>>> I don't mind if you could fix the nonfiction mechanism for those
>>> non-display frame and internal reference state.
>>>
>>> That could be requirement for codec firmware that its driver could
>>> support this DELETE_BUF ioctl() feature.
>>
>> Sorry I don't see the link with my patches here...
>> I have work on non-secure VP9 on stateless codec.
>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>> applications
>> to save memory if they know when they could delete buffers without risk.
>
> I try to explain why I think this design in not "complete". One
> problem resolved, more problems would occur.
>
>
> For non-secure video, those applications have worked:
>
> - It would break what stateful means here, application need to
> acquire(parse) the information that driver should offer.
>
> Or it would break the decoding model.
>
> - Your Gstreamer sample code or this design won't work for AV1.
why ?
>
> For all the future possible secure video:
>
> - This feature could never be used from the current design.
>
>
>>
>>>
>>>>
>>>> Regards,
>>>> Benjamin
>>>>
>>>>>
>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>> + else
>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>> +
>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>> + kfree(vb);
>>>>>>>>> +
>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>> + return 0;
>>>>>>>>> +}
>>>>>>>>> +
>>>>>>>>> /*
>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>> struct
>>>>>>>>> media_device *mdev,
>>>>>>>>> }
>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>
>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>> +{
>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>> +}
>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>> +
>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>> v4l2_create_buffers
>>>>>>>>> *create)
>>>>>>>>> {
>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>> video_device *vdev)
>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>> vidioc_streamon);
>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>> vidioc_streamoff);
>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>> vidioc_delete_buf);
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>>>> +{
>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>> +
>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>> +}
>>>>>>>>> +
>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>> struct file *file, void *fh,
>>>>>>>>> void
>>>>>>>>> *arg)
>>>>>>>>> {
>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>> match)),
>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>> };
>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>
>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>> *b);
>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>
>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>> unsigned
>>>>>>>>> int i);
>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>> *q,
>>>>>>>>> enum vb2_memory memory,
>>>>>>>>> */
>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>> index,
>>>>>>>>> void *pb);
>>>>>>>>>
>>>>>>>>> +/**
>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>> + *
>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>> + */
>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>> index);
>>>>>>>>> +
>>>>>>>>> /**
>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>> *
>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>> struct
>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>> */
>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>> *mdev,
>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>> +/**
>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>> + *
>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>> queue.
>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>> + *
>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>> directly
>>>>>>>>> returned
>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>> + */
>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>>
>>>>>>>>> /**
>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>
>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>> v4l2_buffer)
>>>>>>>>> +
>>>>>>>>>
>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>> them to
>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>> --
>>>>>>>>> 2.39.2
>>>>>>>>>
>>>>>> --
>>>>>> Hsia-Jun(Randy) Li
>>>>>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 10:35 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 10:35 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>
> On 7/3/23 16:52, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>
>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>
>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>> ---
>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>> +++++++++++++++++++
>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>> create mode 100644
>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>
>>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>> + vidioc-delete-buf
>>>>>>>>> vidioc-dqevent
>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>> diff --git
>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> new file mode 100644
>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>> --- /dev/null
>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>> +
>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>> +
>>>>>>>>> +************************
>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>> +************************
>>>>>>>>> +
>>>>>>>>> +Name
>>>>>>>>> +====
>>>>>>>>> +
>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>> +
>>>>>>>>> +Synopsis
>>>>>>>>> +========
>>>>>>>>> +
>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>> +
>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>> *argp)``
>>>>>>>>> +
>>>>>>>>> +Arguments
>>>>>>>>> +=========
>>>>>>>>> +
>>>>>>>>> +``fd``
>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>> +
>>>>>>>>> +``argp``
>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>> +
>>>>>>>>> +Description
>>>>>>>>> +===========
>>>>>>>>> +
>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>> ioctl to
>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>> +
>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>> +:ref:`buffer`.
>>>>>>>>> +
>>>>>>>>> +Return Value
>>>>>>>>> +============
>>>>>>>>> +
>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>> variable is
>>>>>>>>> set
>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>> +
>>>>>>>>> +EBUSY
>>>>>>>>> + File I/O is in progress.
>>>>>>>>> +
>>>>>>>>> +EINVAL
>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>>> *q,
>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>> }
>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>
>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>>> +{
>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>> +
>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>> + if (!vb) {
>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>> index);
>>>>>>>>> + return -EINVAL;
>>>>>>>>> + }
>>>>>>>>> +
>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>> index
>>>>>>>>> %d\n", index);
>>>>>>>>> + return -EINVAL;
>>>>>>>>> + }
>>>>>>>>> +
>>>>>>>> I know the driver could implement its own
>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>> still
>>>>>>>> used by the hardware.
>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>> it is
>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>> could
>>>>>>>> last much longer.
>>>>>>>
>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>> information for that.
>>>>>>
>>>>>> That is true for the stateless codec driver.
>>>>>>
>>>>>> While application for stateful decoder could never do that. It also
>>>>>> breaks what the document said:
>>>>>>
>>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>>> frames by the stream may be read by the hardware even after they are
>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>> memory
>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>>> result in corruption of decoded frames."
>>>>>>
>>>>>>>
>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>> +
>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>
>>>>> Here is another problem for the existing application, the mmap() from
>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>>
>>>>> When you delete a buffer, you may not release it from its virtual
>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>> behavior is right, because mmap(2) says:
>>>>>
>>>>> "After the mmap() call has returned, the file descriptor, fd, can
>>>>> be closed immediately without invalidating the map‐ping."
>>>>
>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>> they will be aware that the buffer is removed.
>>>> I have done it in GStreamer:
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>
>>>>
>>>
>>> I have read that.
>>>
>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>> when deal with the secure video(userspace can't access the data
>>> context at all).
>>>
>>> Besides, this adds extra work for the application for a stateful codec
>>> driver. The application need to parser the bitstream and track the dpb.
>>>
>>> I don't mind if you could fix the nonfiction mechanism for those
>>> non-display frame and internal reference state.
>>>
>>> That could be requirement for codec firmware that its driver could
>>> support this DELETE_BUF ioctl() feature.
>>
>> Sorry I don't see the link with my patches here...
>> I have work on non-secure VP9 on stateless codec.
>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>> applications
>> to save memory if they know when they could delete buffers without risk.
>
> I try to explain why I think this design in not "complete". One
> problem resolved, more problems would occur.
>
>
> For non-secure video, those applications have worked:
>
> - It would break what stateful means here, application need to
> acquire(parse) the information that driver should offer.
>
> Or it would break the decoding model.
>
> - Your Gstreamer sample code or this design won't work for AV1.
why ?
>
> For all the future possible secure video:
>
> - This feature could never be used from the current design.
>
>
>>
>>>
>>>>
>>>> Regards,
>>>> Benjamin
>>>>
>>>>>
>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>> + else
>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>> +
>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>> + kfree(vb);
>>>>>>>>> +
>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>> + return 0;
>>>>>>>>> +}
>>>>>>>>> +
>>>>>>>>> /*
>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>> struct
>>>>>>>>> media_device *mdev,
>>>>>>>>> }
>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>
>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>> +{
>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>> +}
>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>> +
>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>> v4l2_create_buffers
>>>>>>>>> *create)
>>>>>>>>> {
>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>> video_device *vdev)
>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>> vidioc_streamon);
>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>> vidioc_streamoff);
>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>> vidioc_delete_buf);
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>>>> +{
>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>> +
>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>> +}
>>>>>>>>> +
>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>> struct file *file, void *fh,
>>>>>>>>> void
>>>>>>>>> *arg)
>>>>>>>>> {
>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>> match)),
>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>> };
>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>
>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>> *b);
>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>
>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>> unsigned
>>>>>>>>> int i);
>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>> *q,
>>>>>>>>> enum vb2_memory memory,
>>>>>>>>> */
>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>> index,
>>>>>>>>> void *pb);
>>>>>>>>>
>>>>>>>>> +/**
>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>> + *
>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>> + */
>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>> index);
>>>>>>>>> +
>>>>>>>>> /**
>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>> *
>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>> struct
>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>> */
>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>> *mdev,
>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>> +/**
>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>> + *
>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>> queue.
>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>> + *
>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>> directly
>>>>>>>>> returned
>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>> + */
>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>>
>>>>>>>>> /**
>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>
>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>> v4l2_buffer)
>>>>>>>>> +
>>>>>>>>>
>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>> them to
>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>> --
>>>>>>>>> 2.39.2
>>>>>>>>>
>>>>>> --
>>>>>> Hsia-Jun(Randy) Li
>>>>>>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-07-03 10:35 ` Benjamin Gaignard
(?)
@ 2023-07-03 11:06 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 11:06 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Sorry for html version, again.
On 7/3/23 18:35, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>
>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>
>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>
>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>> content is
>>>>>>>>>> safe.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>>> ---
>>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>>> create mode 100644
>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>
>>>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>>> + vidioc-delete-buf
>>>>>>>>>> vidioc-dqevent
>>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>>> diff --git
>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> new file mode 100644
>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>> --- /dev/null
>>>>>>>>>> +++
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>> +
>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>> +
>>>>>>>>>> +************************
>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>> +************************
>>>>>>>>>> +
>>>>>>>>>> +Name
>>>>>>>>>> +====
>>>>>>>>>> +
>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>> +
>>>>>>>>>> +Synopsis
>>>>>>>>>> +========
>>>>>>>>>> +
>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>> +
>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>> *argp)``
>>>>>>>>>> +
>>>>>>>>>> +Arguments
>>>>>>>>>> +=========
>>>>>>>>>> +
>>>>>>>>>> +``fd``
>>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>>> +
>>>>>>>>>> +``argp``
>>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>> +
>>>>>>>>>> +Description
>>>>>>>>>> +===========
>>>>>>>>>> +
>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>> ioctl to
>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>> +
>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>> +
>>>>>>>>>> +Return Value
>>>>>>>>>> +============
>>>>>>>>>> +
>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>> variable is
>>>>>>>>>> set
>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>> +
>>>>>>>>>> +EBUSY
>>>>>>>>>> + File I/O is in progress.
>>>>>>>>>> +
>>>>>>>>>> +EINVAL
>>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>>>> *q,
>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>> }
>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>
>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>> index)
>>>>>>>>>> +{
>>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>>> +
>>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>>> + if (!vb) {
>>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>> index);
>>>>>>>>>> + return -EINVAL;
>>>>>>>>>> + }
>>>>>>>>>> +
>>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>>> index
>>>>>>>>>> %d\n", index);
>>>>>>>>>> + return -EINVAL;
>>>>>>>>>> + }
>>>>>>>>>> +
>>>>>>>>> I know the driver could implement its own
>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a
>>>>>>>>> buffer is
>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>> still
>>>>>>>>> used by the hardware.
>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>> it is
>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>>> could
>>>>>>>>> last much longer.
>>>>>>>>
>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>> information for that.
>>>>>>>
>>>>>>> That is true for the stateless codec driver.
>>>>>>>
>>>>>>> While application for stateful decoder could never do that. It also
>>>>>>> breaks what the document said:
>>>>>>>
>>>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>>>> frames by the stream may be read by the hardware even after they
>>>>>>> are
>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>> memory
>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>>>> result in corruption of decoded frames."
>>>>>>>
>>>>>>>>
>>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>> +
>>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>>
>>>>>> Here is another problem for the existing application, the mmap()
>>>>>> from
>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>>>
>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>> behavior is right, because mmap(2) says:
>>>>>>
>>>>>> "After the mmap() call has returned, the file descriptor, fd, can
>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>
>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>> they will be aware that the buffer is removed.
>>>>> I have done it in GStreamer:
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>>
>>>>>
>>>>>
>>>>
>>>> I have read that.
>>>>
>>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>>> when deal with the secure video(userspace can't access the data
>>>> context at all).
>>>>
>>>> Besides, this adds extra work for the application for a stateful codec
>>>> driver. The application need to parser the bitstream and track the
>>>> dpb.
>>>>
>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>> non-display frame and internal reference state.
>>>>
>>>> That could be requirement for codec firmware that its driver could
>>>> support this DELETE_BUF ioctl() feature.
>>>
>>> Sorry I don't see the link with my patches here...
>>> I have work on non-secure VP9 on stateless codec.
>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>> applications
>>> to save memory if they know when they could delete buffers without
>>> risk.
>>
>> I try to explain why I think this design in not "complete". One
>> problem resolved, more problems would occur.
>>
>>
>> For non-secure video, those applications have worked:
>>
>> - It would break what stateful means here, application need to
>> acquire(parse) the information that driver should offer.
>>
>> Or it would break the decoding model.
>>
>> - Your Gstreamer sample code or this design won't work for AV1.
> why ?
A frame in AV1 could be put to display order with a future frame. That
is quite different to those ITU codecs.
Supposing a frame (we call it frame 0) without show_display frame, you
could delete it (just want to do that) once it is decoded.
Actually, unless 7 future frames have came and parser has acknowledged
them, it is not safe to delete that frame.
Or a show_exist that make frame 0 present.
In my personal opinion, this feature could break many uAPI's, I have
said my worrying in the other emails.
>>
>> For all the future possible secure video:
>>
>> - This feature could never be used from the current design.
>>
>>
>>>
>>>>
>>>>>
>>>>> Regards,
>>>>> Benjamin
>>>>>
>>>>>>
>>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>>> + else
>>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>>> +
>>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>>> + kfree(vb);
>>>>>>>>>> +
>>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>> + return 0;
>>>>>>>>>> +}
>>>>>>>>>> +
>>>>>>>>>> /*
>>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>> struct
>>>>>>>>>> media_device *mdev,
>>>>>>>>>> }
>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>
>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>>> +{
>>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>>> +}
>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>> +
>>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>> *create)
>>>>>>>>>> {
>>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>> video_device *vdev)
>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>> vidioc_streamon);
>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh,
>>>>>>>>>> b);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>> + struct file *file, void *fh, void
>>>>>>>>>> *arg)
>>>>>>>>>> +{
>>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>>> +
>>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>>> +}
>>>>>>>>>> +
>>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>> struct file *file, void *fh,
>>>>>>>>>> void
>>>>>>>>>> *arg)
>>>>>>>>>> {
>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>> match)),
>>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>> };
>>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>
>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that
>>>>>>>>>> implements
>>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>>> *b);
>>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>>
>>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>> unsigned
>>>>>>>>>> int i);
>>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>> *q,
>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>> */
>>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>> index,
>>>>>>>>>> void *pb);
>>>>>>>>>>
>>>>>>>>>> +/**
>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>>> + *
>>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>>> + */
>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>> index);
>>>>>>>>>> +
>>>>>>>>>> /**
>>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>> *
>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>> struct
>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>> */
>>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>> *mdev,
>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>> +/**
>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>> + *
>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>>> queue.
>>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>> + *
>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>> directly
>>>>>>>>>> returned
>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>> + */
>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>>>
>>>>>>>>>> /**
>>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>
>>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>>> v4l2_buffer)
>>>>>>>>>> +
>>>>>>>>>>
>>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>>> them to
>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>> --
>>>>>>>>>> 2.39.2
>>>>>>>>>>
>>>>>>> --
>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 11:06 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 11:06 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Sorry for html version, again.
On 7/3/23 18:35, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>
>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>
>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>
>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>> content is
>>>>>>>>>> safe.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>>> ---
>>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>>> create mode 100644
>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>
>>>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>>> + vidioc-delete-buf
>>>>>>>>>> vidioc-dqevent
>>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>>> diff --git
>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> new file mode 100644
>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>> --- /dev/null
>>>>>>>>>> +++
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>> +
>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>> +
>>>>>>>>>> +************************
>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>> +************************
>>>>>>>>>> +
>>>>>>>>>> +Name
>>>>>>>>>> +====
>>>>>>>>>> +
>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>> +
>>>>>>>>>> +Synopsis
>>>>>>>>>> +========
>>>>>>>>>> +
>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>> +
>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>> *argp)``
>>>>>>>>>> +
>>>>>>>>>> +Arguments
>>>>>>>>>> +=========
>>>>>>>>>> +
>>>>>>>>>> +``fd``
>>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>>> +
>>>>>>>>>> +``argp``
>>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>> +
>>>>>>>>>> +Description
>>>>>>>>>> +===========
>>>>>>>>>> +
>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>> ioctl to
>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>> +
>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>> +
>>>>>>>>>> +Return Value
>>>>>>>>>> +============
>>>>>>>>>> +
>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>> variable is
>>>>>>>>>> set
>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>> +
>>>>>>>>>> +EBUSY
>>>>>>>>>> + File I/O is in progress.
>>>>>>>>>> +
>>>>>>>>>> +EINVAL
>>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>>>> *q,
>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>> }
>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>
>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>> index)
>>>>>>>>>> +{
>>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>>> +
>>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>>> + if (!vb) {
>>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>> index);
>>>>>>>>>> + return -EINVAL;
>>>>>>>>>> + }
>>>>>>>>>> +
>>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>>> index
>>>>>>>>>> %d\n", index);
>>>>>>>>>> + return -EINVAL;
>>>>>>>>>> + }
>>>>>>>>>> +
>>>>>>>>> I know the driver could implement its own
>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a
>>>>>>>>> buffer is
>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>> still
>>>>>>>>> used by the hardware.
>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>> it is
>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>>> could
>>>>>>>>> last much longer.
>>>>>>>>
>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>> information for that.
>>>>>>>
>>>>>>> That is true for the stateless codec driver.
>>>>>>>
>>>>>>> While application for stateful decoder could never do that. It also
>>>>>>> breaks what the document said:
>>>>>>>
>>>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>>>> frames by the stream may be read by the hardware even after they
>>>>>>> are
>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>> memory
>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>>>> result in corruption of decoded frames."
>>>>>>>
>>>>>>>>
>>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>> +
>>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>>
>>>>>> Here is another problem for the existing application, the mmap()
>>>>>> from
>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>>>
>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>> behavior is right, because mmap(2) says:
>>>>>>
>>>>>> "After the mmap() call has returned, the file descriptor, fd, can
>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>
>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>> they will be aware that the buffer is removed.
>>>>> I have done it in GStreamer:
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>>
>>>>>
>>>>>
>>>>
>>>> I have read that.
>>>>
>>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>>> when deal with the secure video(userspace can't access the data
>>>> context at all).
>>>>
>>>> Besides, this adds extra work for the application for a stateful codec
>>>> driver. The application need to parser the bitstream and track the
>>>> dpb.
>>>>
>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>> non-display frame and internal reference state.
>>>>
>>>> That could be requirement for codec firmware that its driver could
>>>> support this DELETE_BUF ioctl() feature.
>>>
>>> Sorry I don't see the link with my patches here...
>>> I have work on non-secure VP9 on stateless codec.
>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>> applications
>>> to save memory if they know when they could delete buffers without
>>> risk.
>>
>> I try to explain why I think this design in not "complete". One
>> problem resolved, more problems would occur.
>>
>>
>> For non-secure video, those applications have worked:
>>
>> - It would break what stateful means here, application need to
>> acquire(parse) the information that driver should offer.
>>
>> Or it would break the decoding model.
>>
>> - Your Gstreamer sample code or this design won't work for AV1.
> why ?
A frame in AV1 could be put to display order with a future frame. That
is quite different to those ITU codecs.
Supposing a frame (we call it frame 0) without show_display frame, you
could delete it (just want to do that) once it is decoded.
Actually, unless 7 future frames have came and parser has acknowledged
them, it is not safe to delete that frame.
Or a show_exist that make frame 0 present.
In my personal opinion, this feature could break many uAPI's, I have
said my worrying in the other emails.
>>
>> For all the future possible secure video:
>>
>> - This feature could never be used from the current design.
>>
>>
>>>
>>>>
>>>>>
>>>>> Regards,
>>>>> Benjamin
>>>>>
>>>>>>
>>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>>> + else
>>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>>> +
>>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>>> + kfree(vb);
>>>>>>>>>> +
>>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>> + return 0;
>>>>>>>>>> +}
>>>>>>>>>> +
>>>>>>>>>> /*
>>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>> struct
>>>>>>>>>> media_device *mdev,
>>>>>>>>>> }
>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>
>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>>> +{
>>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>>> +}
>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>> +
>>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>> *create)
>>>>>>>>>> {
>>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>> video_device *vdev)
>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>> vidioc_streamon);
>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh,
>>>>>>>>>> b);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>> + struct file *file, void *fh, void
>>>>>>>>>> *arg)
>>>>>>>>>> +{
>>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>>> +
>>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>>> +}
>>>>>>>>>> +
>>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>> struct file *file, void *fh,
>>>>>>>>>> void
>>>>>>>>>> *arg)
>>>>>>>>>> {
>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>> match)),
>>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>> };
>>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>
>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that
>>>>>>>>>> implements
>>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>>> *b);
>>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>>
>>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>> unsigned
>>>>>>>>>> int i);
>>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>> *q,
>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>> */
>>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>> index,
>>>>>>>>>> void *pb);
>>>>>>>>>>
>>>>>>>>>> +/**
>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>>> + *
>>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>>> + */
>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>> index);
>>>>>>>>>> +
>>>>>>>>>> /**
>>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>> *
>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>> struct
>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>> */
>>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>> *mdev,
>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>> +/**
>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>> + *
>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>>> queue.
>>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>> + *
>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>> directly
>>>>>>>>>> returned
>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>> + */
>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>>>
>>>>>>>>>> /**
>>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>
>>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>>> v4l2_buffer)
>>>>>>>>>> +
>>>>>>>>>>
>>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>>> them to
>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>> --
>>>>>>>>>> 2.39.2
>>>>>>>>>>
>>>>>>> --
>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 11:06 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-03 11:06 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Sorry for html version, again.
On 7/3/23 18:35, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>
>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>
>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>
>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>> content is
>>>>>>>>>> safe.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>>> ---
>>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>>> create mode 100644
>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>
>>>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>>> + vidioc-delete-buf
>>>>>>>>>> vidioc-dqevent
>>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>>> diff --git
>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> new file mode 100644
>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>> --- /dev/null
>>>>>>>>>> +++
>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>> +
>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>> +
>>>>>>>>>> +************************
>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>> +************************
>>>>>>>>>> +
>>>>>>>>>> +Name
>>>>>>>>>> +====
>>>>>>>>>> +
>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>> +
>>>>>>>>>> +Synopsis
>>>>>>>>>> +========
>>>>>>>>>> +
>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>> +
>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>> *argp)``
>>>>>>>>>> +
>>>>>>>>>> +Arguments
>>>>>>>>>> +=========
>>>>>>>>>> +
>>>>>>>>>> +``fd``
>>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>>> +
>>>>>>>>>> +``argp``
>>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>> +
>>>>>>>>>> +Description
>>>>>>>>>> +===========
>>>>>>>>>> +
>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>> ioctl to
>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>> +
>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>> +
>>>>>>>>>> +Return Value
>>>>>>>>>> +============
>>>>>>>>>> +
>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>> variable is
>>>>>>>>>> set
>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>> +
>>>>>>>>>> +EBUSY
>>>>>>>>>> + File I/O is in progress.
>>>>>>>>>> +
>>>>>>>>>> +EINVAL
>>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue
>>>>>>>>>> *q,
>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>> }
>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>
>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>> index)
>>>>>>>>>> +{
>>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>>> +
>>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>>> + if (!vb) {
>>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>> index);
>>>>>>>>>> + return -EINVAL;
>>>>>>>>>> + }
>>>>>>>>>> +
>>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>>> index
>>>>>>>>>> %d\n", index);
>>>>>>>>>> + return -EINVAL;
>>>>>>>>>> + }
>>>>>>>>>> +
>>>>>>>>> I know the driver could implement its own
>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a
>>>>>>>>> buffer is
>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>> still
>>>>>>>>> used by the hardware.
>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>> it is
>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>>> could
>>>>>>>>> last much longer.
>>>>>>>>
>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>> information for that.
>>>>>>>
>>>>>>> That is true for the stateless codec driver.
>>>>>>>
>>>>>>> While application for stateful decoder could never do that. It also
>>>>>>> breaks what the document said:
>>>>>>>
>>>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>>>> frames by the stream may be read by the hardware even after they
>>>>>>> are
>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>> memory
>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>>>> result in corruption of decoded frames."
>>>>>>>
>>>>>>>>
>>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>> +
>>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>>
>>>>>> Here is another problem for the existing application, the mmap()
>>>>>> from
>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>> buffer in this step(while the exportbuffer would create one itself).
>>>>>>
>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>> behavior is right, because mmap(2) says:
>>>>>>
>>>>>> "After the mmap() call has returned, the file descriptor, fd, can
>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>
>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>> they will be aware that the buffer is removed.
>>>>> I have done it in GStreamer:
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>>
>>>>>
>>>>>
>>>>
>>>> I have read that.
>>>>
>>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>>> when deal with the secure video(userspace can't access the data
>>>> context at all).
>>>>
>>>> Besides, this adds extra work for the application for a stateful codec
>>>> driver. The application need to parser the bitstream and track the
>>>> dpb.
>>>>
>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>> non-display frame and internal reference state.
>>>>
>>>> That could be requirement for codec firmware that its driver could
>>>> support this DELETE_BUF ioctl() feature.
>>>
>>> Sorry I don't see the link with my patches here...
>>> I have work on non-secure VP9 on stateless codec.
>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>> applications
>>> to save memory if they know when they could delete buffers without
>>> risk.
>>
>> I try to explain why I think this design in not "complete". One
>> problem resolved, more problems would occur.
>>
>>
>> For non-secure video, those applications have worked:
>>
>> - It would break what stateful means here, application need to
>> acquire(parse) the information that driver should offer.
>>
>> Or it would break the decoding model.
>>
>> - Your Gstreamer sample code or this design won't work for AV1.
> why ?
A frame in AV1 could be put to display order with a future frame. That
is quite different to those ITU codecs.
Supposing a frame (we call it frame 0) without show_display frame, you
could delete it (just want to do that) once it is decoded.
Actually, unless 7 future frames have came and parser has acknowledged
them, it is not safe to delete that frame.
Or a show_exist that make frame 0 present.
In my personal opinion, this feature could break many uAPI's, I have
said my worrying in the other emails.
>>
>> For all the future possible secure video:
>>
>> - This feature could never be used from the current design.
>>
>>
>>>
>>>>
>>>>>
>>>>> Regards,
>>>>> Benjamin
>>>>>
>>>>>>
>>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>>> + else
>>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>>> +
>>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>>> + kfree(vb);
>>>>>>>>>> +
>>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>> + return 0;
>>>>>>>>>> +}
>>>>>>>>>> +
>>>>>>>>>> /*
>>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>> struct
>>>>>>>>>> media_device *mdev,
>>>>>>>>>> }
>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>
>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>>> +{
>>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>>> +}
>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>> +
>>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>> *create)
>>>>>>>>>> {
>>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>> video_device *vdev)
>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>> vidioc_streamon);
>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh,
>>>>>>>>>> b);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>> + struct file *file, void *fh, void
>>>>>>>>>> *arg)
>>>>>>>>>> +{
>>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>>> +
>>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>>> +}
>>>>>>>>>> +
>>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>> struct file *file, void *fh,
>>>>>>>>>> void
>>>>>>>>>> *arg)
>>>>>>>>>> {
>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>> match)),
>>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>> };
>>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>
>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that
>>>>>>>>>> implements
>>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>>> *b);
>>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>>
>>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>> unsigned
>>>>>>>>>> int i);
>>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>> *q,
>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>> */
>>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>> index,
>>>>>>>>>> void *pb);
>>>>>>>>>>
>>>>>>>>>> +/**
>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>>> + *
>>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>>> + */
>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>> index);
>>>>>>>>>> +
>>>>>>>>>> /**
>>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>> *
>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>> struct
>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>> */
>>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>> *mdev,
>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>> +/**
>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>> + *
>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>>> queue.
>>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>> + *
>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>> directly
>>>>>>>>>> returned
>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>> + */
>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>>>
>>>>>>>>>> /**
>>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>
>>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>>> v4l2_buffer)
>>>>>>>>>> +
>>>>>>>>>>
>>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>>> them to
>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>> --
>>>>>>>>>> 2.39.2
>>>>>>>>>>
>>>>>>> --
>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
[parent not found: <8ca2f66e-8ff9-e885-274f-51417b581b78@synaptics.com>]
* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
[not found] ` <8ca2f66e-8ff9-e885-274f-51417b581b78@synaptics.com>
2023-07-03 11:17 ` Benjamin Gaignard
@ 2023-07-03 11:17 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 11:17 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 03/07/2023 à 13:02, Hsia-Jun Li a écrit :
>
>
> On 7/3/23 18:35, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>>
>>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>>
>>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>>
>>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>>> CAUTION: Email originated externally, do not click links or
>>>>>>>>>>> open
>>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>>> content is
>>>>>>>>>>> safe.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>>>> ---
>>>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>>>> create mode 100644
>>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>
>>>>>>>>>>> diff --git
>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>>>> + vidioc-delete-buf
>>>>>>>>>>> vidioc-dqevent
>>>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>>>> diff --git
>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> new file mode 100644
>>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>>> --- /dev/null
>>>>>>>>>>> +++
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>>> +
>>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>>> +
>>>>>>>>>>> +************************
>>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>>> +************************
>>>>>>>>>>> +
>>>>>>>>>>> +Name
>>>>>>>>>>> +====
>>>>>>>>>>> +
>>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>>> +
>>>>>>>>>>> +Synopsis
>>>>>>>>>>> +========
>>>>>>>>>>> +
>>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>>> +
>>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>>> *argp)``
>>>>>>>>>>> +
>>>>>>>>>>> +Arguments
>>>>>>>>>>> +=========
>>>>>>>>>>> +
>>>>>>>>>>> +``fd``
>>>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>>>> +
>>>>>>>>>>> +``argp``
>>>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>>> +
>>>>>>>>>>> +Description
>>>>>>>>>>> +===========
>>>>>>>>>>> +
>>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>>> ioctl to
>>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>>> +
>>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>>> +
>>>>>>>>>>> +Return Value
>>>>>>>>>>> +============
>>>>>>>>>>> +
>>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>>> variable is
>>>>>>>>>>> set
>>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>>> +
>>>>>>>>>>> +EBUSY
>>>>>>>>>>> + File I/O is in progress.
>>>>>>>>>>> +
>>>>>>>>>>> +EINVAL
>>>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct
>>>>>>>>>>> vb2_queue
>>>>>>>>>>> *q,
>>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>>> }
>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>>
>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>> index)
>>>>>>>>>>> +{
>>>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>>>> +
>>>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>>>> + if (!vb) {
>>>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>>> index);
>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>>>> index
>>>>>>>>>>> %d\n", index);
>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>> I know the driver could implement its own
>>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a
>>>>>>>>>> buffer is
>>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>>> still
>>>>>>>>>> used by the hardware.
>>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>>> it is
>>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>>>> could
>>>>>>>>>> last much longer.
>>>>>>>>>
>>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>>> information for that.
>>>>>>>>
>>>>>>>> That is true for the stateless codec driver.
>>>>>>>>
>>>>>>>> While application for stateful decoder could never do that. It
>>>>>>>> also
>>>>>>>> breaks what the document said:
>>>>>>>>
>>>>>>>> "The backing memory of |CAPTURE| buffers that are used as
>>>>>>>> reference
>>>>>>>> frames by the stream may be read by the hardware even after
>>>>>>>> they are
>>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>>> memory
>>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this
>>>>>>>> may
>>>>>>>> result in corruption of decoded frames."
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>>> +
>>>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>>>
>>>>>>> Here is another problem for the existing application, the mmap()
>>>>>>> from
>>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>>> buffer in this step(while the exportbuffer would create one
>>>>>>> itself).
>>>>>>>
>>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>>> behavior is right, because mmap(2) says:
>>>>>>>
>>>>>>> "After the mmap() call has returned, the file descriptor, fd,
>>>>>>> can
>>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>>
>>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>>> they will be aware that the buffer is removed.
>>>>>> I have done it in GStreamer:
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> I have read that.
>>>>>
>>>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>>>> when deal with the secure video(userspace can't access the data
>>>>> context at all).
>>>>>
>>>>> Besides, this adds extra work for the application for a stateful
>>>>> codec
>>>>> driver. The application need to parser the bitstream and track the
>>>>> dpb.
>>>>>
>>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>>> non-display frame and internal reference state.
>>>>>
>>>>> That could be requirement for codec firmware that its driver could
>>>>> support this DELETE_BUF ioctl() feature.
>>>>
>>>> Sorry I don't see the link with my patches here...
>>>> I have work on non-secure VP9 on stateless codec.
>>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>>> applications
>>>> to save memory if they know when they could delete buffers without
>>>> risk.
>>>
>>> I try to explain why I think this design in not "complete". One
>>> problem resolved, more problems would occur.
>>>
>>>
>>> For non-secure video, those applications have worked:
>>>
>>> - It would break what stateful means here, application need to
>>> acquire(parse) the information that driver should offer.
>>>
>>> Or it would break the decoding model.
>>>
>>> - Your Gstreamer sample code or this design won't work for AV1.
>> why ?
>
> A frame in AV1 could be put to display order with a future frame. That
> is quite different to those ITU codecs.
>
> Supposing a frame (we call it frame 0) without show_display frame, you
> could delete it (just want to do that) once it is decoded.
>
> Actually, unless 7 future frames have came and parser has acknowledged
> them, it is not safe to delete that frame.
>
> Or a show_exist that make frame 0 present.
>
> In my personal opinion, this feature could break many uAPI's, I have
> said my worrying in the other emails.
>
Of course if the application delete a useful buffer it is a problem but an application issue.
I have test this code using GStreamer with VP9, VP8 and HEVC on Verisilicon driver without issues.
I will be happy if someone could test it on another driver and/or other userlands (like ffmpeg).
>>>
>>> For all the future possible secure video:
>>>
>>> - This feature could never be used from the current design.
>>>
>>>
>>>>
>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Benjamin
>>>>>>
>>>>>>>
>>>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>>>> + else
>>>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>>>> +
>>>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>>>> + kfree(vb);
>>>>>>>>>>> +
>>>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>>> + return 0;
>>>>>>>>>>> +}
>>>>>>>>>>> +
>>>>>>>>>>> /*
>>>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>>> struct
>>>>>>>>>>> media_device *mdev,
>>>>>>>>>>> }
>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>>
>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>>>> +{
>>>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>>>> +}
>>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>>> +
>>>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>>> *create)
>>>>>>>>>>> {
>>>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>>> video_device *vdev)
>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>>> vidioc_streamon);
>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file,
>>>>>>>>>>> fh, b);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>> + struct file *file, void *fh, void
>>>>>>>>>>> *arg)
>>>>>>>>>>> +{
>>>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>>>> +
>>>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>>>> +}
>>>>>>>>>>> +
>>>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>> struct file *file, void *fh,
>>>>>>>>>>> void
>>>>>>>>>>> *arg)
>>>>>>>>>>> {
>>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS,
>>>>>>>>>>> v4l_enum_freq_bands,
>>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO,
>>>>>>>>>>> v4l_dbg_g_chip_info,
>>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>>> match)),
>>>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>>> };
>>>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that
>>>>>>>>>>> implements
>>>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>>>> *b);
>>>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>>>
>>>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>>> unsigned
>>>>>>>>>>> int i);
>>>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>>> *q,
>>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>>> */
>>>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>> index,
>>>>>>>>>>> void *pb);
>>>>>>>>>>>
>>>>>>>>>>> +/**
>>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>>>> + *
>>>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>>>> + */
>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>> index);
>>>>>>>>>>> +
>>>>>>>>>>> /**
>>>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>>> *
>>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>>> struct
>>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>>> */
>>>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>>> *mdev,
>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>> +/**
>>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>>> + *
>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>>>> queue.
>>>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>>> + *
>>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>>> directly
>>>>>>>>>>> returned
>>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>>> + */
>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer
>>>>>>>>>>> *b);
>>>>>>>>>>>
>>>>>>>>>>> /**
>>>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>>
>>>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>>>> v4l2_buffer)
>>>>>>>>>>> +
>>>>>>>>>>>
>>>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>>>> them to
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>>> --
>>>>>>>>>>> 2.39.2
>>>>>>>>>>>
>>>>>>>> --
>>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>>
> --
> Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 11:17 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 11:17 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 03/07/2023 à 13:02, Hsia-Jun Li a écrit :
>
>
> On 7/3/23 18:35, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>>
>>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>>
>>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>>
>>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>>> CAUTION: Email originated externally, do not click links or
>>>>>>>>>>> open
>>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>>> content is
>>>>>>>>>>> safe.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>>>> ---
>>>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>>>> create mode 100644
>>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>
>>>>>>>>>>> diff --git
>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>>>> + vidioc-delete-buf
>>>>>>>>>>> vidioc-dqevent
>>>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>>>> diff --git
>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> new file mode 100644
>>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>>> --- /dev/null
>>>>>>>>>>> +++
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>>> +
>>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>>> +
>>>>>>>>>>> +************************
>>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>>> +************************
>>>>>>>>>>> +
>>>>>>>>>>> +Name
>>>>>>>>>>> +====
>>>>>>>>>>> +
>>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>>> +
>>>>>>>>>>> +Synopsis
>>>>>>>>>>> +========
>>>>>>>>>>> +
>>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>>> +
>>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>>> *argp)``
>>>>>>>>>>> +
>>>>>>>>>>> +Arguments
>>>>>>>>>>> +=========
>>>>>>>>>>> +
>>>>>>>>>>> +``fd``
>>>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>>>> +
>>>>>>>>>>> +``argp``
>>>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>>> +
>>>>>>>>>>> +Description
>>>>>>>>>>> +===========
>>>>>>>>>>> +
>>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>>> ioctl to
>>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>>> +
>>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>>> +
>>>>>>>>>>> +Return Value
>>>>>>>>>>> +============
>>>>>>>>>>> +
>>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>>> variable is
>>>>>>>>>>> set
>>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>>> +
>>>>>>>>>>> +EBUSY
>>>>>>>>>>> + File I/O is in progress.
>>>>>>>>>>> +
>>>>>>>>>>> +EINVAL
>>>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct
>>>>>>>>>>> vb2_queue
>>>>>>>>>>> *q,
>>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>>> }
>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>>
>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>> index)
>>>>>>>>>>> +{
>>>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>>>> +
>>>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>>>> + if (!vb) {
>>>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>>> index);
>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>>>> index
>>>>>>>>>>> %d\n", index);
>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>> I know the driver could implement its own
>>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a
>>>>>>>>>> buffer is
>>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>>> still
>>>>>>>>>> used by the hardware.
>>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>>> it is
>>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>>>> could
>>>>>>>>>> last much longer.
>>>>>>>>>
>>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>>> information for that.
>>>>>>>>
>>>>>>>> That is true for the stateless codec driver.
>>>>>>>>
>>>>>>>> While application for stateful decoder could never do that. It
>>>>>>>> also
>>>>>>>> breaks what the document said:
>>>>>>>>
>>>>>>>> "The backing memory of |CAPTURE| buffers that are used as
>>>>>>>> reference
>>>>>>>> frames by the stream may be read by the hardware even after
>>>>>>>> they are
>>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>>> memory
>>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this
>>>>>>>> may
>>>>>>>> result in corruption of decoded frames."
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>>> +
>>>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>>>
>>>>>>> Here is another problem for the existing application, the mmap()
>>>>>>> from
>>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>>> buffer in this step(while the exportbuffer would create one
>>>>>>> itself).
>>>>>>>
>>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>>> behavior is right, because mmap(2) says:
>>>>>>>
>>>>>>> "After the mmap() call has returned, the file descriptor, fd,
>>>>>>> can
>>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>>
>>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>>> they will be aware that the buffer is removed.
>>>>>> I have done it in GStreamer:
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> I have read that.
>>>>>
>>>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>>>> when deal with the secure video(userspace can't access the data
>>>>> context at all).
>>>>>
>>>>> Besides, this adds extra work for the application for a stateful
>>>>> codec
>>>>> driver. The application need to parser the bitstream and track the
>>>>> dpb.
>>>>>
>>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>>> non-display frame and internal reference state.
>>>>>
>>>>> That could be requirement for codec firmware that its driver could
>>>>> support this DELETE_BUF ioctl() feature.
>>>>
>>>> Sorry I don't see the link with my patches here...
>>>> I have work on non-secure VP9 on stateless codec.
>>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>>> applications
>>>> to save memory if they know when they could delete buffers without
>>>> risk.
>>>
>>> I try to explain why I think this design in not "complete". One
>>> problem resolved, more problems would occur.
>>>
>>>
>>> For non-secure video, those applications have worked:
>>>
>>> - It would break what stateful means here, application need to
>>> acquire(parse) the information that driver should offer.
>>>
>>> Or it would break the decoding model.
>>>
>>> - Your Gstreamer sample code or this design won't work for AV1.
>> why ?
>
> A frame in AV1 could be put to display order with a future frame. That
> is quite different to those ITU codecs.
>
> Supposing a frame (we call it frame 0) without show_display frame, you
> could delete it (just want to do that) once it is decoded.
>
> Actually, unless 7 future frames have came and parser has acknowledged
> them, it is not safe to delete that frame.
>
> Or a show_exist that make frame 0 present.
>
> In my personal opinion, this feature could break many uAPI's, I have
> said my worrying in the other emails.
>
Of course if the application delete a useful buffer it is a problem but an application issue.
I have test this code using GStreamer with VP9, VP8 and HEVC on Verisilicon driver without issues.
I will be happy if someone could test it on another driver and/or other userlands (like ffmpeg).
>>>
>>> For all the future possible secure video:
>>>
>>> - This feature could never be used from the current design.
>>>
>>>
>>>>
>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Benjamin
>>>>>>
>>>>>>>
>>>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>>>> + else
>>>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>>>> +
>>>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>>>> + kfree(vb);
>>>>>>>>>>> +
>>>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>>> + return 0;
>>>>>>>>>>> +}
>>>>>>>>>>> +
>>>>>>>>>>> /*
>>>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>>> struct
>>>>>>>>>>> media_device *mdev,
>>>>>>>>>>> }
>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>>
>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>>>> +{
>>>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>>>> +}
>>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>>> +
>>>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>>> *create)
>>>>>>>>>>> {
>>>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>>> video_device *vdev)
>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>>> vidioc_streamon);
>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file,
>>>>>>>>>>> fh, b);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>> + struct file *file, void *fh, void
>>>>>>>>>>> *arg)
>>>>>>>>>>> +{
>>>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>>>> +
>>>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>>>> +}
>>>>>>>>>>> +
>>>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>> struct file *file, void *fh,
>>>>>>>>>>> void
>>>>>>>>>>> *arg)
>>>>>>>>>>> {
>>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS,
>>>>>>>>>>> v4l_enum_freq_bands,
>>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO,
>>>>>>>>>>> v4l_dbg_g_chip_info,
>>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>>> match)),
>>>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>>> };
>>>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that
>>>>>>>>>>> implements
>>>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>>>> *b);
>>>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>>>
>>>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>>> unsigned
>>>>>>>>>>> int i);
>>>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>>> *q,
>>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>>> */
>>>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>> index,
>>>>>>>>>>> void *pb);
>>>>>>>>>>>
>>>>>>>>>>> +/**
>>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>>>> + *
>>>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>>>> + */
>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>> index);
>>>>>>>>>>> +
>>>>>>>>>>> /**
>>>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>>> *
>>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>>> struct
>>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>>> */
>>>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>>> *mdev,
>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>> +/**
>>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>>> + *
>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>>>> queue.
>>>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>>> + *
>>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>>> directly
>>>>>>>>>>> returned
>>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>>> + */
>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer
>>>>>>>>>>> *b);
>>>>>>>>>>>
>>>>>>>>>>> /**
>>>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>>
>>>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>>>> v4l2_buffer)
>>>>>>>>>>> +
>>>>>>>>>>>
>>>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>>>> them to
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>>> --
>>>>>>>>>>> 2.39.2
>>>>>>>>>>>
>>>>>>>> --
>>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>>
> --
> Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 11:17 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-07-03 11:17 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: linux-media, linux-kernel, p.zabel, ming.qian, hverkuil-cisco,
gregkh, ezequiel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, nicolas.dufresne, linux-staging,
kernel, mchehab, tfiga, m.szyprowski, ayaka
Le 03/07/2023 à 13:02, Hsia-Jun Li a écrit :
>
>
> On 7/3/23 18:35, Benjamin Gaignard wrote:
>> CAUTION: Email originated externally, do not click links or open
>> attachments unless you recognize the sender and know the content is
>> safe.
>>
>>
>> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>>
>>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>>
>>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>>
>>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>> content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>>
>>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>>> CAUTION: Email originated externally, do not click links or
>>>>>>>>>>> open
>>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>>> content is
>>>>>>>>>>> safe.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>>>> ---
>>>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>>>> create mode 100644
>>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>
>>>>>>>>>>> diff --git
>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>>>> + vidioc-delete-buf
>>>>>>>>>>> vidioc-dqevent
>>>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>>>> diff --git
>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> new file mode 100644
>>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>>> --- /dev/null
>>>>>>>>>>> +++
>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>>> +
>>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>>> +
>>>>>>>>>>> +************************
>>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>>> +************************
>>>>>>>>>>> +
>>>>>>>>>>> +Name
>>>>>>>>>>> +====
>>>>>>>>>>> +
>>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>>> +
>>>>>>>>>>> +Synopsis
>>>>>>>>>>> +========
>>>>>>>>>>> +
>>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>>> +
>>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>>> *argp)``
>>>>>>>>>>> +
>>>>>>>>>>> +Arguments
>>>>>>>>>>> +=========
>>>>>>>>>>> +
>>>>>>>>>>> +``fd``
>>>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>>>> +
>>>>>>>>>>> +``argp``
>>>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>>> +
>>>>>>>>>>> +Description
>>>>>>>>>>> +===========
>>>>>>>>>>> +
>>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>>> ioctl to
>>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>>> +
>>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>>> +
>>>>>>>>>>> +Return Value
>>>>>>>>>>> +============
>>>>>>>>>>> +
>>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>>> variable is
>>>>>>>>>>> set
>>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>>> +
>>>>>>>>>>> +EBUSY
>>>>>>>>>>> + File I/O is in progress.
>>>>>>>>>>> +
>>>>>>>>>>> +EINVAL
>>>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct
>>>>>>>>>>> vb2_queue
>>>>>>>>>>> *q,
>>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>>> }
>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>>
>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>> index)
>>>>>>>>>>> +{
>>>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>>>> +
>>>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>>>> + if (!vb) {
>>>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>>> index);
>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer
>>>>>>>>>>> index
>>>>>>>>>>> %d\n", index);
>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>> + }
>>>>>>>>>>> +
>>>>>>>>>> I know the driver could implement its own
>>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a
>>>>>>>>>> buffer is
>>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>>> still
>>>>>>>>>> used by the hardware.
>>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>>> it is
>>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>>> While the Golden reference frame, such long term reference frame
>>>>>>>>>> could
>>>>>>>>>> last much longer.
>>>>>>>>>
>>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>>> information for that.
>>>>>>>>
>>>>>>>> That is true for the stateless codec driver.
>>>>>>>>
>>>>>>>> While application for stateful decoder could never do that. It
>>>>>>>> also
>>>>>>>> breaks what the document said:
>>>>>>>>
>>>>>>>> "The backing memory of |CAPTURE| buffers that are used as
>>>>>>>> reference
>>>>>>>> frames by the stream may be read by the hardware even after
>>>>>>>> they are
>>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>>> memory
>>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe this
>>>>>>>> may
>>>>>>>> result in corruption of decoded frames."
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>>> +
>>>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>>>
>>>>>>> Here is another problem for the existing application, the mmap()
>>>>>>> from
>>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>>> buffer in this step(while the exportbuffer would create one
>>>>>>> itself).
>>>>>>>
>>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>>> behavior is right, because mmap(2) says:
>>>>>>>
>>>>>>> "After the mmap() call has returned, the file descriptor, fd,
>>>>>>> can
>>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>>
>>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>>> they will be aware that the buffer is removed.
>>>>>> I have done it in GStreamer:
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> I have read that.
>>>>>
>>>>> There is not a VP8 parser in Gstreamer, while a parser would not work
>>>>> when deal with the secure video(userspace can't access the data
>>>>> context at all).
>>>>>
>>>>> Besides, this adds extra work for the application for a stateful
>>>>> codec
>>>>> driver. The application need to parser the bitstream and track the
>>>>> dpb.
>>>>>
>>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>>> non-display frame and internal reference state.
>>>>>
>>>>> That could be requirement for codec firmware that its driver could
>>>>> support this DELETE_BUF ioctl() feature.
>>>>
>>>> Sorry I don't see the link with my patches here...
>>>> I have work on non-secure VP9 on stateless codec.
>>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>>> applications
>>>> to save memory if they know when they could delete buffers without
>>>> risk.
>>>
>>> I try to explain why I think this design in not "complete". One
>>> problem resolved, more problems would occur.
>>>
>>>
>>> For non-secure video, those applications have worked:
>>>
>>> - It would break what stateful means here, application need to
>>> acquire(parse) the information that driver should offer.
>>>
>>> Or it would break the decoding model.
>>>
>>> - Your Gstreamer sample code or this design won't work for AV1.
>> why ?
>
> A frame in AV1 could be put to display order with a future frame. That
> is quite different to those ITU codecs.
>
> Supposing a frame (we call it frame 0) without show_display frame, you
> could delete it (just want to do that) once it is decoded.
>
> Actually, unless 7 future frames have came and parser has acknowledged
> them, it is not safe to delete that frame.
>
> Or a show_exist that make frame 0 present.
>
> In my personal opinion, this feature could break many uAPI's, I have
> said my worrying in the other emails.
>
Of course if the application delete a useful buffer it is a problem but an application issue.
I have test this code using GStreamer with VP9, VP8 and HEVC on Verisilicon driver without issues.
I will be happy if someone could test it on another driver and/or other userlands (like ffmpeg).
>>>
>>> For all the future possible secure video:
>>>
>>> - This feature could never be used from the current design.
>>>
>>>
>>>>
>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Benjamin
>>>>>>
>>>>>>>
>>>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>>>> + else
>>>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>>>> +
>>>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>>>> + kfree(vb);
>>>>>>>>>>> +
>>>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>>> + return 0;
>>>>>>>>>>> +}
>>>>>>>>>>> +
>>>>>>>>>>> /*
>>>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>>> struct
>>>>>>>>>>> media_device *mdev,
>>>>>>>>>>> }
>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>>
>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>>>>> +{
>>>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>>>> +}
>>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>>> +
>>>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>>> *create)
>>>>>>>>>>> {
>>>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>>> video_device *vdev)
>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>>> vidioc_streamon);
>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file,
>>>>>>>>>>> fh, b);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>> + struct file *file, void *fh, void
>>>>>>>>>>> *arg)
>>>>>>>>>>> +{
>>>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>>>> +
>>>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>>>>> +}
>>>>>>>>>>> +
>>>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>> struct file *file, void *fh,
>>>>>>>>>>> void
>>>>>>>>>>> *arg)
>>>>>>>>>>> {
>>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS,
>>>>>>>>>>> v4l_enum_freq_bands,
>>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO,
>>>>>>>>>>> v4l_dbg_g_chip_info,
>>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>>> match)),
>>>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>>> };
>>>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>>
>>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that
>>>>>>>>>>> implements
>>>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>>>> *b);
>>>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>>>
>>>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>>> unsigned
>>>>>>>>>>> int i);
>>>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>>> *q,
>>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>>> */
>>>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>> index,
>>>>>>>>>>> void *pb);
>>>>>>>>>>>
>>>>>>>>>>> +/**
>>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>>>> + *
>>>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>>>> + */
>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>> index);
>>>>>>>>>>> +
>>>>>>>>>>> /**
>>>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>>> *
>>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>>> struct
>>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>>> */
>>>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>>> *mdev,
>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>> +/**
>>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>>> + *
>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>>>> queue.
>>>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>>> + *
>>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>>> directly
>>>>>>>>>>> returned
>>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>>> + */
>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer
>>>>>>>>>>> *b);
>>>>>>>>>>>
>>>>>>>>>>> /**
>>>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>>
>>>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>>>> v4l2_buffer)
>>>>>>>>>>> +
>>>>>>>>>>>
>>>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>>>> them to
>>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>>> --
>>>>>>>>>>> 2.39.2
>>>>>>>>>>>
>>>>>>>> --
>>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>>
> --
> Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-07-03 11:17 ` Benjamin Gaignard
(?)
@ 2023-07-03 15:42 ` Randy Li
-1 siblings, 0 replies; 183+ messages in thread
From: Randy Li @ 2023-07-03 15:42 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, Hsia-Jun Li, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, tfiga, m.szyprowski
On 2023/7/3 19:17, Benjamin Gaignard wrote:
>
> Le 03/07/2023 à 13:02, Hsia-Jun Li a écrit :
>>
>>
>> On 7/3/23 18:35, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>>>
>>>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>>>
>>>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>> content is
>>>>>>>>>> safe.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>>>
>>>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>>>> CAUTION: Email originated externally, do not click links or
>>>>>>>>>>>> open
>>>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>>>> content is
>>>>>>>>>>>> safe.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a
>>>>>>>>>>>> queue.
>>>>>>>>>>>>
>>>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>>>>> ---
>>>>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>>>>> create mode 100644
>>>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>>
>>>>>>>>>>>> diff --git
>>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>>>>> + vidioc-delete-buf
>>>>>>>>>>>> vidioc-dqevent
>>>>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>>>>> diff --git
>>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> new file mode 100644
>>>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>>>> --- /dev/null
>>>>>>>>>>>> +++
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>>>> +
>>>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>>>> +
>>>>>>>>>>>> +************************
>>>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>>>> +************************
>>>>>>>>>>>> +
>>>>>>>>>>>> +Name
>>>>>>>>>>>> +====
>>>>>>>>>>>> +
>>>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>>>> +
>>>>>>>>>>>> +Synopsis
>>>>>>>>>>>> +========
>>>>>>>>>>>> +
>>>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>>>> +
>>>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>>>> *argp)``
>>>>>>>>>>>> +
>>>>>>>>>>>> +Arguments
>>>>>>>>>>>> +=========
>>>>>>>>>>>> +
>>>>>>>>>>>> +``fd``
>>>>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +``argp``
>>>>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +Description
>>>>>>>>>>>> +===========
>>>>>>>>>>>> +
>>>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>>>> ioctl to
>>>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>>>> +
>>>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +Return Value
>>>>>>>>>>>> +============
>>>>>>>>>>>> +
>>>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>>>> variable is
>>>>>>>>>>>> set
>>>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>>>> +
>>>>>>>>>>>> +EBUSY
>>>>>>>>>>>> + File I/O is in progress.
>>>>>>>>>>>> +
>>>>>>>>>>>> +EINVAL
>>>>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct
>>>>>>>>>>>> vb2_queue
>>>>>>>>>>>> *q,
>>>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>>>> }
>>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>>>
>>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>>> index)
>>>>>>>>>>>> +{
>>>>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>>>>> +
>>>>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>>>>> + if (!vb) {
>>>>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>>>> index);
>>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>>> + }
>>>>>>>>>>>> +
>>>>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued
>>>>>>>>>>>> buffer
>>>>>>>>>>>> index
>>>>>>>>>>>> %d\n", index);
>>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>>> + }
>>>>>>>>>>>> +
>>>>>>>>>>> I know the driver could implement its own
>>>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a
>>>>>>>>>>> buffer is
>>>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>>>> still
>>>>>>>>>>> used by the hardware.
>>>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>>>> it is
>>>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>>>> While the Golden reference frame, such long term reference
>>>>>>>>>>> frame
>>>>>>>>>>> could
>>>>>>>>>>> last much longer.
>>>>>>>>>>
>>>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>>>> information for that.
>>>>>>>>>
>>>>>>>>> That is true for the stateless codec driver.
>>>>>>>>>
>>>>>>>>> While application for stateful decoder could never do that. It
>>>>>>>>> also
>>>>>>>>> breaks what the document said:
>>>>>>>>>
>>>>>>>>> "The backing memory of |CAPTURE| buffers that are used as
>>>>>>>>> reference
>>>>>>>>> frames by the stream may be read by the hardware even after
>>>>>>>>> they are
>>>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>>>> memory
>>>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe
>>>>>>>>> this may
>>>>>>>>> result in corruption of decoded frames."
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>>>>
>>>>>>>> Here is another problem for the existing application, the
>>>>>>>> mmap() from
>>>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>>>> buffer in this step(while the exportbuffer would create one
>>>>>>>> itself).
>>>>>>>>
>>>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>>>> behavior is right, because mmap(2) says:
>>>>>>>>
>>>>>>>> "After the mmap() call has returned, the file descriptor,
>>>>>>>> fd, can
>>>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>>>
>>>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>>>> they will be aware that the buffer is removed.
>>>>>>> I have done it in GStreamer:
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> I have read that.
>>>>>>
>>>>>> There is not a VP8 parser in Gstreamer, while a parser would not
>>>>>> work
>>>>>> when deal with the secure video(userspace can't access the data
>>>>>> context at all).
>>>>>>
>>>>>> Besides, this adds extra work for the application for a stateful
>>>>>> codec
>>>>>> driver. The application need to parser the bitstream and track
>>>>>> the dpb.
>>>>>>
>>>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>>>> non-display frame and internal reference state.
>>>>>>
>>>>>> That could be requirement for codec firmware that its driver could
>>>>>> support this DELETE_BUF ioctl() feature.
>>>>>
>>>>> Sorry I don't see the link with my patches here...
>>>>> I have work on non-secure VP9 on stateless codec.
>>>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>>>> applications
>>>>> to save memory if they know when they could delete buffers without
>>>>> risk.
>>>>
>>>> I try to explain why I think this design in not "complete". One
>>>> problem resolved, more problems would occur.
>>>>
>>>>
>>>> For non-secure video, those applications have worked:
>>>>
>>>> - It would break what stateful means here, application need to
>>>> acquire(parse) the information that driver should offer.
>>>>
>>>> Or it would break the decoding model.
>>>>
>>>> - Your Gstreamer sample code or this design won't work for AV1.
>>> why ?
>>
>> A frame in AV1 could be put to display order with a future frame.
>> That is quite different to those ITU codecs.
>>
>> Supposing a frame (we call it frame 0) without show_display frame,
>> you could delete it (just want to do that) once it is decoded.
>>
>> Actually, unless 7 future frames have came and parser has
>> acknowledged them, it is not safe to delete that frame.
>>
>> Or a show_exist that make frame 0 present.
>>
>> In my personal opinion, this feature could break many uAPI's, I have
>> said my worrying in the other emails.
>>
> Of course if the application delete a useful buffer it is a problem
> but an application issue.
I don't think this feature is only applied to stateless codec. Stateful
codec could reclaim the previous memory as well.
The implementation now just leave all those problems to the conscience
of the application, no protection from the kernel which could crash the
hardware easily.
I am not against on this feature, I just wish it would be more safe and
more complete.
> I have test this code using GStreamer with VP9, VP8 and HEVC on
> Verisilicon driver without issues.
Even with the secure decoding, at least someone should offer the DPB
info or we just don't need this.
I am sure stateless decoder could work well with that despite the
performance impact on buffer looking up.
>
> I will be happy if someone could test it on another driver and/or
> other userlands (like ffmpeg).
>
>>>>
>>>> For all the future possible secure video:
>>>>
>>>> - This feature could never be used from the current design.
>>>>
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Benjamin
>>>>>>>
>>>>>>>>
>>>>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>>>>> + else
>>>>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>>>>> + kfree(vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>>>> + return 0;
>>>>>>>>>>>> +}
>>>>>>>>>>>> +
>>>>>>>>>>>> /*
>>>>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>>>> struct
>>>>>>>>>>>> media_device *mdev,
>>>>>>>>>>>> }
>>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>>>
>>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer
>>>>>>>>>>>> *b)
>>>>>>>>>>>> +{
>>>>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>>>>> +}
>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>>>> +
>>>>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>>>> *create)
>>>>>>>>>>>> {
>>>>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>>>> video_device *vdev)
>>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>>>> vidioc_streamon);
>>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file,
>>>>>>>>>>>> fh, b);
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>>> + struct file *file, void *fh, void
>>>>>>>>>>>> *arg)
>>>>>>>>>>>> +{
>>>>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>>>>> +
>>>>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh,
>>>>>>>>>>>> b);
>>>>>>>>>>>> +}
>>>>>>>>>>>> +
>>>>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>>> struct file *file, void *fh,
>>>>>>>>>>>> void
>>>>>>>>>>>> *arg)
>>>>>>>>>>>> {
>>>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS,
>>>>>>>>>>>> v4l_enum_freq_bands,
>>>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO,
>>>>>>>>>>>> v4l_dbg_g_chip_info,
>>>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>>>> match)),
>>>>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>>>> };
>>>>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>>>
>>>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that
>>>>>>>>>>>> implements
>>>>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that
>>>>>>>>>>>> implements
>>>>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>>>>> *b);
>>>>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void
>>>>>>>>>>>> *fh,
>>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>>>>
>>>>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>>>> unsigned
>>>>>>>>>>>> int i);
>>>>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>>>> *q,
>>>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>>>> */
>>>>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>>> index,
>>>>>>>>>>>> void *pb);
>>>>>>>>>>>>
>>>>>>>>>>>> +/**
>>>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>>>>> + *
>>>>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>>>>> + */
>>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>>> index);
>>>>>>>>>>>> +
>>>>>>>>>>>> /**
>>>>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>>>> *
>>>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>>>> struct
>>>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>>>> */
>>>>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>>>> *mdev,
>>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>>> +/**
>>>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>>>> + *
>>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>>>>> queue.
>>>>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>>>> + *
>>>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>>>> directly
>>>>>>>>>>>> returned
>>>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>>>> + */
>>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer
>>>>>>>>>>>> *b);
>>>>>>>>>>>>
>>>>>>>>>>>> /**
>>>>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>>>
>>>>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>>>>> v4l2_buffer)
>>>>>>>>>>>> +
>>>>>>>>>>>>
>>>>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>>>>> them to
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>>>> --
>>>>>>>>>>>> 2.39.2
>>>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>>>
>> --
>> Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 15:42 ` Randy Li
0 siblings, 0 replies; 183+ messages in thread
From: Randy Li @ 2023-07-03 15:42 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, Hsia-Jun Li, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, tfiga, m.szyprowski
On 2023/7/3 19:17, Benjamin Gaignard wrote:
>
> Le 03/07/2023 à 13:02, Hsia-Jun Li a écrit :
>>
>>
>> On 7/3/23 18:35, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>>>
>>>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>>>
>>>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>> content is
>>>>>>>>>> safe.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>>>
>>>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>>>> CAUTION: Email originated externally, do not click links or
>>>>>>>>>>>> open
>>>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>>>> content is
>>>>>>>>>>>> safe.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a
>>>>>>>>>>>> queue.
>>>>>>>>>>>>
>>>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>>>>> ---
>>>>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>>>>> create mode 100644
>>>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>>
>>>>>>>>>>>> diff --git
>>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>>>>> + vidioc-delete-buf
>>>>>>>>>>>> vidioc-dqevent
>>>>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>>>>> diff --git
>>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> new file mode 100644
>>>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>>>> --- /dev/null
>>>>>>>>>>>> +++
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>>>> +
>>>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>>>> +
>>>>>>>>>>>> +************************
>>>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>>>> +************************
>>>>>>>>>>>> +
>>>>>>>>>>>> +Name
>>>>>>>>>>>> +====
>>>>>>>>>>>> +
>>>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>>>> +
>>>>>>>>>>>> +Synopsis
>>>>>>>>>>>> +========
>>>>>>>>>>>> +
>>>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>>>> +
>>>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>>>> *argp)``
>>>>>>>>>>>> +
>>>>>>>>>>>> +Arguments
>>>>>>>>>>>> +=========
>>>>>>>>>>>> +
>>>>>>>>>>>> +``fd``
>>>>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +``argp``
>>>>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +Description
>>>>>>>>>>>> +===========
>>>>>>>>>>>> +
>>>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>>>> ioctl to
>>>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>>>> +
>>>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +Return Value
>>>>>>>>>>>> +============
>>>>>>>>>>>> +
>>>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>>>> variable is
>>>>>>>>>>>> set
>>>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>>>> +
>>>>>>>>>>>> +EBUSY
>>>>>>>>>>>> + File I/O is in progress.
>>>>>>>>>>>> +
>>>>>>>>>>>> +EINVAL
>>>>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct
>>>>>>>>>>>> vb2_queue
>>>>>>>>>>>> *q,
>>>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>>>> }
>>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>>>
>>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>>> index)
>>>>>>>>>>>> +{
>>>>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>>>>> +
>>>>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>>>>> + if (!vb) {
>>>>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>>>> index);
>>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>>> + }
>>>>>>>>>>>> +
>>>>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued
>>>>>>>>>>>> buffer
>>>>>>>>>>>> index
>>>>>>>>>>>> %d\n", index);
>>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>>> + }
>>>>>>>>>>>> +
>>>>>>>>>>> I know the driver could implement its own
>>>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a
>>>>>>>>>>> buffer is
>>>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>>>> still
>>>>>>>>>>> used by the hardware.
>>>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>>>> it is
>>>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>>>> While the Golden reference frame, such long term reference
>>>>>>>>>>> frame
>>>>>>>>>>> could
>>>>>>>>>>> last much longer.
>>>>>>>>>>
>>>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>>>> information for that.
>>>>>>>>>
>>>>>>>>> That is true for the stateless codec driver.
>>>>>>>>>
>>>>>>>>> While application for stateful decoder could never do that. It
>>>>>>>>> also
>>>>>>>>> breaks what the document said:
>>>>>>>>>
>>>>>>>>> "The backing memory of |CAPTURE| buffers that are used as
>>>>>>>>> reference
>>>>>>>>> frames by the stream may be read by the hardware even after
>>>>>>>>> they are
>>>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>>>> memory
>>>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe
>>>>>>>>> this may
>>>>>>>>> result in corruption of decoded frames."
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>>>>
>>>>>>>> Here is another problem for the existing application, the
>>>>>>>> mmap() from
>>>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>>>> buffer in this step(while the exportbuffer would create one
>>>>>>>> itself).
>>>>>>>>
>>>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>>>> behavior is right, because mmap(2) says:
>>>>>>>>
>>>>>>>> "After the mmap() call has returned, the file descriptor,
>>>>>>>> fd, can
>>>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>>>
>>>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>>>> they will be aware that the buffer is removed.
>>>>>>> I have done it in GStreamer:
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> I have read that.
>>>>>>
>>>>>> There is not a VP8 parser in Gstreamer, while a parser would not
>>>>>> work
>>>>>> when deal with the secure video(userspace can't access the data
>>>>>> context at all).
>>>>>>
>>>>>> Besides, this adds extra work for the application for a stateful
>>>>>> codec
>>>>>> driver. The application need to parser the bitstream and track
>>>>>> the dpb.
>>>>>>
>>>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>>>> non-display frame and internal reference state.
>>>>>>
>>>>>> That could be requirement for codec firmware that its driver could
>>>>>> support this DELETE_BUF ioctl() feature.
>>>>>
>>>>> Sorry I don't see the link with my patches here...
>>>>> I have work on non-secure VP9 on stateless codec.
>>>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>>>> applications
>>>>> to save memory if they know when they could delete buffers without
>>>>> risk.
>>>>
>>>> I try to explain why I think this design in not "complete". One
>>>> problem resolved, more problems would occur.
>>>>
>>>>
>>>> For non-secure video, those applications have worked:
>>>>
>>>> - It would break what stateful means here, application need to
>>>> acquire(parse) the information that driver should offer.
>>>>
>>>> Or it would break the decoding model.
>>>>
>>>> - Your Gstreamer sample code or this design won't work for AV1.
>>> why ?
>>
>> A frame in AV1 could be put to display order with a future frame.
>> That is quite different to those ITU codecs.
>>
>> Supposing a frame (we call it frame 0) without show_display frame,
>> you could delete it (just want to do that) once it is decoded.
>>
>> Actually, unless 7 future frames have came and parser has
>> acknowledged them, it is not safe to delete that frame.
>>
>> Or a show_exist that make frame 0 present.
>>
>> In my personal opinion, this feature could break many uAPI's, I have
>> said my worrying in the other emails.
>>
> Of course if the application delete a useful buffer it is a problem
> but an application issue.
I don't think this feature is only applied to stateless codec. Stateful
codec could reclaim the previous memory as well.
The implementation now just leave all those problems to the conscience
of the application, no protection from the kernel which could crash the
hardware easily.
I am not against on this feature, I just wish it would be more safe and
more complete.
> I have test this code using GStreamer with VP9, VP8 and HEVC on
> Verisilicon driver without issues.
Even with the secure decoding, at least someone should offer the DPB
info or we just don't need this.
I am sure stateless decoder could work well with that despite the
performance impact on buffer looking up.
>
> I will be happy if someone could test it on another driver and/or
> other userlands (like ffmpeg).
>
>>>>
>>>> For all the future possible secure video:
>>>>
>>>> - This feature could never be used from the current design.
>>>>
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Benjamin
>>>>>>>
>>>>>>>>
>>>>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>>>>> + else
>>>>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>>>>> + kfree(vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>>>> + return 0;
>>>>>>>>>>>> +}
>>>>>>>>>>>> +
>>>>>>>>>>>> /*
>>>>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>>>> struct
>>>>>>>>>>>> media_device *mdev,
>>>>>>>>>>>> }
>>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>>>
>>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer
>>>>>>>>>>>> *b)
>>>>>>>>>>>> +{
>>>>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>>>>> +}
>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>>>> +
>>>>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>>>> *create)
>>>>>>>>>>>> {
>>>>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>>>> video_device *vdev)
>>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>>>> vidioc_streamon);
>>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file,
>>>>>>>>>>>> fh, b);
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>>> + struct file *file, void *fh, void
>>>>>>>>>>>> *arg)
>>>>>>>>>>>> +{
>>>>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>>>>> +
>>>>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh,
>>>>>>>>>>>> b);
>>>>>>>>>>>> +}
>>>>>>>>>>>> +
>>>>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>>> struct file *file, void *fh,
>>>>>>>>>>>> void
>>>>>>>>>>>> *arg)
>>>>>>>>>>>> {
>>>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS,
>>>>>>>>>>>> v4l_enum_freq_bands,
>>>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO,
>>>>>>>>>>>> v4l_dbg_g_chip_info,
>>>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>>>> match)),
>>>>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>>>> };
>>>>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>>>
>>>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that
>>>>>>>>>>>> implements
>>>>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that
>>>>>>>>>>>> implements
>>>>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>>>>> *b);
>>>>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void
>>>>>>>>>>>> *fh,
>>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>>>>
>>>>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>>>> unsigned
>>>>>>>>>>>> int i);
>>>>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>>>> *q,
>>>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>>>> */
>>>>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>>> index,
>>>>>>>>>>>> void *pb);
>>>>>>>>>>>>
>>>>>>>>>>>> +/**
>>>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>>>>> + *
>>>>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>>>>> + */
>>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>>> index);
>>>>>>>>>>>> +
>>>>>>>>>>>> /**
>>>>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>>>> *
>>>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>>>> struct
>>>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>>>> */
>>>>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>>>> *mdev,
>>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>>> +/**
>>>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>>>> + *
>>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>>>>> queue.
>>>>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>>>> + *
>>>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>>>> directly
>>>>>>>>>>>> returned
>>>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>>>> + */
>>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer
>>>>>>>>>>>> *b);
>>>>>>>>>>>>
>>>>>>>>>>>> /**
>>>>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>>>
>>>>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>>>>> v4l2_buffer)
>>>>>>>>>>>> +
>>>>>>>>>>>>
>>>>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>>>>> them to
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>>>> --
>>>>>>>>>>>> 2.39.2
>>>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>>>
>> --
>> Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-03 15:42 ` Randy Li
0 siblings, 0 replies; 183+ messages in thread
From: Randy Li @ 2023-07-03 15:42 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, Hsia-Jun Li, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, tfiga, m.szyprowski
On 2023/7/3 19:17, Benjamin Gaignard wrote:
>
> Le 03/07/2023 à 13:02, Hsia-Jun Li a écrit :
>>
>>
>> On 7/3/23 18:35, Benjamin Gaignard wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> Le 03/07/2023 à 11:20, Hsia-Jun Li a écrit :
>>>>
>>>> On 7/3/23 16:52, Benjamin Gaignard wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> Le 03/07/2023 à 10:19, Hsia-Jun Li a écrit :
>>>>>>
>>>>>> On 7/3/23 16:12, Benjamin Gaignard wrote:
>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>> safe.
>>>>>>>
>>>>>>>
>>>>>>> Le 30/06/2023 à 11:43, Hsia-Jun Li a écrit :
>>>>>>>>
>>>>>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>> content is
>>>>>>>>> safe.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>> content is
>>>>>>>>>> safe.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>>>>>>
>>>>>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>>>>>> CAUTION: Email originated externally, do not click links or
>>>>>>>>>>>> open
>>>>>>>>>>>> attachments unless you recognize the sender and know the
>>>>>>>>>>>> content is
>>>>>>>>>>>> safe.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a
>>>>>>>>>>>> queue.
>>>>>>>>>>>>
>>>>>>>>>>>> Signed-off-by: Benjamin Gaignard
>>>>>>>>>>>> <benjamin.gaignard@collabora.com>
>>>>>>>>>>>> ---
>>>>>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>>>>>> +++++++++++++++++++
>>>>>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>>>>>> create mode 100644
>>>>>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>>
>>>>>>>>>>>> diff --git
>>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>>>>>> vidioc-dbg-g-register
>>>>>>>>>>>> vidioc-decoder-cmd
>>>>>>>>>>>> + vidioc-delete-buf
>>>>>>>>>>>> vidioc-dqevent
>>>>>>>>>>>> vidioc-dv-timings-cap
>>>>>>>>>>>> vidioc-encoder-cmd
>>>>>>>>>>>> diff --git
>>>>>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> new file mode 100644
>>>>>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>>>>>> --- /dev/null
>>>>>>>>>>>> +++
>>>>>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>>>>>> +.. c:namespace:: V4L
>>>>>>>>>>>> +
>>>>>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>>>>>> +
>>>>>>>>>>>> +************************
>>>>>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>>>>>> +************************
>>>>>>>>>>>> +
>>>>>>>>>>>> +Name
>>>>>>>>>>>> +====
>>>>>>>>>>>> +
>>>>>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>>>>>> +
>>>>>>>>>>>> +Synopsis
>>>>>>>>>>>> +========
>>>>>>>>>>>> +
>>>>>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>>>>>> +
>>>>>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer
>>>>>>>>>>>> *argp)``
>>>>>>>>>>>> +
>>>>>>>>>>>> +Arguments
>>>>>>>>>>>> +=========
>>>>>>>>>>>> +
>>>>>>>>>>>> +``fd``
>>>>>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +``argp``
>>>>>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +Description
>>>>>>>>>>>> +===========
>>>>>>>>>>>> +
>>>>>>>>>>>> +Applications can optionally call the :ref:`VIDIOC_DELETE_BUF`
>>>>>>>>>>>> ioctl to
>>>>>>>>>>>> +delete a buffer from a queue.
>>>>>>>>>>>> +
>>>>>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>>>>>> +:ref:`buffer`.
>>>>>>>>>>>> +
>>>>>>>>>>>> +Return Value
>>>>>>>>>>>> +============
>>>>>>>>>>>> +
>>>>>>>>>>>> +On success 0 is returned, on error -1 and the ``errno``
>>>>>>>>>>>> variable is
>>>>>>>>>>>> set
>>>>>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>>>>>> +
>>>>>>>>>>>> +EBUSY
>>>>>>>>>>>> + File I/O is in progress.
>>>>>>>>>>>> +
>>>>>>>>>>>> +EINVAL
>>>>>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct
>>>>>>>>>>>> vb2_queue
>>>>>>>>>>>> *q,
>>>>>>>>>>>> unsigned int index, void *pb)
>>>>>>>>>>>> }
>>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>>>>>
>>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>>> index)
>>>>>>>>>>>> +{
>>>>>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>>>>>> +
>>>>>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>>>>>> + if (!vb) {
>>>>>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n",
>>>>>>>>>>>> index);
>>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>>> + }
>>>>>>>>>>>> +
>>>>>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>>>>>> + dprintk(q, 1, "can't delete non dequeued
>>>>>>>>>>>> buffer
>>>>>>>>>>>> index
>>>>>>>>>>>> %d\n", index);
>>>>>>>>>>>> + return -EINVAL;
>>>>>>>>>>>> + }
>>>>>>>>>>>> +
>>>>>>>>>>> I know the driver could implement its own
>>>>>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a
>>>>>>>>>>> buffer is
>>>>>>>>>>> used by the hardware as a future reference frame.
>>>>>>>>>>> But I think we need a flag to let the user know which buffer is
>>>>>>>>>>> still
>>>>>>>>>>> used by the hardware.
>>>>>>>>>>> Alternative ref case is safe, we only know it's existing when
>>>>>>>>>>> it is
>>>>>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>>>>>> While the Golden reference frame, such long term reference
>>>>>>>>>>> frame
>>>>>>>>>>> could
>>>>>>>>>>> last much longer.
>>>>>>>>>>
>>>>>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>>>>>> information for that.
>>>>>>>>>
>>>>>>>>> That is true for the stateless codec driver.
>>>>>>>>>
>>>>>>>>> While application for stateful decoder could never do that. It
>>>>>>>>> also
>>>>>>>>> breaks what the document said:
>>>>>>>>>
>>>>>>>>> "The backing memory of |CAPTURE| buffers that are used as
>>>>>>>>> reference
>>>>>>>>> frames by the stream may be read by the hardware even after
>>>>>>>>> they are
>>>>>>>>> dequeued. Consequently, the client should avoid writing into this
>>>>>>>>> memory
>>>>>>>>> while the |CAPTURE| queue is streaming. Failure to observe
>>>>>>>>> this may
>>>>>>>>> result in corruption of decoded frames."
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>>>>>> + __vb2_buf_mem_free(vb);
>>>>>>>>
>>>>>>>> Here is another problem for the existing application, the
>>>>>>>> mmap() from
>>>>>>>> the mmap offset or exportbuffer fd would not create a reference to
>>>>>>>> buffer in this step(while the exportbuffer would create one
>>>>>>>> itself).
>>>>>>>>
>>>>>>>> When you delete a buffer, you may not release it from its virtual
>>>>>>>> memory space, leaving a corrupted virtual memory space. Also this
>>>>>>>> behavior is right, because mmap(2) says:
>>>>>>>>
>>>>>>>> "After the mmap() call has returned, the file descriptor,
>>>>>>>> fd, can
>>>>>>>> be closed immediately without invalidating the map‐ping."
>>>>>>>
>>>>>>> Existing applications do not call DELETE_BUF ioctl and when call it
>>>>>>> they will be aware that the buffer is removed.
>>>>>>> I have done it in GStreamer:
>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commit_fca0fbc934f4440693ce0ff6c8dc8a2e5f5f17d9&d=DwIDaQ&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=TGH9toTzGRfO5aBsfaMvGbcOw-28q6cPmpX6vScbHjpCtaLtb-RuvBvsJ0z9RvAB&s=Ufl1ccfRZf2EhnfCBvnQzRJV9CDhGxl5spe9WNECspU&e=
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> I have read that.
>>>>>>
>>>>>> There is not a VP8 parser in Gstreamer, while a parser would not
>>>>>> work
>>>>>> when deal with the secure video(userspace can't access the data
>>>>>> context at all).
>>>>>>
>>>>>> Besides, this adds extra work for the application for a stateful
>>>>>> codec
>>>>>> driver. The application need to parser the bitstream and track
>>>>>> the dpb.
>>>>>>
>>>>>> I don't mind if you could fix the nonfiction mechanism for those
>>>>>> non-display frame and internal reference state.
>>>>>>
>>>>>> That could be requirement for codec firmware that its driver could
>>>>>> support this DELETE_BUF ioctl() feature.
>>>>>
>>>>> Sorry I don't see the link with my patches here...
>>>>> I have work on non-secure VP9 on stateless codec.
>>>>> DELETE_BUF ioctl is optional and the main goal is to offer a way to
>>>>> applications
>>>>> to save memory if they know when they could delete buffers without
>>>>> risk.
>>>>
>>>> I try to explain why I think this design in not "complete". One
>>>> problem resolved, more problems would occur.
>>>>
>>>>
>>>> For non-secure video, those applications have worked:
>>>>
>>>> - It would break what stateful means here, application need to
>>>> acquire(parse) the information that driver should offer.
>>>>
>>>> Or it would break the decoding model.
>>>>
>>>> - Your Gstreamer sample code or this design won't work for AV1.
>>> why ?
>>
>> A frame in AV1 could be put to display order with a future frame.
>> That is quite different to those ITU codecs.
>>
>> Supposing a frame (we call it frame 0) without show_display frame,
>> you could delete it (just want to do that) once it is decoded.
>>
>> Actually, unless 7 future frames have came and parser has
>> acknowledged them, it is not safe to delete that frame.
>>
>> Or a show_exist that make frame 0 present.
>>
>> In my personal opinion, this feature could break many uAPI's, I have
>> said my worrying in the other emails.
>>
> Of course if the application delete a useful buffer it is a problem
> but an application issue.
I don't think this feature is only applied to stateless codec. Stateful
codec could reclaim the previous memory as well.
The implementation now just leave all those problems to the conscience
of the application, no protection from the kernel which could crash the
hardware easily.
I am not against on this feature, I just wish it would be more safe and
more complete.
> I have test this code using GStreamer with VP9, VP8 and HEVC on
> Verisilicon driver without issues.
Even with the secure decoding, at least someone should offer the DPB
info or we just don't need this.
I am sure stateless decoder could work well with that despite the
performance impact on buffer looking up.
>
> I will be happy if someone could test it on another driver and/or
> other userlands (like ffmpeg).
>
>>>>
>>>> For all the future possible secure video:
>>>>
>>>> - This feature could never be used from the current design.
>>>>
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Benjamin
>>>>>>>
>>>>>>>>
>>>>>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>>>>>> + else
>>>>>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>>>>>> + kfree(vb);
>>>>>>>>>>>> +
>>>>>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>>>>>> + return 0;
>>>>>>>>>>>> +}
>>>>>>>>>>>> +
>>>>>>>>>>>> /*
>>>>>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>>>>>> * @q: videobuf2 queue
>>>>>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q,
>>>>>>>>>>>> struct
>>>>>>>>>>>> media_device *mdev,
>>>>>>>>>>>> }
>>>>>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>>>>>
>>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer
>>>>>>>>>>>> *b)
>>>>>>>>>>>> +{
>>>>>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>>>>>> +}
>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>>>>>> +
>>>>>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>>>>>> v4l2_create_buffers
>>>>>>>>>>>> *create)
>>>>>>>>>>>> {
>>>>>>>>>>>> unsigned requested_planes = 1;
>>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>>>>>> video_device *vdev)
>>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>>>>>> vidioc_prepare_buf);
>>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>>>>>> vidioc_streamon);
>>>>>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>>>>>> vidioc_streamoff);
>>>>>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>>>>>> vidioc_delete_buf);
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file,
>>>>>>>>>>>> fh, b);
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>>> + struct file *file, void *fh, void
>>>>>>>>>>>> *arg)
>>>>>>>>>>>> +{
>>>>>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>>>>>> +
>>>>>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh,
>>>>>>>>>>>> b);
>>>>>>>>>>>> +}
>>>>>>>>>>>> +
>>>>>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>>>>>> struct file *file, void *fh,
>>>>>>>>>>>> void
>>>>>>>>>>>> *arg)
>>>>>>>>>>>> {
>>>>>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>>>>>> v4l2_ioctls[] = {
>>>>>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS,
>>>>>>>>>>>> v4l_enum_freq_bands,
>>>>>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO,
>>>>>>>>>>>> v4l_dbg_g_chip_info,
>>>>>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info,
>>>>>>>>>>>> match)),
>>>>>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>>>>>> };
>>>>>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>>>>>
>>>>>>>>>>>> diff --git a/include/media/v4l2-ioctl.h
>>>>>>>>>>>> b/include/media/v4l2-ioctl.h
>>>>>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>>>>>> * @vidioc_prepare_buf: pointer to the function that
>>>>>>>>>>>> implements
>>>>>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>>>>>> + * @vidioc_delete_buf: pointer to the function that
>>>>>>>>>>>> implements
>>>>>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>>>>>> struct v4l2_create_buffers
>>>>>>>>>>>> *b);
>>>>>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void
>>>>>>>>>>>> *fh,
>>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>>>>>
>>>>>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh,
>>>>>>>>>>>> unsigned
>>>>>>>>>>>> int i);
>>>>>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue
>>>>>>>>>>>> *q,
>>>>>>>>>>>> enum vb2_memory memory,
>>>>>>>>>>>> */
>>>>>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>>> index,
>>>>>>>>>>>> void *pb);
>>>>>>>>>>>>
>>>>>>>>>>>> +/**
>>>>>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>>>>>> + * @index: id number of the buffer.
>>>>>>>>>>>> + *
>>>>>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>>>>>> + */
>>>>>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int
>>>>>>>>>>>> index);
>>>>>>>>>>>> +
>>>>>>>>>>>> /**
>>>>>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>>>>>> *
>>>>>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q,
>>>>>>>>>>>> struct
>>>>>>>>>>>> v4l2_create_buffers *create);
>>>>>>>>>>>> */
>>>>>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device
>>>>>>>>>>>> *mdev,
>>>>>>>>>>>> struct v4l2_buffer *b);
>>>>>>>>>>>> +/**
>>>>>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>>>>>> + *
>>>>>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2
>>>>>>>>>>>> queue.
>>>>>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>>>>>> + *
>>>>>>>>>>>> + * The return values from this function are intended to be
>>>>>>>>>>>> directly
>>>>>>>>>>>> returned
>>>>>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>>>>>> + */
>>>>>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer
>>>>>>>>>>>> *b);
>>>>>>>>>>>>
>>>>>>>>>>>> /**
>>>>>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>>>>>
>>>>>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct
>>>>>>>>>>>> v4l2_buffer)
>>>>>>>>>>>> +
>>>>>>>>>>>>
>>>>>>>>>>>> /* Reminder: when adding new ioctls please add support for
>>>>>>>>>>>> them to
>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>>>>>> --
>>>>>>>>>>>> 2.39.2
>>>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Hsia-Jun(Randy) Li
>>>>>>>>>
>> --
>> Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-30 9:43 ` Hsia-Jun Li
(?)
@ 2023-07-13 9:09 ` Tomasz Figa
-1 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-13 9:09 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
>
> On 6/27/23 16:47, Hsia-Jun Li wrote:
> > CAUTION: Email originated externally, do not click links or open
> > attachments unless you recognize the sender and know the content is
> > safe.
> >
> >
> > On 6/27/23 16:43, Benjamin Gaignard wrote:
> > > CAUTION: Email originated externally, do not click links or open
> > > attachments unless you recognize the sender and know the content is
> > > safe.
> > >
> > >
> > > Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
> > > >
> > > > On 6/22/23 21:13, Benjamin Gaignard wrote:
> > > > > CAUTION: Email originated externally, do not click links or open
> > > > > attachments unless you recognize the sender and know the content is
> > > > > safe.
> > > > >
> > > > >
> > > > > VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
> > > > >
> > > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > > > ---
> > > > > .../userspace-api/media/v4l/user-func.rst | 1 +
> > > > > .../media/v4l/vidioc-delete-buf.rst | 51
> > > > > +++++++++++++++++++
> > > > > .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> > > > > .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> > > > > drivers/media/v4l2-core/v4l2-dev.c | 1 +
> > > > > drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> > > > > include/media/v4l2-ioctl.h | 4 ++
> > > > > include/media/videobuf2-core.h | 9 ++++
> > > > > include/media/videobuf2-v4l2.h | 11 ++++
> > > > > include/uapi/linux/videodev2.h | 2 +
> > > > > 10 files changed, 128 insertions(+)
> > > > > create mode 100644
> > > > > Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > >
> > > > > diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > b/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > index 15ff0bf7bbe6..8c74016e12fd 100644
> > > > > --- a/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > @@ -17,6 +17,7 @@ Function Reference
> > > > > vidioc-dbg-g-chip-info
> > > > > vidioc-dbg-g-register
> > > > > vidioc-decoder-cmd
> > > > > + vidioc-delete-buf
> > > > > vidioc-dqevent
> > > > > vidioc-dv-timings-cap
> > > > > vidioc-encoder-cmd
> > > > > diff --git
> > > > > a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > new file mode 100644
> > > > > index 000000000000..0e7ce58f91bc
> > > > > --- /dev/null
> > > > > +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > @@ -0,0 +1,51 @@
> > > > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > > > > +.. c:namespace:: V4L
> > > > > +
> > > > > +.. _VIDIOC_DELETE_BUF:
> > > > > +
> > > > > +************************
> > > > > +ioctl VIDIOC_DELETE_BUF
> > > > > +************************
> > > > > +
> > > > > +Name
> > > > > +====
> > > > > +
> > > > > +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> > > > > +
> > > > > +Synopsis
> > > > > +========
> > > > > +
> > > > > +.. c:macro:: VIDIOC_DELETE_BUF
> > > > > +
> > > > > +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> > > > > +
> > > > > +Arguments
> > > > > +=========
> > > > > +
> > > > > +``fd``
> > > > > + File descriptor returned by :c:func:`open()`.
> > > > > +
> > > > > +``argp``
> > > > > + Pointer to struct :c:type:`v4l2_buffer`.
> > > > > +
> > > > > +Description
> > > > > +===========
> > > > > +
> > > > > +Applications can optionally call the
> > > > > :ref:`VIDIOC_DELETE_BUF` ioctl to
> > > > > +delete a buffer from a queue.
> > > > > +
> > > > > +The struct :c:type:`v4l2_buffer` structure is specified in
> > > > > +:ref:`buffer`.
> > > > > +
> > > > > +Return Value
> > > > > +============
> > > > > +
> > > > > +On success 0 is returned, on error -1 and the ``errno`` variable is
> > > > > set
> > > > > +appropriately. The generic error codes are described at the
> > > > > +:ref:`Generic Error Codes <gen-errors>` chapter.
> > > > > +
> > > > > +EBUSY
> > > > > + File I/O is in progress.
> > > > > +
> > > > > +EINVAL
> > > > > + The buffer ``index`` doesn't exist in the queue.
> > > > > diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > index 899783f67580..aa546c972c3d 100644
> > > > > --- a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
> > > > > unsigned int index, void *pb)
> > > > > }
> > > > > EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
> > > > >
> > > > > +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> > > > > +{
> > > > > + struct vb2_buffer *vb;
> > > > > +
> > > > > + vb = vb2_get_buffer(q, index);
> > > > > + if (!vb) {
> > > > > + dprintk(q, 1, "invalid buffer index %d\n", index);
> > > > > + return -EINVAL;
> > > > > + }
> > > > > +
> > > > > + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> > > > > + dprintk(q, 1, "can't delete non dequeued buffer index
> > > > > %d\n", index);
> > > > > + return -EINVAL;
> > > > > + }
> > > > > +
> > > > I know the driver could implement its own
> > > > v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
> > > > used by the hardware as a future reference frame.
> > > > But I think we need a flag to let the user know which buffer is still
> > > > used by the hardware.
> > > > Alternative ref case is safe, we only know it's existing when it is
> > > > dequeued in current V4L2 buffer mechanism.
> > > > While the Golden reference frame, such long term reference frame could
> > > > last much longer.
> > >
> > > It is up to userland stack to know frames life time, it got the
> > > information for that.
> >
> > That is true for the stateless codec driver.
> >
> > While application for stateful decoder could never do that. It also
> > breaks what the document said:
> >
> > "The backing memory of |CAPTURE| buffers that are used as reference
> > frames by the stream may be read by the hardware even after they are
> > dequeued. Consequently, the client should avoid writing into this memory
> > while the |CAPTURE| queue is streaming. Failure to observe this may
> > result in corruption of decoded frames."
> >
> > >
> > > > > + if (vb->planes[0].mem_priv)
> > > > > + call_void_vb_qop(vb, buf_cleanup, vb);
> > > > > +
> > > > > + /* Free MMAP buffers or release USERPTR buffers */
> > > > > + if (q->memory == VB2_MEMORY_MMAP)
> > > > > + __vb2_buf_mem_free(vb);
>
> Here is another problem for the existing application, the mmap() from the
> mmap offset or exportbuffer fd would not create a reference to buffer in
> this step(while the exportbuffer would create one itself).
>
> When you delete a buffer, you may not release it from its virtual memory
> space, leaving a corrupted virtual memory space.
What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
memory, it just decrements a reference counter. The VMA holds its own,
so the buffer is only fully released when the application calls
munmap().
Best regards,
Tomasz
> Also this behavior is
> right, because mmap(2) says:
>
> "After the mmap() call has returned, the file descriptor, fd, can be
> closed immediately without invalidating the map‐ping."
>
> > > > > + else if (q->memory == VB2_MEMORY_DMABUF)
> > > > > + __vb2_buf_dmabuf_put(vb);
> > > > > + else
> > > > > + __vb2_buf_userptr_put(vb);
> > > > > +
> > > > > + vb2_queue_remove_buffer(q, vb);
> > > > > + kfree(vb);
> > > > > +
> > > > > + dprintk(q, 2, "buffer %d deleted\n", index);
> > > > > + return 0;
> > > > > +}
> > > > > +
> > > > > /*
> > > > > * vb2_start_streaming() - Attempt to start streaming.
> > > > > * @q: videobuf2 queue
> > > > > diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > index 724135d41f7f..cea666c17b41 100644
> > > > > --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
> > > > > media_device *mdev,
> > > > > }
> > > > > EXPORT_SYMBOL_GPL(vb2_prepare_buf);
> > > > >
> > > > > +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> > > > > +{
> > > > > + return vb2_core_delete_buf(q, b->index);
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> > > > > +
> > > > > int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
> > > > > *create)
> > > > > {
> > > > > unsigned requested_planes = 1;
> > > > > diff --git a/drivers/media/v4l2-core/v4l2-dev.c
> > > > > b/drivers/media/v4l2-core/v4l2-dev.c
> > > > > index f81279492682..80ace2e1e932 100644
> > > > > --- a/drivers/media/v4l2-core/v4l2-dev.c
> > > > > +++ b/drivers/media/v4l2-core/v4l2-dev.c
> > > > > @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
> > > > > video_device *vdev)
> > > > > SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
> > > > > vidioc_prepare_buf);
> > > > > SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
> > > > > vidioc_streamon);
> > > > > SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
> > > > > vidioc_streamoff);
> > > > > + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
> > > > > vidioc_delete_buf);
> > > > > }
> > > > >
> > > > > if (is_vid || is_vbi || is_meta) {
> > > > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > index a858acea6547..1c737279d3ef 100644
> > > > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
> > > > > v4l2_ioctl_ops *ops,
> > > > > return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
> > > > > }
> > > > >
> > > > > +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> > > > > + struct file *file, void *fh, void *arg)
> > > > > +{
> > > > > + struct v4l2_buffer *b = arg;
> > > > > + int ret = check_fmt(file, b->type);
> > > > > +
> > > > > + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> > > > > +}
> > > > > +
> > > > > static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> > > > > struct file *file, void *fh, void
> > > > > *arg)
> > > > > {
> > > > > @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
> > > > > v4l2_ioctls[] = {
> > > > > IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
> > > > > v4l_print_freq_band, 0),
> > > > > IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
> > > > > v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> > > > > IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
> > > > > v4l_print_query_ext_ctrl, INFO_FL_CTRL |
> > > > > INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> > > > > + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
> > > > > v4l_print_buffer, INFO_FL_QUEUE),
> > > > > };
> > > > > #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
> > > > >
> > > > > diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> > > > > index edb733f21604..2f232ed884c7 100644
> > > > > --- a/include/media/v4l2-ioctl.h
> > > > > +++ b/include/media/v4l2-ioctl.h
> > > > > @@ -163,6 +163,8 @@ struct v4l2_fh;
> > > > > * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> > > > > * @vidioc_prepare_buf: pointer to the function that implements
> > > > > * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> > > > > + * @vidioc_delete_buf: pointer to the function that implements
> > > > > + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> > > > > * @vidioc_overlay: pointer to the function that implements
> > > > > * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> > > > > * @vidioc_g_fbuf: pointer to the function that implements
> > > > > @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
> > > > > struct v4l2_create_buffers *b);
> > > > > int (*vidioc_prepare_buf)(struct file *file, void *fh,
> > > > > struct v4l2_buffer *b);
> > > > > + int (*vidioc_delete_buf)(struct file *file, void *fh,
> > > > > + struct v4l2_buffer *b);
> > > > >
> > > > > int (*vidioc_overlay)(struct file *file, void *fh, unsigned
> > > > > int i);
> > > > > int (*vidioc_g_fbuf)(struct file *file, void *fh,
> > > > > diff --git a/include/media/videobuf2-core.h
> > > > > b/include/media/videobuf2-core.h
> > > > > index 080b783d608d..0f9e68f76b77 100644
> > > > > --- a/include/media/videobuf2-core.h
> > > > > +++ b/include/media/videobuf2-core.h
> > > > > @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
> > > > > enum vb2_memory memory,
> > > > > */
> > > > > int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> > > > > void *pb);
> > > > >
> > > > > +/**
> > > > > + * vb2_core_delete_buf() -
> > > > > + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> > > > > + * @index: id number of the buffer.
> > > > > + *
> > > > > + * Return: returns zero on success; an error code otherwise.
> > > > > + */
> > > > > +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> > > > > +
> > > > > /**
> > > > > * vb2_core_qbuf() - Queue a buffer from userspace
> > > > > *
> > > > > diff --git a/include/media/videobuf2-v4l2.h
> > > > > b/include/media/videobuf2-v4l2.h
> > > > > index 88a7a565170e..3beeb4c735f0 100644
> > > > > --- a/include/media/videobuf2-v4l2.h
> > > > > +++ b/include/media/videobuf2-v4l2.h
> > > > > @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
> > > > > v4l2_create_buffers *create);
> > > > > */
> > > > > int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> > > > > struct v4l2_buffer *b);
> > > > > +/**
> > > > > + * vb2_delete_buf() - Delete the buffer from the queue
> > > > > + *
> > > > > + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> > > > > + * @b: buffer structure passed from userspace to
> > > > > + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> > > > > + *
> > > > > + * The return values from this function are intended to be directly
> > > > > returned
> > > > > + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> > > > > + */
> > > > > +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
> > > > >
> > > > > /**
> > > > > * vb2_qbuf() - Queue a buffer from userspace
> > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > b/include/uapi/linux/videodev2.h
> > > > > index aee75eb9e686..31bba1915642 100644
> > > > > --- a/include/uapi/linux/videodev2.h
> > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
> > > > > #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
> > > > > v4l2_dbg_chip_info)
> > > > >
> > > > > #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
> > > > > v4l2_query_ext_ctrl)
> > > > > +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
> > > > > +
> > > > >
> > > > > /* Reminder: when adding new ioctls please add support for them to
> > > > > drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> > > > > --
> > > > > 2.39.2
> > > > >
> > --
> > Hsia-Jun(Randy) Li
> >
> --
> Hsia-Jun(Randy) Li
>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-13 9:09 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-13 9:09 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
>
> On 6/27/23 16:47, Hsia-Jun Li wrote:
> > CAUTION: Email originated externally, do not click links or open
> > attachments unless you recognize the sender and know the content is
> > safe.
> >
> >
> > On 6/27/23 16:43, Benjamin Gaignard wrote:
> > > CAUTION: Email originated externally, do not click links or open
> > > attachments unless you recognize the sender and know the content is
> > > safe.
> > >
> > >
> > > Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
> > > >
> > > > On 6/22/23 21:13, Benjamin Gaignard wrote:
> > > > > CAUTION: Email originated externally, do not click links or open
> > > > > attachments unless you recognize the sender and know the content is
> > > > > safe.
> > > > >
> > > > >
> > > > > VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
> > > > >
> > > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > > > ---
> > > > > .../userspace-api/media/v4l/user-func.rst | 1 +
> > > > > .../media/v4l/vidioc-delete-buf.rst | 51
> > > > > +++++++++++++++++++
> > > > > .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> > > > > .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> > > > > drivers/media/v4l2-core/v4l2-dev.c | 1 +
> > > > > drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> > > > > include/media/v4l2-ioctl.h | 4 ++
> > > > > include/media/videobuf2-core.h | 9 ++++
> > > > > include/media/videobuf2-v4l2.h | 11 ++++
> > > > > include/uapi/linux/videodev2.h | 2 +
> > > > > 10 files changed, 128 insertions(+)
> > > > > create mode 100644
> > > > > Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > >
> > > > > diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > b/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > index 15ff0bf7bbe6..8c74016e12fd 100644
> > > > > --- a/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > @@ -17,6 +17,7 @@ Function Reference
> > > > > vidioc-dbg-g-chip-info
> > > > > vidioc-dbg-g-register
> > > > > vidioc-decoder-cmd
> > > > > + vidioc-delete-buf
> > > > > vidioc-dqevent
> > > > > vidioc-dv-timings-cap
> > > > > vidioc-encoder-cmd
> > > > > diff --git
> > > > > a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > new file mode 100644
> > > > > index 000000000000..0e7ce58f91bc
> > > > > --- /dev/null
> > > > > +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > @@ -0,0 +1,51 @@
> > > > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > > > > +.. c:namespace:: V4L
> > > > > +
> > > > > +.. _VIDIOC_DELETE_BUF:
> > > > > +
> > > > > +************************
> > > > > +ioctl VIDIOC_DELETE_BUF
> > > > > +************************
> > > > > +
> > > > > +Name
> > > > > +====
> > > > > +
> > > > > +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> > > > > +
> > > > > +Synopsis
> > > > > +========
> > > > > +
> > > > > +.. c:macro:: VIDIOC_DELETE_BUF
> > > > > +
> > > > > +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> > > > > +
> > > > > +Arguments
> > > > > +=========
> > > > > +
> > > > > +``fd``
> > > > > + File descriptor returned by :c:func:`open()`.
> > > > > +
> > > > > +``argp``
> > > > > + Pointer to struct :c:type:`v4l2_buffer`.
> > > > > +
> > > > > +Description
> > > > > +===========
> > > > > +
> > > > > +Applications can optionally call the
> > > > > :ref:`VIDIOC_DELETE_BUF` ioctl to
> > > > > +delete a buffer from a queue.
> > > > > +
> > > > > +The struct :c:type:`v4l2_buffer` structure is specified in
> > > > > +:ref:`buffer`.
> > > > > +
> > > > > +Return Value
> > > > > +============
> > > > > +
> > > > > +On success 0 is returned, on error -1 and the ``errno`` variable is
> > > > > set
> > > > > +appropriately. The generic error codes are described at the
> > > > > +:ref:`Generic Error Codes <gen-errors>` chapter.
> > > > > +
> > > > > +EBUSY
> > > > > + File I/O is in progress.
> > > > > +
> > > > > +EINVAL
> > > > > + The buffer ``index`` doesn't exist in the queue.
> > > > > diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > index 899783f67580..aa546c972c3d 100644
> > > > > --- a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
> > > > > unsigned int index, void *pb)
> > > > > }
> > > > > EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
> > > > >
> > > > > +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> > > > > +{
> > > > > + struct vb2_buffer *vb;
> > > > > +
> > > > > + vb = vb2_get_buffer(q, index);
> > > > > + if (!vb) {
> > > > > + dprintk(q, 1, "invalid buffer index %d\n", index);
> > > > > + return -EINVAL;
> > > > > + }
> > > > > +
> > > > > + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> > > > > + dprintk(q, 1, "can't delete non dequeued buffer index
> > > > > %d\n", index);
> > > > > + return -EINVAL;
> > > > > + }
> > > > > +
> > > > I know the driver could implement its own
> > > > v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
> > > > used by the hardware as a future reference frame.
> > > > But I think we need a flag to let the user know which buffer is still
> > > > used by the hardware.
> > > > Alternative ref case is safe, we only know it's existing when it is
> > > > dequeued in current V4L2 buffer mechanism.
> > > > While the Golden reference frame, such long term reference frame could
> > > > last much longer.
> > >
> > > It is up to userland stack to know frames life time, it got the
> > > information for that.
> >
> > That is true for the stateless codec driver.
> >
> > While application for stateful decoder could never do that. It also
> > breaks what the document said:
> >
> > "The backing memory of |CAPTURE| buffers that are used as reference
> > frames by the stream may be read by the hardware even after they are
> > dequeued. Consequently, the client should avoid writing into this memory
> > while the |CAPTURE| queue is streaming. Failure to observe this may
> > result in corruption of decoded frames."
> >
> > >
> > > > > + if (vb->planes[0].mem_priv)
> > > > > + call_void_vb_qop(vb, buf_cleanup, vb);
> > > > > +
> > > > > + /* Free MMAP buffers or release USERPTR buffers */
> > > > > + if (q->memory == VB2_MEMORY_MMAP)
> > > > > + __vb2_buf_mem_free(vb);
>
> Here is another problem for the existing application, the mmap() from the
> mmap offset or exportbuffer fd would not create a reference to buffer in
> this step(while the exportbuffer would create one itself).
>
> When you delete a buffer, you may not release it from its virtual memory
> space, leaving a corrupted virtual memory space.
What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
memory, it just decrements a reference counter. The VMA holds its own,
so the buffer is only fully released when the application calls
munmap().
Best regards,
Tomasz
> Also this behavior is
> right, because mmap(2) says:
>
> "After the mmap() call has returned, the file descriptor, fd, can be
> closed immediately without invalidating the map‐ping."
>
> > > > > + else if (q->memory == VB2_MEMORY_DMABUF)
> > > > > + __vb2_buf_dmabuf_put(vb);
> > > > > + else
> > > > > + __vb2_buf_userptr_put(vb);
> > > > > +
> > > > > + vb2_queue_remove_buffer(q, vb);
> > > > > + kfree(vb);
> > > > > +
> > > > > + dprintk(q, 2, "buffer %d deleted\n", index);
> > > > > + return 0;
> > > > > +}
> > > > > +
> > > > > /*
> > > > > * vb2_start_streaming() - Attempt to start streaming.
> > > > > * @q: videobuf2 queue
> > > > > diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > index 724135d41f7f..cea666c17b41 100644
> > > > > --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
> > > > > media_device *mdev,
> > > > > }
> > > > > EXPORT_SYMBOL_GPL(vb2_prepare_buf);
> > > > >
> > > > > +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> > > > > +{
> > > > > + return vb2_core_delete_buf(q, b->index);
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> > > > > +
> > > > > int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
> > > > > *create)
> > > > > {
> > > > > unsigned requested_planes = 1;
> > > > > diff --git a/drivers/media/v4l2-core/v4l2-dev.c
> > > > > b/drivers/media/v4l2-core/v4l2-dev.c
> > > > > index f81279492682..80ace2e1e932 100644
> > > > > --- a/drivers/media/v4l2-core/v4l2-dev.c
> > > > > +++ b/drivers/media/v4l2-core/v4l2-dev.c
> > > > > @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
> > > > > video_device *vdev)
> > > > > SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
> > > > > vidioc_prepare_buf);
> > > > > SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
> > > > > vidioc_streamon);
> > > > > SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
> > > > > vidioc_streamoff);
> > > > > + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
> > > > > vidioc_delete_buf);
> > > > > }
> > > > >
> > > > > if (is_vid || is_vbi || is_meta) {
> > > > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > index a858acea6547..1c737279d3ef 100644
> > > > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
> > > > > v4l2_ioctl_ops *ops,
> > > > > return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
> > > > > }
> > > > >
> > > > > +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> > > > > + struct file *file, void *fh, void *arg)
> > > > > +{
> > > > > + struct v4l2_buffer *b = arg;
> > > > > + int ret = check_fmt(file, b->type);
> > > > > +
> > > > > + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> > > > > +}
> > > > > +
> > > > > static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> > > > > struct file *file, void *fh, void
> > > > > *arg)
> > > > > {
> > > > > @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
> > > > > v4l2_ioctls[] = {
> > > > > IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
> > > > > v4l_print_freq_band, 0),
> > > > > IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
> > > > > v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> > > > > IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
> > > > > v4l_print_query_ext_ctrl, INFO_FL_CTRL |
> > > > > INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> > > > > + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
> > > > > v4l_print_buffer, INFO_FL_QUEUE),
> > > > > };
> > > > > #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
> > > > >
> > > > > diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> > > > > index edb733f21604..2f232ed884c7 100644
> > > > > --- a/include/media/v4l2-ioctl.h
> > > > > +++ b/include/media/v4l2-ioctl.h
> > > > > @@ -163,6 +163,8 @@ struct v4l2_fh;
> > > > > * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> > > > > * @vidioc_prepare_buf: pointer to the function that implements
> > > > > * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> > > > > + * @vidioc_delete_buf: pointer to the function that implements
> > > > > + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> > > > > * @vidioc_overlay: pointer to the function that implements
> > > > > * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> > > > > * @vidioc_g_fbuf: pointer to the function that implements
> > > > > @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
> > > > > struct v4l2_create_buffers *b);
> > > > > int (*vidioc_prepare_buf)(struct file *file, void *fh,
> > > > > struct v4l2_buffer *b);
> > > > > + int (*vidioc_delete_buf)(struct file *file, void *fh,
> > > > > + struct v4l2_buffer *b);
> > > > >
> > > > > int (*vidioc_overlay)(struct file *file, void *fh, unsigned
> > > > > int i);
> > > > > int (*vidioc_g_fbuf)(struct file *file, void *fh,
> > > > > diff --git a/include/media/videobuf2-core.h
> > > > > b/include/media/videobuf2-core.h
> > > > > index 080b783d608d..0f9e68f76b77 100644
> > > > > --- a/include/media/videobuf2-core.h
> > > > > +++ b/include/media/videobuf2-core.h
> > > > > @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
> > > > > enum vb2_memory memory,
> > > > > */
> > > > > int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> > > > > void *pb);
> > > > >
> > > > > +/**
> > > > > + * vb2_core_delete_buf() -
> > > > > + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> > > > > + * @index: id number of the buffer.
> > > > > + *
> > > > > + * Return: returns zero on success; an error code otherwise.
> > > > > + */
> > > > > +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> > > > > +
> > > > > /**
> > > > > * vb2_core_qbuf() - Queue a buffer from userspace
> > > > > *
> > > > > diff --git a/include/media/videobuf2-v4l2.h
> > > > > b/include/media/videobuf2-v4l2.h
> > > > > index 88a7a565170e..3beeb4c735f0 100644
> > > > > --- a/include/media/videobuf2-v4l2.h
> > > > > +++ b/include/media/videobuf2-v4l2.h
> > > > > @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
> > > > > v4l2_create_buffers *create);
> > > > > */
> > > > > int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> > > > > struct v4l2_buffer *b);
> > > > > +/**
> > > > > + * vb2_delete_buf() - Delete the buffer from the queue
> > > > > + *
> > > > > + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> > > > > + * @b: buffer structure passed from userspace to
> > > > > + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> > > > > + *
> > > > > + * The return values from this function are intended to be directly
> > > > > returned
> > > > > + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> > > > > + */
> > > > > +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
> > > > >
> > > > > /**
> > > > > * vb2_qbuf() - Queue a buffer from userspace
> > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > b/include/uapi/linux/videodev2.h
> > > > > index aee75eb9e686..31bba1915642 100644
> > > > > --- a/include/uapi/linux/videodev2.h
> > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
> > > > > #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
> > > > > v4l2_dbg_chip_info)
> > > > >
> > > > > #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
> > > > > v4l2_query_ext_ctrl)
> > > > > +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
> > > > > +
> > > > >
> > > > > /* Reminder: when adding new ioctls please add support for them to
> > > > > drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> > > > > --
> > > > > 2.39.2
> > > > >
> > --
> > Hsia-Jun(Randy) Li
> >
> --
> Hsia-Jun(Randy) Li
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-13 9:09 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-13 9:09 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
>
> On 6/27/23 16:47, Hsia-Jun Li wrote:
> > CAUTION: Email originated externally, do not click links or open
> > attachments unless you recognize the sender and know the content is
> > safe.
> >
> >
> > On 6/27/23 16:43, Benjamin Gaignard wrote:
> > > CAUTION: Email originated externally, do not click links or open
> > > attachments unless you recognize the sender and know the content is
> > > safe.
> > >
> > >
> > > Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
> > > >
> > > > On 6/22/23 21:13, Benjamin Gaignard wrote:
> > > > > CAUTION: Email originated externally, do not click links or open
> > > > > attachments unless you recognize the sender and know the content is
> > > > > safe.
> > > > >
> > > > >
> > > > > VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
> > > > >
> > > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > > > ---
> > > > > .../userspace-api/media/v4l/user-func.rst | 1 +
> > > > > .../media/v4l/vidioc-delete-buf.rst | 51
> > > > > +++++++++++++++++++
> > > > > .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> > > > > .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> > > > > drivers/media/v4l2-core/v4l2-dev.c | 1 +
> > > > > drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> > > > > include/media/v4l2-ioctl.h | 4 ++
> > > > > include/media/videobuf2-core.h | 9 ++++
> > > > > include/media/videobuf2-v4l2.h | 11 ++++
> > > > > include/uapi/linux/videodev2.h | 2 +
> > > > > 10 files changed, 128 insertions(+)
> > > > > create mode 100644
> > > > > Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > >
> > > > > diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > b/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > index 15ff0bf7bbe6..8c74016e12fd 100644
> > > > > --- a/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> > > > > @@ -17,6 +17,7 @@ Function Reference
> > > > > vidioc-dbg-g-chip-info
> > > > > vidioc-dbg-g-register
> > > > > vidioc-decoder-cmd
> > > > > + vidioc-delete-buf
> > > > > vidioc-dqevent
> > > > > vidioc-dv-timings-cap
> > > > > vidioc-encoder-cmd
> > > > > diff --git
> > > > > a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > new file mode 100644
> > > > > index 000000000000..0e7ce58f91bc
> > > > > --- /dev/null
> > > > > +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> > > > > @@ -0,0 +1,51 @@
> > > > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > > > > +.. c:namespace:: V4L
> > > > > +
> > > > > +.. _VIDIOC_DELETE_BUF:
> > > > > +
> > > > > +************************
> > > > > +ioctl VIDIOC_DELETE_BUF
> > > > > +************************
> > > > > +
> > > > > +Name
> > > > > +====
> > > > > +
> > > > > +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> > > > > +
> > > > > +Synopsis
> > > > > +========
> > > > > +
> > > > > +.. c:macro:: VIDIOC_DELETE_BUF
> > > > > +
> > > > > +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> > > > > +
> > > > > +Arguments
> > > > > +=========
> > > > > +
> > > > > +``fd``
> > > > > + File descriptor returned by :c:func:`open()`.
> > > > > +
> > > > > +``argp``
> > > > > + Pointer to struct :c:type:`v4l2_buffer`.
> > > > > +
> > > > > +Description
> > > > > +===========
> > > > > +
> > > > > +Applications can optionally call the
> > > > > :ref:`VIDIOC_DELETE_BUF` ioctl to
> > > > > +delete a buffer from a queue.
> > > > > +
> > > > > +The struct :c:type:`v4l2_buffer` structure is specified in
> > > > > +:ref:`buffer`.
> > > > > +
> > > > > +Return Value
> > > > > +============
> > > > > +
> > > > > +On success 0 is returned, on error -1 and the ``errno`` variable is
> > > > > set
> > > > > +appropriately. The generic error codes are described at the
> > > > > +:ref:`Generic Error Codes <gen-errors>` chapter.
> > > > > +
> > > > > +EBUSY
> > > > > + File I/O is in progress.
> > > > > +
> > > > > +EINVAL
> > > > > + The buffer ``index`` doesn't exist in the queue.
> > > > > diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > index 899783f67580..aa546c972c3d 100644
> > > > > --- a/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> > > > > @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
> > > > > unsigned int index, void *pb)
> > > > > }
> > > > > EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
> > > > >
> > > > > +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> > > > > +{
> > > > > + struct vb2_buffer *vb;
> > > > > +
> > > > > + vb = vb2_get_buffer(q, index);
> > > > > + if (!vb) {
> > > > > + dprintk(q, 1, "invalid buffer index %d\n", index);
> > > > > + return -EINVAL;
> > > > > + }
> > > > > +
> > > > > + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> > > > > + dprintk(q, 1, "can't delete non dequeued buffer index
> > > > > %d\n", index);
> > > > > + return -EINVAL;
> > > > > + }
> > > > > +
> > > > I know the driver could implement its own
> > > > v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
> > > > used by the hardware as a future reference frame.
> > > > But I think we need a flag to let the user know which buffer is still
> > > > used by the hardware.
> > > > Alternative ref case is safe, we only know it's existing when it is
> > > > dequeued in current V4L2 buffer mechanism.
> > > > While the Golden reference frame, such long term reference frame could
> > > > last much longer.
> > >
> > > It is up to userland stack to know frames life time, it got the
> > > information for that.
> >
> > That is true for the stateless codec driver.
> >
> > While application for stateful decoder could never do that. It also
> > breaks what the document said:
> >
> > "The backing memory of |CAPTURE| buffers that are used as reference
> > frames by the stream may be read by the hardware even after they are
> > dequeued. Consequently, the client should avoid writing into this memory
> > while the |CAPTURE| queue is streaming. Failure to observe this may
> > result in corruption of decoded frames."
> >
> > >
> > > > > + if (vb->planes[0].mem_priv)
> > > > > + call_void_vb_qop(vb, buf_cleanup, vb);
> > > > > +
> > > > > + /* Free MMAP buffers or release USERPTR buffers */
> > > > > + if (q->memory == VB2_MEMORY_MMAP)
> > > > > + __vb2_buf_mem_free(vb);
>
> Here is another problem for the existing application, the mmap() from the
> mmap offset or exportbuffer fd would not create a reference to buffer in
> this step(while the exportbuffer would create one itself).
>
> When you delete a buffer, you may not release it from its virtual memory
> space, leaving a corrupted virtual memory space.
What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
memory, it just decrements a reference counter. The VMA holds its own,
so the buffer is only fully released when the application calls
munmap().
Best regards,
Tomasz
> Also this behavior is
> right, because mmap(2) says:
>
> "After the mmap() call has returned, the file descriptor, fd, can be
> closed immediately without invalidating the map‐ping."
>
> > > > > + else if (q->memory == VB2_MEMORY_DMABUF)
> > > > > + __vb2_buf_dmabuf_put(vb);
> > > > > + else
> > > > > + __vb2_buf_userptr_put(vb);
> > > > > +
> > > > > + vb2_queue_remove_buffer(q, vb);
> > > > > + kfree(vb);
> > > > > +
> > > > > + dprintk(q, 2, "buffer %d deleted\n", index);
> > > > > + return 0;
> > > > > +}
> > > > > +
> > > > > /*
> > > > > * vb2_start_streaming() - Attempt to start streaming.
> > > > > * @q: videobuf2 queue
> > > > > diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > index 724135d41f7f..cea666c17b41 100644
> > > > > --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> > > > > @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
> > > > > media_device *mdev,
> > > > > }
> > > > > EXPORT_SYMBOL_GPL(vb2_prepare_buf);
> > > > >
> > > > > +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> > > > > +{
> > > > > + return vb2_core_delete_buf(q, b->index);
> > > > > +}
> > > > > +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> > > > > +
> > > > > int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
> > > > > *create)
> > > > > {
> > > > > unsigned requested_planes = 1;
> > > > > diff --git a/drivers/media/v4l2-core/v4l2-dev.c
> > > > > b/drivers/media/v4l2-core/v4l2-dev.c
> > > > > index f81279492682..80ace2e1e932 100644
> > > > > --- a/drivers/media/v4l2-core/v4l2-dev.c
> > > > > +++ b/drivers/media/v4l2-core/v4l2-dev.c
> > > > > @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
> > > > > video_device *vdev)
> > > > > SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
> > > > > vidioc_prepare_buf);
> > > > > SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
> > > > > vidioc_streamon);
> > > > > SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
> > > > > vidioc_streamoff);
> > > > > + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
> > > > > vidioc_delete_buf);
> > > > > }
> > > > >
> > > > > if (is_vid || is_vbi || is_meta) {
> > > > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > index a858acea6547..1c737279d3ef 100644
> > > > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > > @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
> > > > > v4l2_ioctl_ops *ops,
> > > > > return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
> > > > > }
> > > > >
> > > > > +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> > > > > + struct file *file, void *fh, void *arg)
> > > > > +{
> > > > > + struct v4l2_buffer *b = arg;
> > > > > + int ret = check_fmt(file, b->type);
> > > > > +
> > > > > + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> > > > > +}
> > > > > +
> > > > > static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> > > > > struct file *file, void *fh, void
> > > > > *arg)
> > > > > {
> > > > > @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
> > > > > v4l2_ioctls[] = {
> > > > > IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
> > > > > v4l_print_freq_band, 0),
> > > > > IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
> > > > > v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> > > > > IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
> > > > > v4l_print_query_ext_ctrl, INFO_FL_CTRL |
> > > > > INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> > > > > + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
> > > > > v4l_print_buffer, INFO_FL_QUEUE),
> > > > > };
> > > > > #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
> > > > >
> > > > > diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> > > > > index edb733f21604..2f232ed884c7 100644
> > > > > --- a/include/media/v4l2-ioctl.h
> > > > > +++ b/include/media/v4l2-ioctl.h
> > > > > @@ -163,6 +163,8 @@ struct v4l2_fh;
> > > > > * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> > > > > * @vidioc_prepare_buf: pointer to the function that implements
> > > > > * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> > > > > + * @vidioc_delete_buf: pointer to the function that implements
> > > > > + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> > > > > * @vidioc_overlay: pointer to the function that implements
> > > > > * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> > > > > * @vidioc_g_fbuf: pointer to the function that implements
> > > > > @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
> > > > > struct v4l2_create_buffers *b);
> > > > > int (*vidioc_prepare_buf)(struct file *file, void *fh,
> > > > > struct v4l2_buffer *b);
> > > > > + int (*vidioc_delete_buf)(struct file *file, void *fh,
> > > > > + struct v4l2_buffer *b);
> > > > >
> > > > > int (*vidioc_overlay)(struct file *file, void *fh, unsigned
> > > > > int i);
> > > > > int (*vidioc_g_fbuf)(struct file *file, void *fh,
> > > > > diff --git a/include/media/videobuf2-core.h
> > > > > b/include/media/videobuf2-core.h
> > > > > index 080b783d608d..0f9e68f76b77 100644
> > > > > --- a/include/media/videobuf2-core.h
> > > > > +++ b/include/media/videobuf2-core.h
> > > > > @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
> > > > > enum vb2_memory memory,
> > > > > */
> > > > > int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> > > > > void *pb);
> > > > >
> > > > > +/**
> > > > > + * vb2_core_delete_buf() -
> > > > > + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> > > > > + * @index: id number of the buffer.
> > > > > + *
> > > > > + * Return: returns zero on success; an error code otherwise.
> > > > > + */
> > > > > +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> > > > > +
> > > > > /**
> > > > > * vb2_core_qbuf() - Queue a buffer from userspace
> > > > > *
> > > > > diff --git a/include/media/videobuf2-v4l2.h
> > > > > b/include/media/videobuf2-v4l2.h
> > > > > index 88a7a565170e..3beeb4c735f0 100644
> > > > > --- a/include/media/videobuf2-v4l2.h
> > > > > +++ b/include/media/videobuf2-v4l2.h
> > > > > @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
> > > > > v4l2_create_buffers *create);
> > > > > */
> > > > > int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> > > > > struct v4l2_buffer *b);
> > > > > +/**
> > > > > + * vb2_delete_buf() - Delete the buffer from the queue
> > > > > + *
> > > > > + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> > > > > + * @b: buffer structure passed from userspace to
> > > > > + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> > > > > + *
> > > > > + * The return values from this function are intended to be directly
> > > > > returned
> > > > > + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> > > > > + */
> > > > > +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
> > > > >
> > > > > /**
> > > > > * vb2_qbuf() - Queue a buffer from userspace
> > > > > diff --git a/include/uapi/linux/videodev2.h
> > > > > b/include/uapi/linux/videodev2.h
> > > > > index aee75eb9e686..31bba1915642 100644
> > > > > --- a/include/uapi/linux/videodev2.h
> > > > > +++ b/include/uapi/linux/videodev2.h
> > > > > @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
> > > > > #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
> > > > > v4l2_dbg_chip_info)
> > > > >
> > > > > #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
> > > > > v4l2_query_ext_ctrl)
> > > > > +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
> > > > > +
> > > > >
> > > > > /* Reminder: when adding new ioctls please add support for them to
> > > > > drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> > > > > --
> > > > > 2.39.2
> > > > >
> > --
> > Hsia-Jun(Randy) Li
> >
> --
> Hsia-Jun(Randy) Li
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-07-13 9:09 ` Tomasz Figa
(?)
@ 2023-07-17 2:16 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-17 2:16 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On 7/13/23 17:09, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>
>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>> ---
>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>> +++++++++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>> 10 files changed, 128 insertions(+)
>>>>>> create mode 100644
>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>
>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>> vidioc-dbg-g-chip-info
>>>>>> vidioc-dbg-g-register
>>>>>> vidioc-decoder-cmd
>>>>>> + vidioc-delete-buf
>>>>>> vidioc-dqevent
>>>>>> vidioc-dv-timings-cap
>>>>>> vidioc-encoder-cmd
>>>>>> diff --git
>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> new file mode 100644
>>>>>> index 000000000000..0e7ce58f91bc
>>>>>> --- /dev/null
>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> @@ -0,0 +1,51 @@
>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>> +.. c:namespace:: V4L
>>>>>> +
>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>> +
>>>>>> +************************
>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>> +************************
>>>>>> +
>>>>>> +Name
>>>>>> +====
>>>>>> +
>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>> +
>>>>>> +Synopsis
>>>>>> +========
>>>>>> +
>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>> +
>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>> +
>>>>>> +Arguments
>>>>>> +=========
>>>>>> +
>>>>>> +``fd``
>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>> +
>>>>>> +``argp``
>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>> +
>>>>>> +Description
>>>>>> +===========
>>>>>> +
>>>>>> +Applications can optionally call the
>>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
>>>>>> +delete a buffer from a queue.
>>>>>> +
>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>> +:ref:`buffer`.
>>>>>> +
>>>>>> +Return Value
>>>>>> +============
>>>>>> +
>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>> set
>>>>>> +appropriately. The generic error codes are described at the
>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>> +
>>>>>> +EBUSY
>>>>>> + File I/O is in progress.
>>>>>> +
>>>>>> +EINVAL
>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>> unsigned int index, void *pb)
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>> +{
>>>>>> + struct vb2_buffer *vb;
>>>>>> +
>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>> + if (!vb) {
>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>>>> %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>> I know the driver could implement its own
>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>> used by the hardware as a future reference frame.
>>>>> But I think we need a flag to let the user know which buffer is still
>>>>> used by the hardware.
>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>> dequeued in current V4L2 buffer mechanism.
>>>>> While the Golden reference frame, such long term reference frame could
>>>>> last much longer.
>>>> It is up to userland stack to know frames life time, it got the
>>>> information for that.
>>> That is true for the stateless codec driver.
>>>
>>> While application for stateful decoder could never do that. It also
>>> breaks what the document said:
>>>
>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>> frames by the stream may be read by the hardware even after they are
>>> dequeued. Consequently, the client should avoid writing into this memory
>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>> result in corruption of decoded frames."
>>>
>>>>>> + if (vb->planes[0].mem_priv)
>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>> +
>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>> + __vb2_buf_mem_free(vb);
>> Here is another problem for the existing application, the mmap() from the
>> mmap offset or exportbuffer fd would not create a reference to buffer in
>> this step(while the exportbuffer would create one itself).
>>
>> When you delete a buffer, you may not release it from its virtual memory
>> space, leaving a corrupted virtual memory space.
> What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
> memory, it just decrements a reference counter.
struct dma_buf_ops->mmap() may not increase a reference to its buffer.
While struct vb2_mem_ops->get_dmabuf() would.
> The VMA holds its own,
> so the buffer is only fully released when the application calls
> munmap().
DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF
ioctl() make the no other user could access to this.
>
> Best regards,
> Tomasz
>
>> Also this behavior is
>> right, because mmap(2) says:
>>
>> "After the mmap() call has returned, the file descriptor, fd, can be
>> closed immediately without invalidating the map‐ping."
>>
>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>> + else
>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>> +
>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>> + kfree(vb);
>>>>>> +
>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>> + return 0;
>>>>>> +}
>>>>>> +
>>>>>> /*
>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>> * @q: videobuf2 queue
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>> media_device *mdev,
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>> +{
>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>> +}
>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>> +
>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>>> *create)
>>>>>> {
>>>>>> unsigned requested_planes = 1;
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>> video_device *vdev)
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>> vidioc_prepare_buf);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>> vidioc_streamon);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>> vidioc_streamoff);
>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>> vidioc_delete_buf);
>>>>>> }
>>>>>>
>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>> v4l2_ioctl_ops *ops,
>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>> }
>>>>>>
>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>> + struct file *file, void *fh, void *arg)
>>>>>> +{
>>>>>> + struct v4l2_buffer *b = arg;
>>>>>> + int ret = check_fmt(file, b->type);
>>>>>> +
>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>> +}
>>>>>> +
>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>> struct file *file, void *fh, void
>>>>>> *arg)
>>>>>> {
>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>> v4l2_ioctls[] = {
>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>> v4l_print_freq_band, 0),
>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>> };
>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>
>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>> struct v4l2_create_buffers *b);
>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>> struct v4l2_buffer *b);
>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>> + struct v4l2_buffer *b);
>>>>>>
>>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>> int i);
>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>> b/include/media/videobuf2-core.h
>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>> --- a/include/media/videobuf2-core.h
>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>> enum vb2_memory memory,
>>>>>> */
>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>> void *pb);
>>>>>>
>>>>>> +/**
>>>>>> + * vb2_core_delete_buf() -
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @index: id number of the buffer.
>>>>>> + *
>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>> + */
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>> +
>>>>>> /**
>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>> *
>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>> v4l2_create_buffers *create);
>>>>>> */
>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>>> struct v4l2_buffer *b);
>>>>>> +/**
>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>> + *
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @b: buffer structure passed from userspace to
>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>> + *
>>>>>> + * The return values from this function are intended to be directly
>>>>>> returned
>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>> + */
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>
>>>>>> /**
>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>> b/include/uapi/linux/videodev2.h
>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>> v4l2_dbg_chip_info)
>>>>>>
>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>> v4l2_query_ext_ctrl)
>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>> +
>>>>>>
>>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>> --
>>>>>> 2.39.2
>>>>>>
>>> --
>>> Hsia-Jun(Randy) Li
>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-17 2:16 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-17 2:16 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On 7/13/23 17:09, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>
>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>> ---
>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>> +++++++++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>> 10 files changed, 128 insertions(+)
>>>>>> create mode 100644
>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>
>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>> vidioc-dbg-g-chip-info
>>>>>> vidioc-dbg-g-register
>>>>>> vidioc-decoder-cmd
>>>>>> + vidioc-delete-buf
>>>>>> vidioc-dqevent
>>>>>> vidioc-dv-timings-cap
>>>>>> vidioc-encoder-cmd
>>>>>> diff --git
>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> new file mode 100644
>>>>>> index 000000000000..0e7ce58f91bc
>>>>>> --- /dev/null
>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> @@ -0,0 +1,51 @@
>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>> +.. c:namespace:: V4L
>>>>>> +
>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>> +
>>>>>> +************************
>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>> +************************
>>>>>> +
>>>>>> +Name
>>>>>> +====
>>>>>> +
>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>> +
>>>>>> +Synopsis
>>>>>> +========
>>>>>> +
>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>> +
>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>> +
>>>>>> +Arguments
>>>>>> +=========
>>>>>> +
>>>>>> +``fd``
>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>> +
>>>>>> +``argp``
>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>> +
>>>>>> +Description
>>>>>> +===========
>>>>>> +
>>>>>> +Applications can optionally call the
>>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
>>>>>> +delete a buffer from a queue.
>>>>>> +
>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>> +:ref:`buffer`.
>>>>>> +
>>>>>> +Return Value
>>>>>> +============
>>>>>> +
>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>> set
>>>>>> +appropriately. The generic error codes are described at the
>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>> +
>>>>>> +EBUSY
>>>>>> + File I/O is in progress.
>>>>>> +
>>>>>> +EINVAL
>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>> unsigned int index, void *pb)
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>> +{
>>>>>> + struct vb2_buffer *vb;
>>>>>> +
>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>> + if (!vb) {
>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>>>> %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>> I know the driver could implement its own
>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>> used by the hardware as a future reference frame.
>>>>> But I think we need a flag to let the user know which buffer is still
>>>>> used by the hardware.
>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>> dequeued in current V4L2 buffer mechanism.
>>>>> While the Golden reference frame, such long term reference frame could
>>>>> last much longer.
>>>> It is up to userland stack to know frames life time, it got the
>>>> information for that.
>>> That is true for the stateless codec driver.
>>>
>>> While application for stateful decoder could never do that. It also
>>> breaks what the document said:
>>>
>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>> frames by the stream may be read by the hardware even after they are
>>> dequeued. Consequently, the client should avoid writing into this memory
>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>> result in corruption of decoded frames."
>>>
>>>>>> + if (vb->planes[0].mem_priv)
>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>> +
>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>> + __vb2_buf_mem_free(vb);
>> Here is another problem for the existing application, the mmap() from the
>> mmap offset or exportbuffer fd would not create a reference to buffer in
>> this step(while the exportbuffer would create one itself).
>>
>> When you delete a buffer, you may not release it from its virtual memory
>> space, leaving a corrupted virtual memory space.
> What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
> memory, it just decrements a reference counter.
struct dma_buf_ops->mmap() may not increase a reference to its buffer.
While struct vb2_mem_ops->get_dmabuf() would.
> The VMA holds its own,
> so the buffer is only fully released when the application calls
> munmap().
DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF
ioctl() make the no other user could access to this.
>
> Best regards,
> Tomasz
>
>> Also this behavior is
>> right, because mmap(2) says:
>>
>> "After the mmap() call has returned, the file descriptor, fd, can be
>> closed immediately without invalidating the map‐ping."
>>
>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>> + else
>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>> +
>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>> + kfree(vb);
>>>>>> +
>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>> + return 0;
>>>>>> +}
>>>>>> +
>>>>>> /*
>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>> * @q: videobuf2 queue
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>> media_device *mdev,
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>> +{
>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>> +}
>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>> +
>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>>> *create)
>>>>>> {
>>>>>> unsigned requested_planes = 1;
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>> video_device *vdev)
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>> vidioc_prepare_buf);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>> vidioc_streamon);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>> vidioc_streamoff);
>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>> vidioc_delete_buf);
>>>>>> }
>>>>>>
>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>> v4l2_ioctl_ops *ops,
>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>> }
>>>>>>
>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>> + struct file *file, void *fh, void *arg)
>>>>>> +{
>>>>>> + struct v4l2_buffer *b = arg;
>>>>>> + int ret = check_fmt(file, b->type);
>>>>>> +
>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>> +}
>>>>>> +
>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>> struct file *file, void *fh, void
>>>>>> *arg)
>>>>>> {
>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>> v4l2_ioctls[] = {
>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>> v4l_print_freq_band, 0),
>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>> };
>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>
>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>> struct v4l2_create_buffers *b);
>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>> struct v4l2_buffer *b);
>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>> + struct v4l2_buffer *b);
>>>>>>
>>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>> int i);
>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>> b/include/media/videobuf2-core.h
>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>> --- a/include/media/videobuf2-core.h
>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>> enum vb2_memory memory,
>>>>>> */
>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>> void *pb);
>>>>>>
>>>>>> +/**
>>>>>> + * vb2_core_delete_buf() -
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @index: id number of the buffer.
>>>>>> + *
>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>> + */
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>> +
>>>>>> /**
>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>> *
>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>> v4l2_create_buffers *create);
>>>>>> */
>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>>> struct v4l2_buffer *b);
>>>>>> +/**
>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>> + *
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @b: buffer structure passed from userspace to
>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>> + *
>>>>>> + * The return values from this function are intended to be directly
>>>>>> returned
>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>> + */
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>
>>>>>> /**
>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>> b/include/uapi/linux/videodev2.h
>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>> v4l2_dbg_chip_info)
>>>>>>
>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>> v4l2_query_ext_ctrl)
>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>> +
>>>>>>
>>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>> --
>>>>>> 2.39.2
>>>>>>
>>> --
>>> Hsia-Jun(Randy) Li
>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-17 2:16 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-17 2:16 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On 7/13/23 17:09, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>> CAUTION: Email originated externally, do not click links or open
>>> attachments unless you recognize the sender and know the content is
>>> safe.
>>>
>>>
>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>> CAUTION: Email originated externally, do not click links or open
>>>> attachments unless you recognize the sender and know the content is
>>>> safe.
>>>>
>>>>
>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>
>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>> ---
>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>> +++++++++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>> 10 files changed, 128 insertions(+)
>>>>>> create mode 100644
>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>
>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>> vidioc-dbg-g-chip-info
>>>>>> vidioc-dbg-g-register
>>>>>> vidioc-decoder-cmd
>>>>>> + vidioc-delete-buf
>>>>>> vidioc-dqevent
>>>>>> vidioc-dv-timings-cap
>>>>>> vidioc-encoder-cmd
>>>>>> diff --git
>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> new file mode 100644
>>>>>> index 000000000000..0e7ce58f91bc
>>>>>> --- /dev/null
>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>> @@ -0,0 +1,51 @@
>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>> +.. c:namespace:: V4L
>>>>>> +
>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>> +
>>>>>> +************************
>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>> +************************
>>>>>> +
>>>>>> +Name
>>>>>> +====
>>>>>> +
>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>> +
>>>>>> +Synopsis
>>>>>> +========
>>>>>> +
>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>> +
>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>> +
>>>>>> +Arguments
>>>>>> +=========
>>>>>> +
>>>>>> +``fd``
>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>> +
>>>>>> +``argp``
>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>> +
>>>>>> +Description
>>>>>> +===========
>>>>>> +
>>>>>> +Applications can optionally call the
>>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
>>>>>> +delete a buffer from a queue.
>>>>>> +
>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>> +:ref:`buffer`.
>>>>>> +
>>>>>> +Return Value
>>>>>> +============
>>>>>> +
>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>> set
>>>>>> +appropriately. The generic error codes are described at the
>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>> +
>>>>>> +EBUSY
>>>>>> + File I/O is in progress.
>>>>>> +
>>>>>> +EINVAL
>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>> unsigned int index, void *pb)
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>> +{
>>>>>> + struct vb2_buffer *vb;
>>>>>> +
>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>> + if (!vb) {
>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>>>> %d\n", index);
>>>>>> + return -EINVAL;
>>>>>> + }
>>>>>> +
>>>>> I know the driver could implement its own
>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>> used by the hardware as a future reference frame.
>>>>> But I think we need a flag to let the user know which buffer is still
>>>>> used by the hardware.
>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>> dequeued in current V4L2 buffer mechanism.
>>>>> While the Golden reference frame, such long term reference frame could
>>>>> last much longer.
>>>> It is up to userland stack to know frames life time, it got the
>>>> information for that.
>>> That is true for the stateless codec driver.
>>>
>>> While application for stateful decoder could never do that. It also
>>> breaks what the document said:
>>>
>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>> frames by the stream may be read by the hardware even after they are
>>> dequeued. Consequently, the client should avoid writing into this memory
>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>> result in corruption of decoded frames."
>>>
>>>>>> + if (vb->planes[0].mem_priv)
>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>> +
>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>> + __vb2_buf_mem_free(vb);
>> Here is another problem for the existing application, the mmap() from the
>> mmap offset or exportbuffer fd would not create a reference to buffer in
>> this step(while the exportbuffer would create one itself).
>>
>> When you delete a buffer, you may not release it from its virtual memory
>> space, leaving a corrupted virtual memory space.
> What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
> memory, it just decrements a reference counter.
struct dma_buf_ops->mmap() may not increase a reference to its buffer.
While struct vb2_mem_ops->get_dmabuf() would.
> The VMA holds its own,
> so the buffer is only fully released when the application calls
> munmap().
DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF
ioctl() make the no other user could access to this.
>
> Best regards,
> Tomasz
>
>> Also this behavior is
>> right, because mmap(2) says:
>>
>> "After the mmap() call has returned, the file descriptor, fd, can be
>> closed immediately without invalidating the map‐ping."
>>
>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>> + else
>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>> +
>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>> + kfree(vb);
>>>>>> +
>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>> + return 0;
>>>>>> +}
>>>>>> +
>>>>>> /*
>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>> * @q: videobuf2 queue
>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>> media_device *mdev,
>>>>>> }
>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>> +{
>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>> +}
>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>> +
>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>>> *create)
>>>>>> {
>>>>>> unsigned requested_planes = 1;
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>> video_device *vdev)
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>> vidioc_prepare_buf);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>> vidioc_streamon);
>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>> vidioc_streamoff);
>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>> vidioc_delete_buf);
>>>>>> }
>>>>>>
>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>> v4l2_ioctl_ops *ops,
>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>> }
>>>>>>
>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>> + struct file *file, void *fh, void *arg)
>>>>>> +{
>>>>>> + struct v4l2_buffer *b = arg;
>>>>>> + int ret = check_fmt(file, b->type);
>>>>>> +
>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>> +}
>>>>>> +
>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>> struct file *file, void *fh, void
>>>>>> *arg)
>>>>>> {
>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>> v4l2_ioctls[] = {
>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>> v4l_print_freq_band, 0),
>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>> };
>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>
>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>> struct v4l2_create_buffers *b);
>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>> struct v4l2_buffer *b);
>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>> + struct v4l2_buffer *b);
>>>>>>
>>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>> int i);
>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>> b/include/media/videobuf2-core.h
>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>> --- a/include/media/videobuf2-core.h
>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>> enum vb2_memory memory,
>>>>>> */
>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>> void *pb);
>>>>>>
>>>>>> +/**
>>>>>> + * vb2_core_delete_buf() -
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @index: id number of the buffer.
>>>>>> + *
>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>> + */
>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>> +
>>>>>> /**
>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>> *
>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>> v4l2_create_buffers *create);
>>>>>> */
>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>>> struct v4l2_buffer *b);
>>>>>> +/**
>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>> + *
>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>> + * @b: buffer structure passed from userspace to
>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>> + *
>>>>>> + * The return values from this function are intended to be directly
>>>>>> returned
>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>> + */
>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>
>>>>>> /**
>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>> b/include/uapi/linux/videodev2.h
>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>> v4l2_dbg_chip_info)
>>>>>>
>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>> v4l2_query_ext_ctrl)
>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>> +
>>>>>>
>>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>> --
>>>>>> 2.39.2
>>>>>>
>>> --
>>> Hsia-Jun(Randy) Li
>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-07-17 2:16 ` Hsia-Jun Li
(?)
@ 2023-07-28 6:57 ` Tomasz Figa
-1 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-28 6:57 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On Mon, Jul 17, 2023 at 11:17 AM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
> On 7/13/23 17:09, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
> >> On 6/27/23 16:47, Hsia-Jun Li wrote:
> >>> CAUTION: Email originated externally, do not click links or open
> >>> attachments unless you recognize the sender and know the content is
> >>> safe.
> >>>
> >>>
> >>> On 6/27/23 16:43, Benjamin Gaignard wrote:
> >>>> CAUTION: Email originated externally, do not click links or open
> >>>> attachments unless you recognize the sender and know the content is
> >>>> safe.
> >>>>
> >>>>
> >>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
> >>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>> attachments unless you recognize the sender and know the content is
> >>>>>> safe.
> >>>>>>
> >>>>>>
> >>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
> >>>>>>
> >>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>>> ---
> >>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
> >>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
> >>>>>> +++++++++++++++++++
> >>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> >>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> >>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> >>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> >>>>>> include/media/v4l2-ioctl.h | 4 ++
> >>>>>> include/media/videobuf2-core.h | 9 ++++
> >>>>>> include/media/videobuf2-v4l2.h | 11 ++++
> >>>>>> include/uapi/linux/videodev2.h | 2 +
> >>>>>> 10 files changed, 128 insertions(+)
> >>>>>> create mode 100644
> >>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>>
> >>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
> >>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> @@ -17,6 +17,7 @@ Function Reference
> >>>>>> vidioc-dbg-g-chip-info
> >>>>>> vidioc-dbg-g-register
> >>>>>> vidioc-decoder-cmd
> >>>>>> + vidioc-delete-buf
> >>>>>> vidioc-dqevent
> >>>>>> vidioc-dv-timings-cap
> >>>>>> vidioc-encoder-cmd
> >>>>>> diff --git
> >>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> new file mode 100644
> >>>>>> index 000000000000..0e7ce58f91bc
> >>>>>> --- /dev/null
> >>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> @@ -0,0 +1,51 @@
> >>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> >>>>>> +.. c:namespace:: V4L
> >>>>>> +
> >>>>>> +.. _VIDIOC_DELETE_BUF:
> >>>>>> +
> >>>>>> +************************
> >>>>>> +ioctl VIDIOC_DELETE_BUF
> >>>>>> +************************
> >>>>>> +
> >>>>>> +Name
> >>>>>> +====
> >>>>>> +
> >>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> >>>>>> +
> >>>>>> +Synopsis
> >>>>>> +========
> >>>>>> +
> >>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
> >>>>>> +
> >>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> >>>>>> +
> >>>>>> +Arguments
> >>>>>> +=========
> >>>>>> +
> >>>>>> +``fd``
> >>>>>> + File descriptor returned by :c:func:`open()`.
> >>>>>> +
> >>>>>> +``argp``
> >>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
> >>>>>> +
> >>>>>> +Description
> >>>>>> +===========
> >>>>>> +
> >>>>>> +Applications can optionally call the
> >>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
> >>>>>> +delete a buffer from a queue.
> >>>>>> +
> >>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
> >>>>>> +:ref:`buffer`.
> >>>>>> +
> >>>>>> +Return Value
> >>>>>> +============
> >>>>>> +
> >>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
> >>>>>> set
> >>>>>> +appropriately. The generic error codes are described at the
> >>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
> >>>>>> +
> >>>>>> +EBUSY
> >>>>>> + File I/O is in progress.
> >>>>>> +
> >>>>>> +EINVAL
> >>>>>> + The buffer ``index`` doesn't exist in the queue.
> >>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> index 899783f67580..aa546c972c3d 100644
> >>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
> >>>>>> unsigned int index, void *pb)
> >>>>>> }
> >>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
> >>>>>>
> >>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> >>>>>> +{
> >>>>>> + struct vb2_buffer *vb;
> >>>>>> +
> >>>>>> + vb = vb2_get_buffer(q, index);
> >>>>>> + if (!vb) {
> >>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
> >>>>>> + return -EINVAL;
> >>>>>> + }
> >>>>>> +
> >>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> >>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
> >>>>>> %d\n", index);
> >>>>>> + return -EINVAL;
> >>>>>> + }
> >>>>>> +
> >>>>> I know the driver could implement its own
> >>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
> >>>>> used by the hardware as a future reference frame.
> >>>>> But I think we need a flag to let the user know which buffer is still
> >>>>> used by the hardware.
> >>>>> Alternative ref case is safe, we only know it's existing when it is
> >>>>> dequeued in current V4L2 buffer mechanism.
> >>>>> While the Golden reference frame, such long term reference frame could
> >>>>> last much longer.
> >>>> It is up to userland stack to know frames life time, it got the
> >>>> information for that.
> >>> That is true for the stateless codec driver.
> >>>
> >>> While application for stateful decoder could never do that. It also
> >>> breaks what the document said:
> >>>
> >>> "The backing memory of |CAPTURE| buffers that are used as reference
> >>> frames by the stream may be read by the hardware even after they are
> >>> dequeued. Consequently, the client should avoid writing into this memory
> >>> while the |CAPTURE| queue is streaming. Failure to observe this may
> >>> result in corruption of decoded frames."
> >>>
> >>>>>> + if (vb->planes[0].mem_priv)
> >>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
> >>>>>> +
> >>>>>> + /* Free MMAP buffers or release USERPTR buffers */
> >>>>>> + if (q->memory == VB2_MEMORY_MMAP)
> >>>>>> + __vb2_buf_mem_free(vb);
> >> Here is another problem for the existing application, the mmap() from the
> >> mmap offset or exportbuffer fd would not create a reference to buffer in
> >> this step(while the exportbuffer would create one itself).
> >>
> >> When you delete a buffer, you may not release it from its virtual memory
> >> space, leaving a corrupted virtual memory space.
> > What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
> > memory, it just decrements a reference counter.
>
> struct dma_buf_ops->mmap() may not increase a reference to its buffer.
Both V4L2 mmap() and DMA-buf mmap() of buffers exported from V4L2
would increase a reference to the buffer. They both lead to
vb2_{dc,sg,vmalloc}_mmap() which open the VMA, which in turn calls
vb2_common_vm_open() that increases the buffer refcount.
Best regards,
Tomasz
>
> While struct vb2_mem_ops->get_dmabuf() would.
>
> > The VMA holds its own,
> > so the buffer is only fully released when the application calls
> > munmap().
>
> DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF
> ioctl() make the no other user could access to this.
>
> >
> > Best regards,
> > Tomasz
> >
> >> Also this behavior is
> >> right, because mmap(2) says:
> >>
> >> "After the mmap() call has returned, the file descriptor, fd, can be
> >> closed immediately without invalidating the map‐ping."
> >>
> >>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
> >>>>>> + __vb2_buf_dmabuf_put(vb);
> >>>>>> + else
> >>>>>> + __vb2_buf_userptr_put(vb);
> >>>>>> +
> >>>>>> + vb2_queue_remove_buffer(q, vb);
> >>>>>> + kfree(vb);
> >>>>>> +
> >>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
> >>>>>> + return 0;
> >>>>>> +}
> >>>>>> +
> >>>>>> /*
> >>>>>> * vb2_start_streaming() - Attempt to start streaming.
> >>>>>> * @q: videobuf2 queue
> >>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> index 724135d41f7f..cea666c17b41 100644
> >>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
> >>>>>> media_device *mdev,
> >>>>>> }
> >>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
> >>>>>>
> >>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> >>>>>> +{
> >>>>>> + return vb2_core_delete_buf(q, b->index);
> >>>>>> +}
> >>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> >>>>>> +
> >>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
> >>>>>> *create)
> >>>>>> {
> >>>>>> unsigned requested_planes = 1;
> >>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> index f81279492682..80ace2e1e932 100644
> >>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
> >>>>>> video_device *vdev)
> >>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
> >>>>>> vidioc_prepare_buf);
> >>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
> >>>>>> vidioc_streamon);
> >>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
> >>>>>> vidioc_streamoff);
> >>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
> >>>>>> vidioc_delete_buf);
> >>>>>> }
> >>>>>>
> >>>>>> if (is_vid || is_vbi || is_meta) {
> >>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> index a858acea6547..1c737279d3ef 100644
> >>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
> >>>>>> v4l2_ioctl_ops *ops,
> >>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
> >>>>>> }
> >>>>>>
> >>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> >>>>>> + struct file *file, void *fh, void *arg)
> >>>>>> +{
> >>>>>> + struct v4l2_buffer *b = arg;
> >>>>>> + int ret = check_fmt(file, b->type);
> >>>>>> +
> >>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> >>>>>> +}
> >>>>>> +
> >>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> >>>>>> struct file *file, void *fh, void
> >>>>>> *arg)
> >>>>>> {
> >>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
> >>>>>> v4l2_ioctls[] = {
> >>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
> >>>>>> v4l_print_freq_band, 0),
> >>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
> >>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> >>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
> >>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
> >>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> >>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
> >>>>>> v4l_print_buffer, INFO_FL_QUEUE),
> >>>>>> };
> >>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
> >>>>>>
> >>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> >>>>>> index edb733f21604..2f232ed884c7 100644
> >>>>>> --- a/include/media/v4l2-ioctl.h
> >>>>>> +++ b/include/media/v4l2-ioctl.h
> >>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
> >>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> >>>>>> * @vidioc_prepare_buf: pointer to the function that implements
> >>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> >>>>>> + * @vidioc_delete_buf: pointer to the function that implements
> >>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> >>>>>> * @vidioc_overlay: pointer to the function that implements
> >>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> >>>>>> * @vidioc_g_fbuf: pointer to the function that implements
> >>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
> >>>>>> struct v4l2_create_buffers *b);
> >>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
> >>>>>> struct v4l2_buffer *b);
> >>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
> >>>>>> + struct v4l2_buffer *b);
> >>>>>>
> >>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
> >>>>>> int i);
> >>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
> >>>>>> diff --git a/include/media/videobuf2-core.h
> >>>>>> b/include/media/videobuf2-core.h
> >>>>>> index 080b783d608d..0f9e68f76b77 100644
> >>>>>> --- a/include/media/videobuf2-core.h
> >>>>>> +++ b/include/media/videobuf2-core.h
> >>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
> >>>>>> enum vb2_memory memory,
> >>>>>> */
> >>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> >>>>>> void *pb);
> >>>>>>
> >>>>>> +/**
> >>>>>> + * vb2_core_delete_buf() -
> >>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> >>>>>> + * @index: id number of the buffer.
> >>>>>> + *
> >>>>>> + * Return: returns zero on success; an error code otherwise.
> >>>>>> + */
> >>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> >>>>>> +
> >>>>>> /**
> >>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
> >>>>>> *
> >>>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>>> b/include/media/videobuf2-v4l2.h
> >>>>>> index 88a7a565170e..3beeb4c735f0 100644
> >>>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
> >>>>>> v4l2_create_buffers *create);
> >>>>>> */
> >>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> >>>>>> struct v4l2_buffer *b);
> >>>>>> +/**
> >>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
> >>>>>> + *
> >>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> >>>>>> + * @b: buffer structure passed from userspace to
> >>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> >>>>>> + *
> >>>>>> + * The return values from this function are intended to be directly
> >>>>>> returned
> >>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> >>>>>> + */
> >>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
> >>>>>>
> >>>>>> /**
> >>>>>> * vb2_qbuf() - Queue a buffer from userspace
> >>>>>> diff --git a/include/uapi/linux/videodev2.h
> >>>>>> b/include/uapi/linux/videodev2.h
> >>>>>> index aee75eb9e686..31bba1915642 100644
> >>>>>> --- a/include/uapi/linux/videodev2.h
> >>>>>> +++ b/include/uapi/linux/videodev2.h
> >>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
> >>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
> >>>>>> v4l2_dbg_chip_info)
> >>>>>>
> >>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
> >>>>>> v4l2_query_ext_ctrl)
> >>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
> >>>>>> +
> >>>>>>
> >>>>>> /* Reminder: when adding new ioctls please add support for them to
> >>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> >>>>>> --
> >>>>>> 2.39.2
> >>>>>>
> >>> --
> >>> Hsia-Jun(Randy) Li
> >>>
> >> --
> >> Hsia-Jun(Randy) Li
> >>
> --
> Hsia-Jun(Randy) Li
>
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-28 6:57 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-28 6:57 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On Mon, Jul 17, 2023 at 11:17 AM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
> On 7/13/23 17:09, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
> >> On 6/27/23 16:47, Hsia-Jun Li wrote:
> >>> CAUTION: Email originated externally, do not click links or open
> >>> attachments unless you recognize the sender and know the content is
> >>> safe.
> >>>
> >>>
> >>> On 6/27/23 16:43, Benjamin Gaignard wrote:
> >>>> CAUTION: Email originated externally, do not click links or open
> >>>> attachments unless you recognize the sender and know the content is
> >>>> safe.
> >>>>
> >>>>
> >>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
> >>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>> attachments unless you recognize the sender and know the content is
> >>>>>> safe.
> >>>>>>
> >>>>>>
> >>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
> >>>>>>
> >>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>>> ---
> >>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
> >>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
> >>>>>> +++++++++++++++++++
> >>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> >>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> >>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> >>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> >>>>>> include/media/v4l2-ioctl.h | 4 ++
> >>>>>> include/media/videobuf2-core.h | 9 ++++
> >>>>>> include/media/videobuf2-v4l2.h | 11 ++++
> >>>>>> include/uapi/linux/videodev2.h | 2 +
> >>>>>> 10 files changed, 128 insertions(+)
> >>>>>> create mode 100644
> >>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>>
> >>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
> >>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> @@ -17,6 +17,7 @@ Function Reference
> >>>>>> vidioc-dbg-g-chip-info
> >>>>>> vidioc-dbg-g-register
> >>>>>> vidioc-decoder-cmd
> >>>>>> + vidioc-delete-buf
> >>>>>> vidioc-dqevent
> >>>>>> vidioc-dv-timings-cap
> >>>>>> vidioc-encoder-cmd
> >>>>>> diff --git
> >>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> new file mode 100644
> >>>>>> index 000000000000..0e7ce58f91bc
> >>>>>> --- /dev/null
> >>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> @@ -0,0 +1,51 @@
> >>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> >>>>>> +.. c:namespace:: V4L
> >>>>>> +
> >>>>>> +.. _VIDIOC_DELETE_BUF:
> >>>>>> +
> >>>>>> +************************
> >>>>>> +ioctl VIDIOC_DELETE_BUF
> >>>>>> +************************
> >>>>>> +
> >>>>>> +Name
> >>>>>> +====
> >>>>>> +
> >>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> >>>>>> +
> >>>>>> +Synopsis
> >>>>>> +========
> >>>>>> +
> >>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
> >>>>>> +
> >>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> >>>>>> +
> >>>>>> +Arguments
> >>>>>> +=========
> >>>>>> +
> >>>>>> +``fd``
> >>>>>> + File descriptor returned by :c:func:`open()`.
> >>>>>> +
> >>>>>> +``argp``
> >>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
> >>>>>> +
> >>>>>> +Description
> >>>>>> +===========
> >>>>>> +
> >>>>>> +Applications can optionally call the
> >>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
> >>>>>> +delete a buffer from a queue.
> >>>>>> +
> >>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
> >>>>>> +:ref:`buffer`.
> >>>>>> +
> >>>>>> +Return Value
> >>>>>> +============
> >>>>>> +
> >>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
> >>>>>> set
> >>>>>> +appropriately. The generic error codes are described at the
> >>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
> >>>>>> +
> >>>>>> +EBUSY
> >>>>>> + File I/O is in progress.
> >>>>>> +
> >>>>>> +EINVAL
> >>>>>> + The buffer ``index`` doesn't exist in the queue.
> >>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> index 899783f67580..aa546c972c3d 100644
> >>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
> >>>>>> unsigned int index, void *pb)
> >>>>>> }
> >>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
> >>>>>>
> >>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> >>>>>> +{
> >>>>>> + struct vb2_buffer *vb;
> >>>>>> +
> >>>>>> + vb = vb2_get_buffer(q, index);
> >>>>>> + if (!vb) {
> >>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
> >>>>>> + return -EINVAL;
> >>>>>> + }
> >>>>>> +
> >>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> >>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
> >>>>>> %d\n", index);
> >>>>>> + return -EINVAL;
> >>>>>> + }
> >>>>>> +
> >>>>> I know the driver could implement its own
> >>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
> >>>>> used by the hardware as a future reference frame.
> >>>>> But I think we need a flag to let the user know which buffer is still
> >>>>> used by the hardware.
> >>>>> Alternative ref case is safe, we only know it's existing when it is
> >>>>> dequeued in current V4L2 buffer mechanism.
> >>>>> While the Golden reference frame, such long term reference frame could
> >>>>> last much longer.
> >>>> It is up to userland stack to know frames life time, it got the
> >>>> information for that.
> >>> That is true for the stateless codec driver.
> >>>
> >>> While application for stateful decoder could never do that. It also
> >>> breaks what the document said:
> >>>
> >>> "The backing memory of |CAPTURE| buffers that are used as reference
> >>> frames by the stream may be read by the hardware even after they are
> >>> dequeued. Consequently, the client should avoid writing into this memory
> >>> while the |CAPTURE| queue is streaming. Failure to observe this may
> >>> result in corruption of decoded frames."
> >>>
> >>>>>> + if (vb->planes[0].mem_priv)
> >>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
> >>>>>> +
> >>>>>> + /* Free MMAP buffers or release USERPTR buffers */
> >>>>>> + if (q->memory == VB2_MEMORY_MMAP)
> >>>>>> + __vb2_buf_mem_free(vb);
> >> Here is another problem for the existing application, the mmap() from the
> >> mmap offset or exportbuffer fd would not create a reference to buffer in
> >> this step(while the exportbuffer would create one itself).
> >>
> >> When you delete a buffer, you may not release it from its virtual memory
> >> space, leaving a corrupted virtual memory space.
> > What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
> > memory, it just decrements a reference counter.
>
> struct dma_buf_ops->mmap() may not increase a reference to its buffer.
Both V4L2 mmap() and DMA-buf mmap() of buffers exported from V4L2
would increase a reference to the buffer. They both lead to
vb2_{dc,sg,vmalloc}_mmap() which open the VMA, which in turn calls
vb2_common_vm_open() that increases the buffer refcount.
Best regards,
Tomasz
>
> While struct vb2_mem_ops->get_dmabuf() would.
>
> > The VMA holds its own,
> > so the buffer is only fully released when the application calls
> > munmap().
>
> DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF
> ioctl() make the no other user could access to this.
>
> >
> > Best regards,
> > Tomasz
> >
> >> Also this behavior is
> >> right, because mmap(2) says:
> >>
> >> "After the mmap() call has returned, the file descriptor, fd, can be
> >> closed immediately without invalidating the map‐ping."
> >>
> >>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
> >>>>>> + __vb2_buf_dmabuf_put(vb);
> >>>>>> + else
> >>>>>> + __vb2_buf_userptr_put(vb);
> >>>>>> +
> >>>>>> + vb2_queue_remove_buffer(q, vb);
> >>>>>> + kfree(vb);
> >>>>>> +
> >>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
> >>>>>> + return 0;
> >>>>>> +}
> >>>>>> +
> >>>>>> /*
> >>>>>> * vb2_start_streaming() - Attempt to start streaming.
> >>>>>> * @q: videobuf2 queue
> >>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> index 724135d41f7f..cea666c17b41 100644
> >>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
> >>>>>> media_device *mdev,
> >>>>>> }
> >>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
> >>>>>>
> >>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> >>>>>> +{
> >>>>>> + return vb2_core_delete_buf(q, b->index);
> >>>>>> +}
> >>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> >>>>>> +
> >>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
> >>>>>> *create)
> >>>>>> {
> >>>>>> unsigned requested_planes = 1;
> >>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> index f81279492682..80ace2e1e932 100644
> >>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
> >>>>>> video_device *vdev)
> >>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
> >>>>>> vidioc_prepare_buf);
> >>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
> >>>>>> vidioc_streamon);
> >>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
> >>>>>> vidioc_streamoff);
> >>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
> >>>>>> vidioc_delete_buf);
> >>>>>> }
> >>>>>>
> >>>>>> if (is_vid || is_vbi || is_meta) {
> >>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> index a858acea6547..1c737279d3ef 100644
> >>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
> >>>>>> v4l2_ioctl_ops *ops,
> >>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
> >>>>>> }
> >>>>>>
> >>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> >>>>>> + struct file *file, void *fh, void *arg)
> >>>>>> +{
> >>>>>> + struct v4l2_buffer *b = arg;
> >>>>>> + int ret = check_fmt(file, b->type);
> >>>>>> +
> >>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> >>>>>> +}
> >>>>>> +
> >>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> >>>>>> struct file *file, void *fh, void
> >>>>>> *arg)
> >>>>>> {
> >>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
> >>>>>> v4l2_ioctls[] = {
> >>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
> >>>>>> v4l_print_freq_band, 0),
> >>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
> >>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> >>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
> >>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
> >>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> >>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
> >>>>>> v4l_print_buffer, INFO_FL_QUEUE),
> >>>>>> };
> >>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
> >>>>>>
> >>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> >>>>>> index edb733f21604..2f232ed884c7 100644
> >>>>>> --- a/include/media/v4l2-ioctl.h
> >>>>>> +++ b/include/media/v4l2-ioctl.h
> >>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
> >>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> >>>>>> * @vidioc_prepare_buf: pointer to the function that implements
> >>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> >>>>>> + * @vidioc_delete_buf: pointer to the function that implements
> >>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> >>>>>> * @vidioc_overlay: pointer to the function that implements
> >>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> >>>>>> * @vidioc_g_fbuf: pointer to the function that implements
> >>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
> >>>>>> struct v4l2_create_buffers *b);
> >>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
> >>>>>> struct v4l2_buffer *b);
> >>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
> >>>>>> + struct v4l2_buffer *b);
> >>>>>>
> >>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
> >>>>>> int i);
> >>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
> >>>>>> diff --git a/include/media/videobuf2-core.h
> >>>>>> b/include/media/videobuf2-core.h
> >>>>>> index 080b783d608d..0f9e68f76b77 100644
> >>>>>> --- a/include/media/videobuf2-core.h
> >>>>>> +++ b/include/media/videobuf2-core.h
> >>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
> >>>>>> enum vb2_memory memory,
> >>>>>> */
> >>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> >>>>>> void *pb);
> >>>>>>
> >>>>>> +/**
> >>>>>> + * vb2_core_delete_buf() -
> >>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> >>>>>> + * @index: id number of the buffer.
> >>>>>> + *
> >>>>>> + * Return: returns zero on success; an error code otherwise.
> >>>>>> + */
> >>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> >>>>>> +
> >>>>>> /**
> >>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
> >>>>>> *
> >>>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>>> b/include/media/videobuf2-v4l2.h
> >>>>>> index 88a7a565170e..3beeb4c735f0 100644
> >>>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
> >>>>>> v4l2_create_buffers *create);
> >>>>>> */
> >>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> >>>>>> struct v4l2_buffer *b);
> >>>>>> +/**
> >>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
> >>>>>> + *
> >>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> >>>>>> + * @b: buffer structure passed from userspace to
> >>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> >>>>>> + *
> >>>>>> + * The return values from this function are intended to be directly
> >>>>>> returned
> >>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> >>>>>> + */
> >>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
> >>>>>>
> >>>>>> /**
> >>>>>> * vb2_qbuf() - Queue a buffer from userspace
> >>>>>> diff --git a/include/uapi/linux/videodev2.h
> >>>>>> b/include/uapi/linux/videodev2.h
> >>>>>> index aee75eb9e686..31bba1915642 100644
> >>>>>> --- a/include/uapi/linux/videodev2.h
> >>>>>> +++ b/include/uapi/linux/videodev2.h
> >>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
> >>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
> >>>>>> v4l2_dbg_chip_info)
> >>>>>>
> >>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
> >>>>>> v4l2_query_ext_ctrl)
> >>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
> >>>>>> +
> >>>>>>
> >>>>>> /* Reminder: when adding new ioctls please add support for them to
> >>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> >>>>>> --
> >>>>>> 2.39.2
> >>>>>>
> >>> --
> >>> Hsia-Jun(Randy) Li
> >>>
> >> --
> >> Hsia-Jun(Randy) Li
> >>
> --
> Hsia-Jun(Randy) Li
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-28 6:57 ` Tomasz Figa
0 siblings, 0 replies; 183+ messages in thread
From: Tomasz Figa @ 2023-07-28 6:57 UTC (permalink / raw)
To: Hsia-Jun Li
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On Mon, Jul 17, 2023 at 11:17 AM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>
>
> On 7/13/23 17:09, Tomasz Figa wrote:
> > CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> > On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
> >> On 6/27/23 16:47, Hsia-Jun Li wrote:
> >>> CAUTION: Email originated externally, do not click links or open
> >>> attachments unless you recognize the sender and know the content is
> >>> safe.
> >>>
> >>>
> >>> On 6/27/23 16:43, Benjamin Gaignard wrote:
> >>>> CAUTION: Email originated externally, do not click links or open
> >>>> attachments unless you recognize the sender and know the content is
> >>>> safe.
> >>>>
> >>>>
> >>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
> >>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
> >>>>>> CAUTION: Email originated externally, do not click links or open
> >>>>>> attachments unless you recognize the sender and know the content is
> >>>>>> safe.
> >>>>>>
> >>>>>>
> >>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
> >>>>>>
> >>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >>>>>> ---
> >>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
> >>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
> >>>>>> +++++++++++++++++++
> >>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
> >>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
> >>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> >>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
> >>>>>> include/media/v4l2-ioctl.h | 4 ++
> >>>>>> include/media/videobuf2-core.h | 9 ++++
> >>>>>> include/media/videobuf2-v4l2.h | 11 ++++
> >>>>>> include/uapi/linux/videodev2.h | 2 +
> >>>>>> 10 files changed, 128 insertions(+)
> >>>>>> create mode 100644
> >>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>>
> >>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
> >>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
> >>>>>> @@ -17,6 +17,7 @@ Function Reference
> >>>>>> vidioc-dbg-g-chip-info
> >>>>>> vidioc-dbg-g-register
> >>>>>> vidioc-decoder-cmd
> >>>>>> + vidioc-delete-buf
> >>>>>> vidioc-dqevent
> >>>>>> vidioc-dv-timings-cap
> >>>>>> vidioc-encoder-cmd
> >>>>>> diff --git
> >>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> new file mode 100644
> >>>>>> index 000000000000..0e7ce58f91bc
> >>>>>> --- /dev/null
> >>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
> >>>>>> @@ -0,0 +1,51 @@
> >>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> >>>>>> +.. c:namespace:: V4L
> >>>>>> +
> >>>>>> +.. _VIDIOC_DELETE_BUF:
> >>>>>> +
> >>>>>> +************************
> >>>>>> +ioctl VIDIOC_DELETE_BUF
> >>>>>> +************************
> >>>>>> +
> >>>>>> +Name
> >>>>>> +====
> >>>>>> +
> >>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
> >>>>>> +
> >>>>>> +Synopsis
> >>>>>> +========
> >>>>>> +
> >>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
> >>>>>> +
> >>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
> >>>>>> +
> >>>>>> +Arguments
> >>>>>> +=========
> >>>>>> +
> >>>>>> +``fd``
> >>>>>> + File descriptor returned by :c:func:`open()`.
> >>>>>> +
> >>>>>> +``argp``
> >>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
> >>>>>> +
> >>>>>> +Description
> >>>>>> +===========
> >>>>>> +
> >>>>>> +Applications can optionally call the
> >>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
> >>>>>> +delete a buffer from a queue.
> >>>>>> +
> >>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
> >>>>>> +:ref:`buffer`.
> >>>>>> +
> >>>>>> +Return Value
> >>>>>> +============
> >>>>>> +
> >>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
> >>>>>> set
> >>>>>> +appropriately. The generic error codes are described at the
> >>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
> >>>>>> +
> >>>>>> +EBUSY
> >>>>>> + File I/O is in progress.
> >>>>>> +
> >>>>>> +EINVAL
> >>>>>> + The buffer ``index`` doesn't exist in the queue.
> >>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> index 899783f67580..aa546c972c3d 100644
> >>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> >>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
> >>>>>> unsigned int index, void *pb)
> >>>>>> }
> >>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
> >>>>>>
> >>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
> >>>>>> +{
> >>>>>> + struct vb2_buffer *vb;
> >>>>>> +
> >>>>>> + vb = vb2_get_buffer(q, index);
> >>>>>> + if (!vb) {
> >>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
> >>>>>> + return -EINVAL;
> >>>>>> + }
> >>>>>> +
> >>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> >>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
> >>>>>> %d\n", index);
> >>>>>> + return -EINVAL;
> >>>>>> + }
> >>>>>> +
> >>>>> I know the driver could implement its own
> >>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
> >>>>> used by the hardware as a future reference frame.
> >>>>> But I think we need a flag to let the user know which buffer is still
> >>>>> used by the hardware.
> >>>>> Alternative ref case is safe, we only know it's existing when it is
> >>>>> dequeued in current V4L2 buffer mechanism.
> >>>>> While the Golden reference frame, such long term reference frame could
> >>>>> last much longer.
> >>>> It is up to userland stack to know frames life time, it got the
> >>>> information for that.
> >>> That is true for the stateless codec driver.
> >>>
> >>> While application for stateful decoder could never do that. It also
> >>> breaks what the document said:
> >>>
> >>> "The backing memory of |CAPTURE| buffers that are used as reference
> >>> frames by the stream may be read by the hardware even after they are
> >>> dequeued. Consequently, the client should avoid writing into this memory
> >>> while the |CAPTURE| queue is streaming. Failure to observe this may
> >>> result in corruption of decoded frames."
> >>>
> >>>>>> + if (vb->planes[0].mem_priv)
> >>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
> >>>>>> +
> >>>>>> + /* Free MMAP buffers or release USERPTR buffers */
> >>>>>> + if (q->memory == VB2_MEMORY_MMAP)
> >>>>>> + __vb2_buf_mem_free(vb);
> >> Here is another problem for the existing application, the mmap() from the
> >> mmap offset or exportbuffer fd would not create a reference to buffer in
> >> this step(while the exportbuffer would create one itself).
> >>
> >> When you delete a buffer, you may not release it from its virtual memory
> >> space, leaving a corrupted virtual memory space.
> > What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
> > memory, it just decrements a reference counter.
>
> struct dma_buf_ops->mmap() may not increase a reference to its buffer.
Both V4L2 mmap() and DMA-buf mmap() of buffers exported from V4L2
would increase a reference to the buffer. They both lead to
vb2_{dc,sg,vmalloc}_mmap() which open the VMA, which in turn calls
vb2_common_vm_open() that increases the buffer refcount.
Best regards,
Tomasz
>
> While struct vb2_mem_ops->get_dmabuf() would.
>
> > The VMA holds its own,
> > so the buffer is only fully released when the application calls
> > munmap().
>
> DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF
> ioctl() make the no other user could access to this.
>
> >
> > Best regards,
> > Tomasz
> >
> >> Also this behavior is
> >> right, because mmap(2) says:
> >>
> >> "After the mmap() call has returned, the file descriptor, fd, can be
> >> closed immediately without invalidating the map‐ping."
> >>
> >>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
> >>>>>> + __vb2_buf_dmabuf_put(vb);
> >>>>>> + else
> >>>>>> + __vb2_buf_userptr_put(vb);
> >>>>>> +
> >>>>>> + vb2_queue_remove_buffer(q, vb);
> >>>>>> + kfree(vb);
> >>>>>> +
> >>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
> >>>>>> + return 0;
> >>>>>> +}
> >>>>>> +
> >>>>>> /*
> >>>>>> * vb2_start_streaming() - Attempt to start streaming.
> >>>>>> * @q: videobuf2 queue
> >>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> index 724135d41f7f..cea666c17b41 100644
> >>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> >>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
> >>>>>> media_device *mdev,
> >>>>>> }
> >>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
> >>>>>>
> >>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> >>>>>> +{
> >>>>>> + return vb2_core_delete_buf(q, b->index);
> >>>>>> +}
> >>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
> >>>>>> +
> >>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
> >>>>>> *create)
> >>>>>> {
> >>>>>> unsigned requested_planes = 1;
> >>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> index f81279492682..80ace2e1e932 100644
> >>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
> >>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
> >>>>>> video_device *vdev)
> >>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
> >>>>>> vidioc_prepare_buf);
> >>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
> >>>>>> vidioc_streamon);
> >>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
> >>>>>> vidioc_streamoff);
> >>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
> >>>>>> vidioc_delete_buf);
> >>>>>> }
> >>>>>>
> >>>>>> if (is_vid || is_vbi || is_meta) {
> >>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> index a858acea6547..1c737279d3ef 100644
> >>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
> >>>>>> v4l2_ioctl_ops *ops,
> >>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
> >>>>>> }
> >>>>>>
> >>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
> >>>>>> + struct file *file, void *fh, void *arg)
> >>>>>> +{
> >>>>>> + struct v4l2_buffer *b = arg;
> >>>>>> + int ret = check_fmt(file, b->type);
> >>>>>> +
> >>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
> >>>>>> +}
> >>>>>> +
> >>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
> >>>>>> struct file *file, void *fh, void
> >>>>>> *arg)
> >>>>>> {
> >>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
> >>>>>> v4l2_ioctls[] = {
> >>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
> >>>>>> v4l_print_freq_band, 0),
> >>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
> >>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> >>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
> >>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
> >>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> >>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
> >>>>>> v4l_print_buffer, INFO_FL_QUEUE),
> >>>>>> };
> >>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
> >>>>>>
> >>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> >>>>>> index edb733f21604..2f232ed884c7 100644
> >>>>>> --- a/include/media/v4l2-ioctl.h
> >>>>>> +++ b/include/media/v4l2-ioctl.h
> >>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
> >>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
> >>>>>> * @vidioc_prepare_buf: pointer to the function that implements
> >>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
> >>>>>> + * @vidioc_delete_buf: pointer to the function that implements
> >>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
> >>>>>> * @vidioc_overlay: pointer to the function that implements
> >>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
> >>>>>> * @vidioc_g_fbuf: pointer to the function that implements
> >>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
> >>>>>> struct v4l2_create_buffers *b);
> >>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
> >>>>>> struct v4l2_buffer *b);
> >>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
> >>>>>> + struct v4l2_buffer *b);
> >>>>>>
> >>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
> >>>>>> int i);
> >>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
> >>>>>> diff --git a/include/media/videobuf2-core.h
> >>>>>> b/include/media/videobuf2-core.h
> >>>>>> index 080b783d608d..0f9e68f76b77 100644
> >>>>>> --- a/include/media/videobuf2-core.h
> >>>>>> +++ b/include/media/videobuf2-core.h
> >>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
> >>>>>> enum vb2_memory memory,
> >>>>>> */
> >>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> >>>>>> void *pb);
> >>>>>>
> >>>>>> +/**
> >>>>>> + * vb2_core_delete_buf() -
> >>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> >>>>>> + * @index: id number of the buffer.
> >>>>>> + *
> >>>>>> + * Return: returns zero on success; an error code otherwise.
> >>>>>> + */
> >>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
> >>>>>> +
> >>>>>> /**
> >>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
> >>>>>> *
> >>>>>> diff --git a/include/media/videobuf2-v4l2.h
> >>>>>> b/include/media/videobuf2-v4l2.h
> >>>>>> index 88a7a565170e..3beeb4c735f0 100644
> >>>>>> --- a/include/media/videobuf2-v4l2.h
> >>>>>> +++ b/include/media/videobuf2-v4l2.h
> >>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
> >>>>>> v4l2_create_buffers *create);
> >>>>>> */
> >>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> >>>>>> struct v4l2_buffer *b);
> >>>>>> +/**
> >>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
> >>>>>> + *
> >>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
> >>>>>> + * @b: buffer structure passed from userspace to
> >>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
> >>>>>> + *
> >>>>>> + * The return values from this function are intended to be directly
> >>>>>> returned
> >>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
> >>>>>> + */
> >>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
> >>>>>>
> >>>>>> /**
> >>>>>> * vb2_qbuf() - Queue a buffer from userspace
> >>>>>> diff --git a/include/uapi/linux/videodev2.h
> >>>>>> b/include/uapi/linux/videodev2.h
> >>>>>> index aee75eb9e686..31bba1915642 100644
> >>>>>> --- a/include/uapi/linux/videodev2.h
> >>>>>> +++ b/include/uapi/linux/videodev2.h
> >>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
> >>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
> >>>>>> v4l2_dbg_chip_info)
> >>>>>>
> >>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
> >>>>>> v4l2_query_ext_ctrl)
> >>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
> >>>>>> +
> >>>>>>
> >>>>>> /* Reminder: when adding new ioctls please add support for them to
> >>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
> >>>>>> --
> >>>>>> 2.39.2
> >>>>>>
> >>> --
> >>> Hsia-Jun(Randy) Li
> >>>
> >> --
> >> Hsia-Jun(Randy) Li
> >>
> --
> Hsia-Jun(Randy) Li
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-07-28 6:57 ` Tomasz Figa
(?)
@ 2023-07-28 7:26 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-28 7:26 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On 7/28/23 14:57, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Mon, Jul 17, 2023 at 11:17 AM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>> On 7/13/23 17:09, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>
>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>> ---
>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>> +++++++++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>> create mode 100644
>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>
>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>> vidioc-dbg-g-register
>>>>>>>> vidioc-decoder-cmd
>>>>>>>> + vidioc-delete-buf
>>>>>>>> vidioc-dqevent
>>>>>>>> vidioc-dv-timings-cap
>>>>>>>> vidioc-encoder-cmd
>>>>>>>> diff --git
>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> new file mode 100644
>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>> --- /dev/null
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>> +.. c:namespace:: V4L
>>>>>>>> +
>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>> +
>>>>>>>> +************************
>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>> +************************
>>>>>>>> +
>>>>>>>> +Name
>>>>>>>> +====
>>>>>>>> +
>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>> +
>>>>>>>> +Synopsis
>>>>>>>> +========
>>>>>>>> +
>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>> +
>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>>>> +
>>>>>>>> +Arguments
>>>>>>>> +=========
>>>>>>>> +
>>>>>>>> +``fd``
>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>> +
>>>>>>>> +``argp``
>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>> +
>>>>>>>> +Description
>>>>>>>> +===========
>>>>>>>> +
>>>>>>>> +Applications can optionally call the
>>>>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
>>>>>>>> +delete a buffer from a queue.
>>>>>>>> +
>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>> +:ref:`buffer`.
>>>>>>>> +
>>>>>>>> +Return Value
>>>>>>>> +============
>>>>>>>> +
>>>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>>>> set
>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>> +
>>>>>>>> +EBUSY
>>>>>>>> + File I/O is in progress.
>>>>>>>> +
>>>>>>>> +EINVAL
>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>>>> unsigned int index, void *pb)
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>> +{
>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>> +
>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>> + if (!vb) {
>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>>>>>> %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>> I know the driver could implement its own
>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>> used by the hardware as a future reference frame.
>>>>>>> But I think we need a flag to let the user know which buffer is still
>>>>>>> used by the hardware.
>>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>> While the Golden reference frame, such long term reference frame could
>>>>>>> last much longer.
>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>> information for that.
>>>>> That is true for the stateless codec driver.
>>>>>
>>>>> While application for stateful decoder could never do that. It also
>>>>> breaks what the document said:
>>>>>
>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>> frames by the stream may be read by the hardware even after they are
>>>>> dequeued. Consequently, the client should avoid writing into this memory
>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>> result in corruption of decoded frames."
>>>>>
>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>> +
>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>> + __vb2_buf_mem_free(vb);
>>>> Here is another problem for the existing application, the mmap() from the
>>>> mmap offset or exportbuffer fd would not create a reference to buffer in
>>>> this step(while the exportbuffer would create one itself).
>>>>
>>>> When you delete a buffer, you may not release it from its virtual memory
>>>> space, leaving a corrupted virtual memory space.
>>> What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
>>> memory, it just decrements a reference counter.
>>
>> struct dma_buf_ops->mmap() may not increase a reference to its buffer.
I think we are talking the same refcount.
That is vb2_vmarea_handler->refcount.
While, I am thinking about refcount from vb2_dc_buf.
>
> Both V4L2 mmap() and DMA-buf mmap() of buffers exported from V4L2
> would increase a reference to the buffer. They both lead to
> vb2_{dc,sg,vmalloc}_mmap() which open the VMA, which in turn calls
> vb2_common_vm_open() that increases the buffer refcount.
>
> Best regards,
> Tomasz
>
>>
>> While struct vb2_mem_ops->get_dmabuf() would.
>>
>>> The VMA holds its own,
>>> so the buffer is only fully released when the application calls
>>> munmap().
>>
>> DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF
>> ioctl() make the no other user could access to this.
>>
>>>
>>> Best regards,
>>> Tomasz
>>>
>>>> Also this behavior is
>>>> right, because mmap(2) says:
>>>>
>>>> "After the mmap() call has returned, the file descriptor, fd, can be
>>>> closed immediately without invalidating the map‐ping."
>>>>
>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>> + else
>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>> +
>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>> + kfree(vb);
>>>>>>>> +
>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>> + return 0;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> /*
>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>> * @q: videobuf2 queue
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>>>> media_device *mdev,
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>> +{
>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>> +}
>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>> +
>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>>>>> *create)
>>>>>>>> {
>>>>>>>> unsigned requested_planes = 1;
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>> video_device *vdev)
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>> vidioc_prepare_buf);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>> vidioc_streamon);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>> vidioc_streamoff);
>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>> vidioc_delete_buf);
>>>>>>>> }
>>>>>>>>
>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>> }
>>>>>>>>
>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>>> +{
>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>> +
>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>> struct file *file, void *fh, void
>>>>>>>> *arg)
>>>>>>>> {
>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>> v4l2_ioctls[] = {
>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>> };
>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>
>>>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>> struct v4l2_create_buffers *b);
>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>>>> int i);
>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>>> enum vb2_memory memory,
>>>>>>>> */
>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>>>> void *pb);
>>>>>>>>
>>>>>>>> +/**
>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @index: id number of the buffer.
>>>>>>>> + *
>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>> + */
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>>> +
>>>>>>>> /**
>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>> *
>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>> v4l2_create_buffers *create);
>>>>>>>> */
>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> +/**
>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>> + *
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>> + *
>>>>>>>> + * The return values from this function are intended to be directly
>>>>>>>> returned
>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>> + */
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> /**
>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>
>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>>>> +
>>>>>>>>
>>>>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>> --
>>>>>>>> 2.39.2
>>>>>>>>
>>>>> --
>>>>> Hsia-Jun(Randy) Li
>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-28 7:26 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-28 7:26 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On 7/28/23 14:57, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Mon, Jul 17, 2023 at 11:17 AM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>> On 7/13/23 17:09, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>
>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>> ---
>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>> +++++++++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>> create mode 100644
>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>
>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>> vidioc-dbg-g-register
>>>>>>>> vidioc-decoder-cmd
>>>>>>>> + vidioc-delete-buf
>>>>>>>> vidioc-dqevent
>>>>>>>> vidioc-dv-timings-cap
>>>>>>>> vidioc-encoder-cmd
>>>>>>>> diff --git
>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> new file mode 100644
>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>> --- /dev/null
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>> +.. c:namespace:: V4L
>>>>>>>> +
>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>> +
>>>>>>>> +************************
>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>> +************************
>>>>>>>> +
>>>>>>>> +Name
>>>>>>>> +====
>>>>>>>> +
>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>> +
>>>>>>>> +Synopsis
>>>>>>>> +========
>>>>>>>> +
>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>> +
>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>>>> +
>>>>>>>> +Arguments
>>>>>>>> +=========
>>>>>>>> +
>>>>>>>> +``fd``
>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>> +
>>>>>>>> +``argp``
>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>> +
>>>>>>>> +Description
>>>>>>>> +===========
>>>>>>>> +
>>>>>>>> +Applications can optionally call the
>>>>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
>>>>>>>> +delete a buffer from a queue.
>>>>>>>> +
>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>> +:ref:`buffer`.
>>>>>>>> +
>>>>>>>> +Return Value
>>>>>>>> +============
>>>>>>>> +
>>>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>>>> set
>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>> +
>>>>>>>> +EBUSY
>>>>>>>> + File I/O is in progress.
>>>>>>>> +
>>>>>>>> +EINVAL
>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>>>> unsigned int index, void *pb)
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>> +{
>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>> +
>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>> + if (!vb) {
>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>>>>>> %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>> I know the driver could implement its own
>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>> used by the hardware as a future reference frame.
>>>>>>> But I think we need a flag to let the user know which buffer is still
>>>>>>> used by the hardware.
>>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>> While the Golden reference frame, such long term reference frame could
>>>>>>> last much longer.
>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>> information for that.
>>>>> That is true for the stateless codec driver.
>>>>>
>>>>> While application for stateful decoder could never do that. It also
>>>>> breaks what the document said:
>>>>>
>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>> frames by the stream may be read by the hardware even after they are
>>>>> dequeued. Consequently, the client should avoid writing into this memory
>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>> result in corruption of decoded frames."
>>>>>
>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>> +
>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>> + __vb2_buf_mem_free(vb);
>>>> Here is another problem for the existing application, the mmap() from the
>>>> mmap offset or exportbuffer fd would not create a reference to buffer in
>>>> this step(while the exportbuffer would create one itself).
>>>>
>>>> When you delete a buffer, you may not release it from its virtual memory
>>>> space, leaving a corrupted virtual memory space.
>>> What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
>>> memory, it just decrements a reference counter.
>>
>> struct dma_buf_ops->mmap() may not increase a reference to its buffer.
I think we are talking the same refcount.
That is vb2_vmarea_handler->refcount.
While, I am thinking about refcount from vb2_dc_buf.
>
> Both V4L2 mmap() and DMA-buf mmap() of buffers exported from V4L2
> would increase a reference to the buffer. They both lead to
> vb2_{dc,sg,vmalloc}_mmap() which open the VMA, which in turn calls
> vb2_common_vm_open() that increases the buffer refcount.
>
> Best regards,
> Tomasz
>
>>
>> While struct vb2_mem_ops->get_dmabuf() would.
>>
>>> The VMA holds its own,
>>> so the buffer is only fully released when the application calls
>>> munmap().
>>
>> DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF
>> ioctl() make the no other user could access to this.
>>
>>>
>>> Best regards,
>>> Tomasz
>>>
>>>> Also this behavior is
>>>> right, because mmap(2) says:
>>>>
>>>> "After the mmap() call has returned, the file descriptor, fd, can be
>>>> closed immediately without invalidating the map‐ping."
>>>>
>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>> + else
>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>> +
>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>> + kfree(vb);
>>>>>>>> +
>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>> + return 0;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> /*
>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>> * @q: videobuf2 queue
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>>>> media_device *mdev,
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>> +{
>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>> +}
>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>> +
>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>>>>> *create)
>>>>>>>> {
>>>>>>>> unsigned requested_planes = 1;
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>> video_device *vdev)
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>> vidioc_prepare_buf);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>> vidioc_streamon);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>> vidioc_streamoff);
>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>> vidioc_delete_buf);
>>>>>>>> }
>>>>>>>>
>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>> }
>>>>>>>>
>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>>> +{
>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>> +
>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>> struct file *file, void *fh, void
>>>>>>>> *arg)
>>>>>>>> {
>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>> v4l2_ioctls[] = {
>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>> };
>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>
>>>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>> struct v4l2_create_buffers *b);
>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>>>> int i);
>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>>> enum vb2_memory memory,
>>>>>>>> */
>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>>>> void *pb);
>>>>>>>>
>>>>>>>> +/**
>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @index: id number of the buffer.
>>>>>>>> + *
>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>> + */
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>>> +
>>>>>>>> /**
>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>> *
>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>> v4l2_create_buffers *create);
>>>>>>>> */
>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> +/**
>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>> + *
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>> + *
>>>>>>>> + * The return values from this function are intended to be directly
>>>>>>>> returned
>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>> + */
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> /**
>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>
>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>>>> +
>>>>>>>>
>>>>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>> --
>>>>>>>> 2.39.2
>>>>>>>>
>>>>> --
>>>>> Hsia-Jun(Randy) Li
>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-28 7:26 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-07-28 7:26 UTC (permalink / raw)
To: Tomasz Figa
Cc: Benjamin Gaignard, linux-media, linux-kernel, p.zabel, ming.qian,
hverkuil-cisco, gregkh, ezequiel, linux-arm-kernel,
linux-mediatek, linux-arm-msm, linux-rockchip, nicolas.dufresne,
linux-staging, kernel, mchehab, m.szyprowski, ayaka
On 7/28/23 14:57, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Mon, Jul 17, 2023 at 11:17 AM Hsia-Jun Li <Randy.Li@synaptics.com> wrote:
>>
>>
>> On 7/13/23 17:09, Tomasz Figa wrote:
>>> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>>
>>> On Fri, Jun 30, 2023 at 05:43:51PM +0800, Hsia-Jun Li wrote:
>>>> On 6/27/23 16:47, Hsia-Jun Li wrote:
>>>>> CAUTION: Email originated externally, do not click links or open
>>>>> attachments unless you recognize the sender and know the content is
>>>>> safe.
>>>>>
>>>>>
>>>>> On 6/27/23 16:43, Benjamin Gaignard wrote:
>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>> attachments unless you recognize the sender and know the content is
>>>>>> safe.
>>>>>>
>>>>>>
>>>>>> Le 27/06/2023 à 09:30, Hsia-Jun Li a écrit :
>>>>>>> On 6/22/23 21:13, Benjamin Gaignard wrote:
>>>>>>>> CAUTION: Email originated externally, do not click links or open
>>>>>>>> attachments unless you recognize the sender and know the content is
>>>>>>>> safe.
>>>>>>>>
>>>>>>>>
>>>>>>>> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
>>>>>>>>
>>>>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>>>>> ---
>>>>>>>> .../userspace-api/media/v4l/user-func.rst | 1 +
>>>>>>>> .../media/v4l/vidioc-delete-buf.rst | 51
>>>>>>>> +++++++++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-core.c | 33 ++++++++++++
>>>>>>>> .../media/common/videobuf2/videobuf2-v4l2.c | 6 +++
>>>>>>>> drivers/media/v4l2-core/v4l2-dev.c | 1 +
>>>>>>>> drivers/media/v4l2-core/v4l2-ioctl.c | 10 ++++
>>>>>>>> include/media/v4l2-ioctl.h | 4 ++
>>>>>>>> include/media/videobuf2-core.h | 9 ++++
>>>>>>>> include/media/videobuf2-v4l2.h | 11 ++++
>>>>>>>> include/uapi/linux/videodev2.h | 2 +
>>>>>>>> 10 files changed, 128 insertions(+)
>>>>>>>> create mode 100644
>>>>>>>> Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>>
>>>>>>>> diff --git a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> index 15ff0bf7bbe6..8c74016e12fd 100644
>>>>>>>> --- a/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/user-func.rst
>>>>>>>> @@ -17,6 +17,7 @@ Function Reference
>>>>>>>> vidioc-dbg-g-chip-info
>>>>>>>> vidioc-dbg-g-register
>>>>>>>> vidioc-decoder-cmd
>>>>>>>> + vidioc-delete-buf
>>>>>>>> vidioc-dqevent
>>>>>>>> vidioc-dv-timings-cap
>>>>>>>> vidioc-encoder-cmd
>>>>>>>> diff --git
>>>>>>>> a/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> new file mode 100644
>>>>>>>> index 000000000000..0e7ce58f91bc
>>>>>>>> --- /dev/null
>>>>>>>> +++ b/Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>>>>>>>> @@ -0,0 +1,51 @@
>>>>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>>>>> +.. c:namespace:: V4L
>>>>>>>> +
>>>>>>>> +.. _VIDIOC_DELETE_BUF:
>>>>>>>> +
>>>>>>>> +************************
>>>>>>>> +ioctl VIDIOC_DELETE_BUF
>>>>>>>> +************************
>>>>>>>> +
>>>>>>>> +Name
>>>>>>>> +====
>>>>>>>> +
>>>>>>>> +VIDIOC_DELETE_BUF - Delete a buffer from a queue
>>>>>>>> +
>>>>>>>> +Synopsis
>>>>>>>> +========
>>>>>>>> +
>>>>>>>> +.. c:macro:: VIDIOC_DELETE_BUF
>>>>>>>> +
>>>>>>>> +``int ioctl(int fd, VIDIOC_DELETE_BUF, struct v4l2_buffer *argp)``
>>>>>>>> +
>>>>>>>> +Arguments
>>>>>>>> +=========
>>>>>>>> +
>>>>>>>> +``fd``
>>>>>>>> + File descriptor returned by :c:func:`open()`.
>>>>>>>> +
>>>>>>>> +``argp``
>>>>>>>> + Pointer to struct :c:type:`v4l2_buffer`.
>>>>>>>> +
>>>>>>>> +Description
>>>>>>>> +===========
>>>>>>>> +
>>>>>>>> +Applications can optionally call the
>>>>>>>> :ref:`VIDIOC_DELETE_BUF` ioctl to
>>>>>>>> +delete a buffer from a queue.
>>>>>>>> +
>>>>>>>> +The struct :c:type:`v4l2_buffer` structure is specified in
>>>>>>>> +:ref:`buffer`.
>>>>>>>> +
>>>>>>>> +Return Value
>>>>>>>> +============
>>>>>>>> +
>>>>>>>> +On success 0 is returned, on error -1 and the ``errno`` variable is
>>>>>>>> set
>>>>>>>> +appropriately. The generic error codes are described at the
>>>>>>>> +:ref:`Generic Error Codes <gen-errors>` chapter.
>>>>>>>> +
>>>>>>>> +EBUSY
>>>>>>>> + File I/O is in progress.
>>>>>>>> +
>>>>>>>> +EINVAL
>>>>>>>> + The buffer ``index`` doesn't exist in the queue.
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> index 899783f67580..aa546c972c3d 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>>>>>>>> @@ -1637,6 +1637,39 @@ int vb2_core_prepare_buf(struct vb2_queue *q,
>>>>>>>> unsigned int index, void *pb)
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index)
>>>>>>>> +{
>>>>>>>> + struct vb2_buffer *vb;
>>>>>>>> +
>>>>>>>> + vb = vb2_get_buffer(q, index);
>>>>>>>> + if (!vb) {
>>>>>>>> + dprintk(q, 1, "invalid buffer index %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>>> + if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>>>>>>>> + dprintk(q, 1, "can't delete non dequeued buffer index
>>>>>>>> %d\n", index);
>>>>>>>> + return -EINVAL;
>>>>>>>> + }
>>>>>>>> +
>>>>>>> I know the driver could implement its own
>>>>>>> v4l2_ioctl_ops->vidioc_delete_buf() that check whether a buffer is
>>>>>>> used by the hardware as a future reference frame.
>>>>>>> But I think we need a flag to let the user know which buffer is still
>>>>>>> used by the hardware.
>>>>>>> Alternative ref case is safe, we only know it's existing when it is
>>>>>>> dequeued in current V4L2 buffer mechanism.
>>>>>>> While the Golden reference frame, such long term reference frame could
>>>>>>> last much longer.
>>>>>> It is up to userland stack to know frames life time, it got the
>>>>>> information for that.
>>>>> That is true for the stateless codec driver.
>>>>>
>>>>> While application for stateful decoder could never do that. It also
>>>>> breaks what the document said:
>>>>>
>>>>> "The backing memory of |CAPTURE| buffers that are used as reference
>>>>> frames by the stream may be read by the hardware even after they are
>>>>> dequeued. Consequently, the client should avoid writing into this memory
>>>>> while the |CAPTURE| queue is streaming. Failure to observe this may
>>>>> result in corruption of decoded frames."
>>>>>
>>>>>>>> + if (vb->planes[0].mem_priv)
>>>>>>>> + call_void_vb_qop(vb, buf_cleanup, vb);
>>>>>>>> +
>>>>>>>> + /* Free MMAP buffers or release USERPTR buffers */
>>>>>>>> + if (q->memory == VB2_MEMORY_MMAP)
>>>>>>>> + __vb2_buf_mem_free(vb);
>>>> Here is another problem for the existing application, the mmap() from the
>>>> mmap offset or exportbuffer fd would not create a reference to buffer in
>>>> this step(while the exportbuffer would create one itself).
>>>>
>>>> When you delete a buffer, you may not release it from its virtual memory
>>>> space, leaving a corrupted virtual memory space.
>>> What do you mean? __vb2_buf_mem_free() doesn't unconditionally free the
>>> memory, it just decrements a reference counter.
>>
>> struct dma_buf_ops->mmap() may not increase a reference to its buffer.
I think we are talking the same refcount.
That is vb2_vmarea_handler->refcount.
While, I am thinking about refcount from vb2_dc_buf.
>
> Both V4L2 mmap() and DMA-buf mmap() of buffers exported from V4L2
> would increase a reference to the buffer. They both lead to
> vb2_{dc,sg,vmalloc}_mmap() which open the VMA, which in turn calls
> vb2_common_vm_open() that increases the buffer refcount.
>
> Best regards,
> Tomasz
>
>>
>> While struct vb2_mem_ops->get_dmabuf() would.
>>
>>> The VMA holds its own,
>>> so the buffer is only fully released when the application calls
>>> munmap().
>>
>> DELETE_BUF ioctl() didn't answer to this problem. Should the DELETE_BUF
>> ioctl() make the no other user could access to this.
>>
>>>
>>> Best regards,
>>> Tomasz
>>>
>>>> Also this behavior is
>>>> right, because mmap(2) says:
>>>>
>>>> "After the mmap() call has returned, the file descriptor, fd, can be
>>>> closed immediately without invalidating the map‐ping."
>>>>
>>>>>>>> + else if (q->memory == VB2_MEMORY_DMABUF)
>>>>>>>> + __vb2_buf_dmabuf_put(vb);
>>>>>>>> + else
>>>>>>>> + __vb2_buf_userptr_put(vb);
>>>>>>>> +
>>>>>>>> + vb2_queue_remove_buffer(q, vb);
>>>>>>>> + kfree(vb);
>>>>>>>> +
>>>>>>>> + dprintk(q, 2, "buffer %d deleted\n", index);
>>>>>>>> + return 0;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> /*
>>>>>>>> * vb2_start_streaming() - Attempt to start streaming.
>>>>>>>> * @q: videobuf2 queue
>>>>>>>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> index 724135d41f7f..cea666c17b41 100644
>>>>>>>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>>>>>>>> @@ -751,6 +751,12 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
>>>>>>>> media_device *mdev,
>>>>>>>> }
>>>>>>>> EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>>>>>>>>
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>>>>>>>> +{
>>>>>>>> + return vb2_core_delete_buf(q, b->index);
>>>>>>>> +}
>>>>>>>> +EXPORT_SYMBOL_GPL(vb2_delete_buf);
>>>>>>>> +
>>>>>>>> int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers
>>>>>>>> *create)
>>>>>>>> {
>>>>>>>> unsigned requested_planes = 1;
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> index f81279492682..80ace2e1e932 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>>>>>>>> @@ -720,6 +720,7 @@ static void determine_valid_ioctls(struct
>>>>>>>> video_device *vdev)
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_PREPARE_BUF,
>>>>>>>> vidioc_prepare_buf);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMON,
>>>>>>>> vidioc_streamon);
>>>>>>>> SET_VALID_IOCTL(ops, VIDIOC_STREAMOFF,
>>>>>>>> vidioc_streamoff);
>>>>>>>> + SET_VALID_IOCTL(ops, VIDIOC_DELETE_BUF,
>>>>>>>> vidioc_delete_buf);
>>>>>>>> }
>>>>>>>>
>>>>>>>> if (is_vid || is_vbi || is_meta) {
>>>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> index a858acea6547..1c737279d3ef 100644
>>>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>>>> @@ -2156,6 +2156,15 @@ static int v4l_prepare_buf(const struct
>>>>>>>> v4l2_ioctl_ops *ops,
>>>>>>>> return ret ? ret : ops->vidioc_prepare_buf(file, fh, b);
>>>>>>>> }
>>>>>>>>
>>>>>>>> +static int v4l_delete_buf(const struct v4l2_ioctl_ops *ops,
>>>>>>>> + struct file *file, void *fh, void *arg)
>>>>>>>> +{
>>>>>>>> + struct v4l2_buffer *b = arg;
>>>>>>>> + int ret = check_fmt(file, b->type);
>>>>>>>> +
>>>>>>>> + return ret ? ret : ops->vidioc_delete_buf(file, fh, b);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> static int v4l_g_parm(const struct v4l2_ioctl_ops *ops,
>>>>>>>> struct file *file, void *fh, void
>>>>>>>> *arg)
>>>>>>>> {
>>>>>>>> @@ -2905,6 +2914,7 @@ static const struct v4l2_ioctl_info
>>>>>>>> v4l2_ioctls[] = {
>>>>>>>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands,
>>>>>>>> v4l_print_freq_band, 0),
>>>>>>>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info,
>>>>>>>> v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>>>>>>>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl,
>>>>>>>> v4l_print_query_ext_ctrl, INFO_FL_CTRL |
>>>>>>>> INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>>>>>>>> + IOCTL_INFO(VIDIOC_DELETE_BUF, v4l_delete_buf,
>>>>>>>> v4l_print_buffer, INFO_FL_QUEUE),
>>>>>>>> };
>>>>>>>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>>>>>>>
>>>>>>>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>>>>>>>> index edb733f21604..2f232ed884c7 100644
>>>>>>>> --- a/include/media/v4l2-ioctl.h
>>>>>>>> +++ b/include/media/v4l2-ioctl.h
>>>>>>>> @@ -163,6 +163,8 @@ struct v4l2_fh;
>>>>>>>> * :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
>>>>>>>> * @vidioc_prepare_buf: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_PREPARE_BUF <vidioc_prepare_buf>` ioctl
>>>>>>>> + * @vidioc_delete_buf: pointer to the function that implements
>>>>>>>> + * :ref:`VIDIOC_DELETE_BUF <vidioc_delete_buf>` ioctl
>>>>>>>> * @vidioc_overlay: pointer to the function that implements
>>>>>>>> * :ref:`VIDIOC_OVERLAY <vidioc_overlay>` ioctl
>>>>>>>> * @vidioc_g_fbuf: pointer to the function that implements
>>>>>>>> @@ -422,6 +424,8 @@ struct v4l2_ioctl_ops {
>>>>>>>> struct v4l2_create_buffers *b);
>>>>>>>> int (*vidioc_prepare_buf)(struct file *file, void *fh,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> + int (*vidioc_delete_buf)(struct file *file, void *fh,
>>>>>>>> + struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> int (*vidioc_overlay)(struct file *file, void *fh, unsigned
>>>>>>>> int i);
>>>>>>>> int (*vidioc_g_fbuf)(struct file *file, void *fh,
>>>>>>>> diff --git a/include/media/videobuf2-core.h
>>>>>>>> b/include/media/videobuf2-core.h
>>>>>>>> index 080b783d608d..0f9e68f76b77 100644
>>>>>>>> --- a/include/media/videobuf2-core.h
>>>>>>>> +++ b/include/media/videobuf2-core.h
>>>>>>>> @@ -840,6 +840,15 @@ int vb2_core_create_bufs(struct vb2_queue *q,
>>>>>>>> enum vb2_memory memory,
>>>>>>>> */
>>>>>>>> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
>>>>>>>> void *pb);
>>>>>>>>
>>>>>>>> +/**
>>>>>>>> + * vb2_core_delete_buf() -
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @index: id number of the buffer.
>>>>>>>> + *
>>>>>>>> + * Return: returns zero on success; an error code otherwise.
>>>>>>>> + */
>>>>>>>> +int vb2_core_delete_buf(struct vb2_queue *q, unsigned int index);
>>>>>>>> +
>>>>>>>> /**
>>>>>>>> * vb2_core_qbuf() - Queue a buffer from userspace
>>>>>>>> *
>>>>>>>> diff --git a/include/media/videobuf2-v4l2.h
>>>>>>>> b/include/media/videobuf2-v4l2.h
>>>>>>>> index 88a7a565170e..3beeb4c735f0 100644
>>>>>>>> --- a/include/media/videobuf2-v4l2.h
>>>>>>>> +++ b/include/media/videobuf2-v4l2.h
>>>>>>>> @@ -114,6 +114,17 @@ int vb2_create_bufs(struct vb2_queue *q, struct
>>>>>>>> v4l2_create_buffers *create);
>>>>>>>> */
>>>>>>>> int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>>>>>>>> struct v4l2_buffer *b);
>>>>>>>> +/**
>>>>>>>> + * vb2_delete_buf() - Delete the buffer from the queue
>>>>>>>> + *
>>>>>>>> + * @q: pointer to &struct vb2_queue with videobuf2 queue.
>>>>>>>> + * @b: buffer structure passed from userspace to
>>>>>>>> + * &v4l2_ioctl_ops->vidioc_delete_buf handler in driver
>>>>>>>> + *
>>>>>>>> + * The return values from this function are intended to be directly
>>>>>>>> returned
>>>>>>>> + * from &v4l2_ioctl_ops->vidioc_delete_buf handler in driver.
>>>>>>>> + */
>>>>>>>> +int vb2_delete_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>>>>>>>>
>>>>>>>> /**
>>>>>>>> * vb2_qbuf() - Queue a buffer from userspace
>>>>>>>> diff --git a/include/uapi/linux/videodev2.h
>>>>>>>> b/include/uapi/linux/videodev2.h
>>>>>>>> index aee75eb9e686..31bba1915642 100644
>>>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>>>> @@ -2702,6 +2702,8 @@ struct v4l2_create_buffers {
>>>>>>>> #define VIDIOC_DBG_G_CHIP_INFO _IOWR('V', 102, struct
>>>>>>>> v4l2_dbg_chip_info)
>>>>>>>>
>>>>>>>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct
>>>>>>>> v4l2_query_ext_ctrl)
>>>>>>>> +#define VIDIOC_DELETE_BUF _IOWR('V', 104, struct v4l2_buffer)
>>>>>>>> +
>>>>>>>>
>>>>>>>> /* Reminder: when adding new ioctls please add support for them to
>>>>>>>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>>>>>>> --
>>>>>>>> 2.39.2
>>>>>>>>
>>>>> --
>>>>> Hsia-Jun(Randy) Li
>>>>>
>>>> --
>>>> Hsia-Jun(Randy) Li
>>>>
>> --
>> Hsia-Jun(Randy) Li
>>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-07-02 10:20 ` Markus Elfring
-1 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 10:20 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
Please choose another imperative change suggestion.
See also:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst?h=v6.4#n94
Regards,
Markus
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-02 10:20 ` Markus Elfring
0 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 10:20 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
Please choose another imperative change suggestion.
See also:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst?h=v6.4#n94
Regards,
Markus
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 10/11] media: v4l2: Add DELETE_BUF ioctl
@ 2023-07-02 10:20 ` Markus Elfring
0 siblings, 0 replies; 183+ messages in thread
From: Markus Elfring @ 2023-07-02 10:20 UTC (permalink / raw)
To: Benjamin Gaignard, linux-staging, linux-media, linux-arm-kernel,
linux-arm-msm, linux-rockchip, linux-mediatek, kernel-janitors,
Ezequiel Garcia, Greg Kroah-Hartman, Hans Verkuil,
Marek Szyprowski, Mauro Carvalho Chehab, Ming Qian,
Nicolas Dufresne, Philipp Zabel, Tomasz Figa
Cc: LKML, kernel
> VIDIOC_DELETE_BUF ioctl allows to delete a buffer from a queue.
Please choose another imperative change suggestion.
See also:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst?h=v6.4#n94
Regards,
Markus
^ permalink raw reply [flat|nested] 183+ messages in thread
* [PATCH v3 11/11] media: v4l2: Add mem2mem helpers for DELETE_BUF ioctl
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-22 13:13 ` Benjamin Gaignard
-1 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Create v4l2-mem2mem helpers for VIDIOC_DELETE_BUF ioctl.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/platform/verisilicon/hantro_v4l2.c | 1 +
drivers/media/test-drivers/vim2m.c | 1 +
drivers/media/v4l2-core/v4l2-mem2mem.c | 20 +++++++++++++++++++
include/media/v4l2-mem2mem.h | 12 +++++++++++
4 files changed, 34 insertions(+)
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index cdddfb4179aa..bf418191278f 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -720,6 +720,7 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = {
.vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
.vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
.vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_delete_buf = v4l2_m2m_ioctl_delete_buf,
.vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
diff --git a/drivers/media/test-drivers/vim2m.c b/drivers/media/test-drivers/vim2m.c
index 3e3b424b4860..3014b8ee13d0 100644
--- a/drivers/media/test-drivers/vim2m.c
+++ b/drivers/media/test-drivers/vim2m.c
@@ -960,6 +960,7 @@ static const struct v4l2_ioctl_ops vim2m_ioctl_ops = {
.vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
.vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
.vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_delete_buf = v4l2_m2m_ioctl_delete_buf,
.vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
.vidioc_streamon = v4l2_m2m_ioctl_streamon,
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
index 0cc30397fbad..42f51ca25379 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -831,6 +831,17 @@ int v4l2_m2m_prepare_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
}
EXPORT_SYMBOL_GPL(v4l2_m2m_prepare_buf);
+int v4l2_m2m_delete_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
+ struct v4l2_buffer *buf)
+{
+ struct vb2_queue *vq;
+
+ vq = v4l2_m2m_get_vq(m2m_ctx, buf->type);
+
+ return vb2_delete_buf(vq, buf);
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_delete_buf);
+
int v4l2_m2m_create_bufs(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
struct v4l2_create_buffers *create)
{
@@ -1377,6 +1388,15 @@ int v4l2_m2m_ioctl_create_bufs(struct file *file, void *priv,
}
EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_create_bufs);
+int v4l2_m2m_ioctl_delete_buf(struct file *file, void *priv,
+ struct v4l2_buffer *buf)
+{
+ struct v4l2_fh *fh = file->private_data;
+
+ return v4l2_m2m_delete_buf(file, fh->m2m_ctx, buf);
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_delete_buf);
+
int v4l2_m2m_ioctl_querybuf(struct file *file, void *priv,
struct v4l2_buffer *buf)
{
diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h
index bb9de6a899e0..d062a35c8246 100644
--- a/include/media/v4l2-mem2mem.h
+++ b/include/media/v4l2-mem2mem.h
@@ -381,6 +381,16 @@ int v4l2_m2m_dqbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
int v4l2_m2m_prepare_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
struct v4l2_buffer *buf);
+/**
+ * v4l2_m2m_delete_buf() - delete buffer from the queue
+ *
+ * @file: pointer to struct &file
+ * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
+ * @buf: pointer to struct &v4l2_buffer
+ */
+int v4l2_m2m_delete_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
+ struct v4l2_buffer *buf);
+
/**
* v4l2_m2m_create_bufs() - create a source or destination buffer, depending
* on the type
@@ -846,6 +856,8 @@ int v4l2_m2m_ioctl_reqbufs(struct file *file, void *priv,
struct v4l2_requestbuffers *rb);
int v4l2_m2m_ioctl_create_bufs(struct file *file, void *fh,
struct v4l2_create_buffers *create);
+int v4l2_m2m_ioctl_delete_buf(struct file *file, void *priv,
+ struct v4l2_buffer *buf);
int v4l2_m2m_ioctl_querybuf(struct file *file, void *fh,
struct v4l2_buffer *buf);
int v4l2_m2m_ioctl_expbuf(struct file *file, void *fh,
--
2.39.2
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 11/11] media: v4l2: Add mem2mem helpers for DELETE_BUF ioctl
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Create v4l2-mem2mem helpers for VIDIOC_DELETE_BUF ioctl.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/platform/verisilicon/hantro_v4l2.c | 1 +
drivers/media/test-drivers/vim2m.c | 1 +
drivers/media/v4l2-core/v4l2-mem2mem.c | 20 +++++++++++++++++++
include/media/v4l2-mem2mem.h | 12 +++++++++++
4 files changed, 34 insertions(+)
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index cdddfb4179aa..bf418191278f 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -720,6 +720,7 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = {
.vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
.vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
.vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_delete_buf = v4l2_m2m_ioctl_delete_buf,
.vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
diff --git a/drivers/media/test-drivers/vim2m.c b/drivers/media/test-drivers/vim2m.c
index 3e3b424b4860..3014b8ee13d0 100644
--- a/drivers/media/test-drivers/vim2m.c
+++ b/drivers/media/test-drivers/vim2m.c
@@ -960,6 +960,7 @@ static const struct v4l2_ioctl_ops vim2m_ioctl_ops = {
.vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
.vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
.vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_delete_buf = v4l2_m2m_ioctl_delete_buf,
.vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
.vidioc_streamon = v4l2_m2m_ioctl_streamon,
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
index 0cc30397fbad..42f51ca25379 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -831,6 +831,17 @@ int v4l2_m2m_prepare_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
}
EXPORT_SYMBOL_GPL(v4l2_m2m_prepare_buf);
+int v4l2_m2m_delete_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
+ struct v4l2_buffer *buf)
+{
+ struct vb2_queue *vq;
+
+ vq = v4l2_m2m_get_vq(m2m_ctx, buf->type);
+
+ return vb2_delete_buf(vq, buf);
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_delete_buf);
+
int v4l2_m2m_create_bufs(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
struct v4l2_create_buffers *create)
{
@@ -1377,6 +1388,15 @@ int v4l2_m2m_ioctl_create_bufs(struct file *file, void *priv,
}
EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_create_bufs);
+int v4l2_m2m_ioctl_delete_buf(struct file *file, void *priv,
+ struct v4l2_buffer *buf)
+{
+ struct v4l2_fh *fh = file->private_data;
+
+ return v4l2_m2m_delete_buf(file, fh->m2m_ctx, buf);
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_delete_buf);
+
int v4l2_m2m_ioctl_querybuf(struct file *file, void *priv,
struct v4l2_buffer *buf)
{
diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h
index bb9de6a899e0..d062a35c8246 100644
--- a/include/media/v4l2-mem2mem.h
+++ b/include/media/v4l2-mem2mem.h
@@ -381,6 +381,16 @@ int v4l2_m2m_dqbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
int v4l2_m2m_prepare_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
struct v4l2_buffer *buf);
+/**
+ * v4l2_m2m_delete_buf() - delete buffer from the queue
+ *
+ * @file: pointer to struct &file
+ * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
+ * @buf: pointer to struct &v4l2_buffer
+ */
+int v4l2_m2m_delete_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
+ struct v4l2_buffer *buf);
+
/**
* v4l2_m2m_create_bufs() - create a source or destination buffer, depending
* on the type
@@ -846,6 +856,8 @@ int v4l2_m2m_ioctl_reqbufs(struct file *file, void *priv,
struct v4l2_requestbuffers *rb);
int v4l2_m2m_ioctl_create_bufs(struct file *file, void *fh,
struct v4l2_create_buffers *create);
+int v4l2_m2m_ioctl_delete_buf(struct file *file, void *priv,
+ struct v4l2_buffer *buf);
int v4l2_m2m_ioctl_querybuf(struct file *file, void *fh,
struct v4l2_buffer *buf);
int v4l2_m2m_ioctl_expbuf(struct file *file, void *fh,
--
2.39.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 183+ messages in thread* [PATCH v3 11/11] media: v4l2: Add mem2mem helpers for DELETE_BUF ioctl
@ 2023-06-22 13:13 ` Benjamin Gaignard
0 siblings, 0 replies; 183+ messages in thread
From: Benjamin Gaignard @ 2023-06-22 13:13 UTC (permalink / raw)
To: mchehab, tfiga, m.szyprowski, ming.qian, ezequiel, p.zabel,
gregkh, hverkuil-cisco, nicolas.dufresne
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, linux-staging, kernel,
Benjamin Gaignard
Create v4l2-mem2mem helpers for VIDIOC_DELETE_BUF ioctl.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../media/platform/verisilicon/hantro_v4l2.c | 1 +
drivers/media/test-drivers/vim2m.c | 1 +
drivers/media/v4l2-core/v4l2-mem2mem.c | 20 +++++++++++++++++++
include/media/v4l2-mem2mem.h | 12 +++++++++++
4 files changed, 34 insertions(+)
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index cdddfb4179aa..bf418191278f 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -720,6 +720,7 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = {
.vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
.vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
.vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_delete_buf = v4l2_m2m_ioctl_delete_buf,
.vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
diff --git a/drivers/media/test-drivers/vim2m.c b/drivers/media/test-drivers/vim2m.c
index 3e3b424b4860..3014b8ee13d0 100644
--- a/drivers/media/test-drivers/vim2m.c
+++ b/drivers/media/test-drivers/vim2m.c
@@ -960,6 +960,7 @@ static const struct v4l2_ioctl_ops vim2m_ioctl_ops = {
.vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
.vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
.vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_delete_buf = v4l2_m2m_ioctl_delete_buf,
.vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
.vidioc_streamon = v4l2_m2m_ioctl_streamon,
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
index 0cc30397fbad..42f51ca25379 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -831,6 +831,17 @@ int v4l2_m2m_prepare_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
}
EXPORT_SYMBOL_GPL(v4l2_m2m_prepare_buf);
+int v4l2_m2m_delete_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
+ struct v4l2_buffer *buf)
+{
+ struct vb2_queue *vq;
+
+ vq = v4l2_m2m_get_vq(m2m_ctx, buf->type);
+
+ return vb2_delete_buf(vq, buf);
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_delete_buf);
+
int v4l2_m2m_create_bufs(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
struct v4l2_create_buffers *create)
{
@@ -1377,6 +1388,15 @@ int v4l2_m2m_ioctl_create_bufs(struct file *file, void *priv,
}
EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_create_bufs);
+int v4l2_m2m_ioctl_delete_buf(struct file *file, void *priv,
+ struct v4l2_buffer *buf)
+{
+ struct v4l2_fh *fh = file->private_data;
+
+ return v4l2_m2m_delete_buf(file, fh->m2m_ctx, buf);
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_delete_buf);
+
int v4l2_m2m_ioctl_querybuf(struct file *file, void *priv,
struct v4l2_buffer *buf)
{
diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h
index bb9de6a899e0..d062a35c8246 100644
--- a/include/media/v4l2-mem2mem.h
+++ b/include/media/v4l2-mem2mem.h
@@ -381,6 +381,16 @@ int v4l2_m2m_dqbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
int v4l2_m2m_prepare_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
struct v4l2_buffer *buf);
+/**
+ * v4l2_m2m_delete_buf() - delete buffer from the queue
+ *
+ * @file: pointer to struct &file
+ * @m2m_ctx: m2m context assigned to the instance given by struct &v4l2_m2m_ctx
+ * @buf: pointer to struct &v4l2_buffer
+ */
+int v4l2_m2m_delete_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
+ struct v4l2_buffer *buf);
+
/**
* v4l2_m2m_create_bufs() - create a source or destination buffer, depending
* on the type
@@ -846,6 +856,8 @@ int v4l2_m2m_ioctl_reqbufs(struct file *file, void *priv,
struct v4l2_requestbuffers *rb);
int v4l2_m2m_ioctl_create_bufs(struct file *file, void *fh,
struct v4l2_create_buffers *create);
+int v4l2_m2m_ioctl_delete_buf(struct file *file, void *priv,
+ struct v4l2_buffer *buf);
int v4l2_m2m_ioctl_querybuf(struct file *file, void *fh,
struct v4l2_buffer *buf);
int v4l2_m2m_ioctl_expbuf(struct file *file, void *fh,
--
2.39.2
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 183+ messages in thread
* Re: [PATCH v3 00/11] Add DELETE_BUF ioctl
2023-06-22 13:13 ` Benjamin Gaignard
(?)
@ 2023-06-27 7:40 ` Hsia-Jun Li
-1 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-27 7:40 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, ming.qian, mchehab, linux-staging,
kernel, p.zabel, ezequiel, m.szyprowski, gregkh, tfiga,
nicolas.dufresne, hverkuil-cisco
On 6/22/23 21:13, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> Unlike when resolution change on keyframes, dynamic resolution change
> on inter frames doesn't allow to do a stream off/on sequence because
> it is need to keep all previous references alive to decode inter frames.
> This constraint have two main problems:
> - more memory consumption.
> - more buffers in use.
> To solve these issue this series introduce DELETE_BUF ioctl and remove
> the 32 buffers limit per queue.
I know the VIDIOC_CREATE_BUFS allows creating a buffer with a different
size than the driver suggests in G_FMT.
But the vb2_ops->queue_setup() could check whether the sizeimages meet
its minimal requirement with the current format.
This enables a problem that the driver need to check the buffer size
before they make a hardware use a buffer from the rdy_queue.
Thinking of such case, we know a AV1 sequence(VP9 or VP8 didn't have a
sequence header) would need a much large buffer for the alternative
reference frame.
Then create one special buffer for the altref, the driver need a
hardware to pick it from the rdy_queue first or it would be a waste to
use it as a regular frame buffer.
Also missing such step would not solve the memory allocation problem.
>
> VP9 conformance tests using fluster give a score of 210/305.
> The 25 resize inter tests (vp90-2-21-resize_inter_* files) are ok
> but require to use postprocessor.
>
> Kernel branch is available here:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.collabora.com_benjamin.gaignard_for-2Dupstream_-2D_commits_remove-5Fvb2-5Fqueue-5Flimit-5Fv3&d=DwIDAg&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=DCpeuc2fAyJ_XUCYsydYOB5ynn0uW4JsFKVbEiXj-6AhZ5d2vm3GkOClPl8cfN9U&s=8whob9PKPu98WlyK6J9DcmFFiDPbwI3ws-nLfWR0oTE&e=
>
> GStreamer branch to use DELETE_BUF ioctl and testing dynamic resolution
> change is here:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commits_VP9-5Fdrc&d=DwIDAg&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=DCpeuc2fAyJ_XUCYsydYOB5ynn0uW4JsFKVbEiXj-6AhZ5d2vm3GkOClPl8cfN9U&s=SEexoIeuXbraR1zvtSkz0MQFGyZSeKQ7Pt6mJoNrS0A&e=
>
> changes in version 3:
> - Use Xarray API to store allocated video buffers.
> - No module parameter to limit the number of buffer per queue.
> - Use Xarray inside Verisilicon driver to store postprocessor buffers
> and remove VB2_MAX_FRAME limit.
> - Allow Versilicon driver to change of resolution while streaming
> - Various fixes the Verisilicon VP9 code to improve fluster score.
>
> changes in version 2:
> - Use a dynamic array and not a list to keep trace of allocated buffers.
> Not use IDR interface because it is marked as deprecated in kernel
> documentation.
> - Add a module parameter to limit the number of buffer per queue.
> - Add DELETE_BUF ioctl and m2m helpers.
>
> Benjamin Gaignard (11):
> media: videobuf2: Access vb2_queue bufs array through helper functions
> media: videobuf2: Use Xarray instead of static buffers array
> media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
> media: videobuf2: Stop define VB2_MAX_FRAME as global
> media: verisilicon: Refactor postprocessor to store more buffers
> media: verisilicon: Store chroma and motion vectors offset
> media: verisilicon: vp9: Use destination buffer height to compute
> chroma offset
> media: verisilicon: postproc: Fix down scale test
> media: verisilicon: vp9: Allow to change resolution while streaming
> media: v4l2: Add DELETE_BUF ioctl
> media: v4l2: Add mem2mem helpers for DELETE_BUF ioctl
>
> .../userspace-api/media/v4l/user-func.rst | 1 +
> .../media/v4l/vidioc-delete-buf.rst | 51 ++++
> .../media/common/videobuf2/videobuf2-core.c | 275 ++++++++++++++----
> .../media/common/videobuf2/videobuf2-v4l2.c | 34 ++-
> drivers/media/platform/amphion/vdec.c | 1 +
> drivers/media/platform/amphion/vpu_dbg.c | 22 +-
> .../platform/mediatek/jpeg/mtk_jpeg_core.c | 6 +-
> .../vcodec/vdec/vdec_vp9_req_lat_if.c | 4 +-
> drivers/media/platform/qcom/venus/hfi.h | 2 +
> drivers/media/platform/st/sti/hva/hva-v4l2.c | 4 +
> drivers/media/platform/verisilicon/hantro.h | 8 +-
> .../platform/verisilicon/hantro_g2_vp9_dec.c | 10 +-
> .../media/platform/verisilicon/hantro_hw.h | 4 +-
> .../platform/verisilicon/hantro_postproc.c | 114 +++++---
> .../media/platform/verisilicon/hantro_v4l2.c | 37 +--
> drivers/media/test-drivers/vim2m.c | 1 +
> drivers/media/test-drivers/visl/visl-dec.c | 28 +-
> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 10 +
> drivers/media/v4l2-core/v4l2-mem2mem.c | 20 ++
> .../staging/media/atomisp/pci/atomisp_ioctl.c | 2 +-
> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 +
> include/media/v4l2-ioctl.h | 4 +
> include/media/v4l2-mem2mem.h | 12 +
> include/media/videobuf2-core.h | 16 +-
> include/media/videobuf2-v4l2.h | 15 +-
> include/uapi/linux/videodev2.h | 2 +
> 27 files changed, 523 insertions(+), 163 deletions(-)
> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>
> --
> 2.39.2
>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 00/11] Add DELETE_BUF ioctl
@ 2023-06-27 7:40 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-27 7:40 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, ming.qian, mchehab, linux-staging,
kernel, p.zabel, ezequiel, m.szyprowski, gregkh, tfiga,
nicolas.dufresne, hverkuil-cisco
On 6/22/23 21:13, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> Unlike when resolution change on keyframes, dynamic resolution change
> on inter frames doesn't allow to do a stream off/on sequence because
> it is need to keep all previous references alive to decode inter frames.
> This constraint have two main problems:
> - more memory consumption.
> - more buffers in use.
> To solve these issue this series introduce DELETE_BUF ioctl and remove
> the 32 buffers limit per queue.
I know the VIDIOC_CREATE_BUFS allows creating a buffer with a different
size than the driver suggests in G_FMT.
But the vb2_ops->queue_setup() could check whether the sizeimages meet
its minimal requirement with the current format.
This enables a problem that the driver need to check the buffer size
before they make a hardware use a buffer from the rdy_queue.
Thinking of such case, we know a AV1 sequence(VP9 or VP8 didn't have a
sequence header) would need a much large buffer for the alternative
reference frame.
Then create one special buffer for the altref, the driver need a
hardware to pick it from the rdy_queue first or it would be a waste to
use it as a regular frame buffer.
Also missing such step would not solve the memory allocation problem.
>
> VP9 conformance tests using fluster give a score of 210/305.
> The 25 resize inter tests (vp90-2-21-resize_inter_* files) are ok
> but require to use postprocessor.
>
> Kernel branch is available here:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.collabora.com_benjamin.gaignard_for-2Dupstream_-2D_commits_remove-5Fvb2-5Fqueue-5Flimit-5Fv3&d=DwIDAg&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=DCpeuc2fAyJ_XUCYsydYOB5ynn0uW4JsFKVbEiXj-6AhZ5d2vm3GkOClPl8cfN9U&s=8whob9PKPu98WlyK6J9DcmFFiDPbwI3ws-nLfWR0oTE&e=
>
> GStreamer branch to use DELETE_BUF ioctl and testing dynamic resolution
> change is here:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commits_VP9-5Fdrc&d=DwIDAg&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=DCpeuc2fAyJ_XUCYsydYOB5ynn0uW4JsFKVbEiXj-6AhZ5d2vm3GkOClPl8cfN9U&s=SEexoIeuXbraR1zvtSkz0MQFGyZSeKQ7Pt6mJoNrS0A&e=
>
> changes in version 3:
> - Use Xarray API to store allocated video buffers.
> - No module parameter to limit the number of buffer per queue.
> - Use Xarray inside Verisilicon driver to store postprocessor buffers
> and remove VB2_MAX_FRAME limit.
> - Allow Versilicon driver to change of resolution while streaming
> - Various fixes the Verisilicon VP9 code to improve fluster score.
>
> changes in version 2:
> - Use a dynamic array and not a list to keep trace of allocated buffers.
> Not use IDR interface because it is marked as deprecated in kernel
> documentation.
> - Add a module parameter to limit the number of buffer per queue.
> - Add DELETE_BUF ioctl and m2m helpers.
>
> Benjamin Gaignard (11):
> media: videobuf2: Access vb2_queue bufs array through helper functions
> media: videobuf2: Use Xarray instead of static buffers array
> media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
> media: videobuf2: Stop define VB2_MAX_FRAME as global
> media: verisilicon: Refactor postprocessor to store more buffers
> media: verisilicon: Store chroma and motion vectors offset
> media: verisilicon: vp9: Use destination buffer height to compute
> chroma offset
> media: verisilicon: postproc: Fix down scale test
> media: verisilicon: vp9: Allow to change resolution while streaming
> media: v4l2: Add DELETE_BUF ioctl
> media: v4l2: Add mem2mem helpers for DELETE_BUF ioctl
>
> .../userspace-api/media/v4l/user-func.rst | 1 +
> .../media/v4l/vidioc-delete-buf.rst | 51 ++++
> .../media/common/videobuf2/videobuf2-core.c | 275 ++++++++++++++----
> .../media/common/videobuf2/videobuf2-v4l2.c | 34 ++-
> drivers/media/platform/amphion/vdec.c | 1 +
> drivers/media/platform/amphion/vpu_dbg.c | 22 +-
> .../platform/mediatek/jpeg/mtk_jpeg_core.c | 6 +-
> .../vcodec/vdec/vdec_vp9_req_lat_if.c | 4 +-
> drivers/media/platform/qcom/venus/hfi.h | 2 +
> drivers/media/platform/st/sti/hva/hva-v4l2.c | 4 +
> drivers/media/platform/verisilicon/hantro.h | 8 +-
> .../platform/verisilicon/hantro_g2_vp9_dec.c | 10 +-
> .../media/platform/verisilicon/hantro_hw.h | 4 +-
> .../platform/verisilicon/hantro_postproc.c | 114 +++++---
> .../media/platform/verisilicon/hantro_v4l2.c | 37 +--
> drivers/media/test-drivers/vim2m.c | 1 +
> drivers/media/test-drivers/visl/visl-dec.c | 28 +-
> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 10 +
> drivers/media/v4l2-core/v4l2-mem2mem.c | 20 ++
> .../staging/media/atomisp/pci/atomisp_ioctl.c | 2 +-
> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 +
> include/media/v4l2-ioctl.h | 4 +
> include/media/v4l2-mem2mem.h | 12 +
> include/media/videobuf2-core.h | 16 +-
> include/media/videobuf2-v4l2.h | 15 +-
> include/uapi/linux/videodev2.h | 2 +
> 27 files changed, 523 insertions(+), 163 deletions(-)
> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>
> --
> 2.39.2
>
--
Hsia-Jun(Randy) Li
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 183+ messages in thread
* Re: [PATCH v3 00/11] Add DELETE_BUF ioctl
@ 2023-06-27 7:40 ` Hsia-Jun Li
0 siblings, 0 replies; 183+ messages in thread
From: Hsia-Jun Li @ 2023-06-27 7:40 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-arm-kernel, linux-mediatek,
linux-arm-msm, linux-rockchip, ming.qian, mchehab, linux-staging,
kernel, p.zabel, ezequiel, m.szyprowski, gregkh, tfiga,
nicolas.dufresne, hverkuil-cisco
On 6/22/23 21:13, Benjamin Gaignard wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> Unlike when resolution change on keyframes, dynamic resolution change
> on inter frames doesn't allow to do a stream off/on sequence because
> it is need to keep all previous references alive to decode inter frames.
> This constraint have two main problems:
> - more memory consumption.
> - more buffers in use.
> To solve these issue this series introduce DELETE_BUF ioctl and remove
> the 32 buffers limit per queue.
I know the VIDIOC_CREATE_BUFS allows creating a buffer with a different
size than the driver suggests in G_FMT.
But the vb2_ops->queue_setup() could check whether the sizeimages meet
its minimal requirement with the current format.
This enables a problem that the driver need to check the buffer size
before they make a hardware use a buffer from the rdy_queue.
Thinking of such case, we know a AV1 sequence(VP9 or VP8 didn't have a
sequence header) would need a much large buffer for the alternative
reference frame.
Then create one special buffer for the altref, the driver need a
hardware to pick it from the rdy_queue first or it would be a waste to
use it as a regular frame buffer.
Also missing such step would not solve the memory allocation problem.
>
> VP9 conformance tests using fluster give a score of 210/305.
> The 25 resize inter tests (vp90-2-21-resize_inter_* files) are ok
> but require to use postprocessor.
>
> Kernel branch is available here:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.collabora.com_benjamin.gaignard_for-2Dupstream_-2D_commits_remove-5Fvb2-5Fqueue-5Flimit-5Fv3&d=DwIDAg&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=DCpeuc2fAyJ_XUCYsydYOB5ynn0uW4JsFKVbEiXj-6AhZ5d2vm3GkOClPl8cfN9U&s=8whob9PKPu98WlyK6J9DcmFFiDPbwI3ws-nLfWR0oTE&e=
>
> GStreamer branch to use DELETE_BUF ioctl and testing dynamic resolution
> change is here:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commits_VP9-5Fdrc&d=DwIDAg&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=DCpeuc2fAyJ_XUCYsydYOB5ynn0uW4JsFKVbEiXj-6AhZ5d2vm3GkOClPl8cfN9U&s=SEexoIeuXbraR1zvtSkz0MQFGyZSeKQ7Pt6mJoNrS0A&e=
>
> changes in version 3:
> - Use Xarray API to store allocated video buffers.
> - No module parameter to limit the number of buffer per queue.
> - Use Xarray inside Verisilicon driver to store postprocessor buffers
> and remove VB2_MAX_FRAME limit.
> - Allow Versilicon driver to change of resolution while streaming
> - Various fixes the Verisilicon VP9 code to improve fluster score.
>
> changes in version 2:
> - Use a dynamic array and not a list to keep trace of allocated buffers.
> Not use IDR interface because it is marked as deprecated in kernel
> documentation.
> - Add a module parameter to limit the number of buffer per queue.
> - Add DELETE_BUF ioctl and m2m helpers.
>
> Benjamin Gaignard (11):
> media: videobuf2: Access vb2_queue bufs array through helper functions
> media: videobuf2: Use Xarray instead of static buffers array
> media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage
> media: videobuf2: Stop define VB2_MAX_FRAME as global
> media: verisilicon: Refactor postprocessor to store more buffers
> media: verisilicon: Store chroma and motion vectors offset
> media: verisilicon: vp9: Use destination buffer height to compute
> chroma offset
> media: verisilicon: postproc: Fix down scale test
> media: verisilicon: vp9: Allow to change resolution while streaming
> media: v4l2: Add DELETE_BUF ioctl
> media: v4l2: Add mem2mem helpers for DELETE_BUF ioctl
>
> .../userspace-api/media/v4l/user-func.rst | 1 +
> .../media/v4l/vidioc-delete-buf.rst | 51 ++++
> .../media/common/videobuf2/videobuf2-core.c | 275 ++++++++++++++----
> .../media/common/videobuf2/videobuf2-v4l2.c | 34 ++-
> drivers/media/platform/amphion/vdec.c | 1 +
> drivers/media/platform/amphion/vpu_dbg.c | 22 +-
> .../platform/mediatek/jpeg/mtk_jpeg_core.c | 6 +-
> .../vcodec/vdec/vdec_vp9_req_lat_if.c | 4 +-
> drivers/media/platform/qcom/venus/hfi.h | 2 +
> drivers/media/platform/st/sti/hva/hva-v4l2.c | 4 +
> drivers/media/platform/verisilicon/hantro.h | 8 +-
> .../platform/verisilicon/hantro_g2_vp9_dec.c | 10 +-
> .../media/platform/verisilicon/hantro_hw.h | 4 +-
> .../platform/verisilicon/hantro_postproc.c | 114 +++++---
> .../media/platform/verisilicon/hantro_v4l2.c | 37 +--
> drivers/media/test-drivers/vim2m.c | 1 +
> drivers/media/test-drivers/visl/visl-dec.c | 28 +-
> drivers/media/v4l2-core/v4l2-dev.c | 1 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 10 +
> drivers/media/v4l2-core/v4l2-mem2mem.c | 20 ++
> .../staging/media/atomisp/pci/atomisp_ioctl.c | 2 +-
> drivers/staging/media/ipu3/ipu3-v4l2.c | 2 +
> include/media/v4l2-ioctl.h | 4 +
> include/media/v4l2-mem2mem.h | 12 +
> include/media/videobuf2-core.h | 16 +-
> include/media/videobuf2-v4l2.h | 15 +-
> include/uapi/linux/videodev2.h | 2 +
> 27 files changed, 523 insertions(+), 163 deletions(-)
> create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst
>
> --
> 2.39.2
>
--
Hsia-Jun(Randy) Li
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 183+ messages in thread