* [RFC v4 00/17] V4L2 Explicit Synchronization support
@ 2017-10-20 21:49 Gustavo Padovan
2017-10-20 21:49 ` [RFC v4 01/17] [media] v4l: create v4l2_event_subscribe_v4l2() Gustavo Padovan
` (16 more replies)
0 siblings, 17 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:49 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
Hi,
I renamed this back to RFC as many things are still under
discussion/open but it integrates all comments received on the
previous round[1].
My main goal now is to fit as many use cases as possible into this
Explicit Synchronization implementation, but first I'd like to recap
some detail of the implementation, as some pieces changed since the last
patchset, that means the explanation on previous cover letters aren't
entirely valid anymore.
Explicit Synchronization allows us to control the synchronization of
shared buffers from userspace by passing fences to the kernel and/or
receiving them from it. Fences passed to the kernel are named in-fences
and the kernel should wait them to signal before using the buffer, i.e.,
queueing it to the driver. On the other side, the kernel can create
out-fences for the buffers it queues to the drivers, out-fences signal
when the driver is finished with buffer, that is, the buffer is ready.
Before we talk about more details, lets differentiate ordering concepts,
there is two orthogonal places in the pipeline where buffers may or may
not be ordered. Inside the drivers and inside vb2 core:
* Ordering in the driver: In this RFC to use out-fences v4l2 drivers
need to guarantee ordering of the buffers inside the driver, that is,
buffers should become done (think calling vb2_buffer_done() here) in
the same order they are queued to the driver. Drivers should set
q->ordered_in_driver to 1 if they can keep ordering. (See patch 08/17)
* Ordering in vb2: Some queues (OUTPUT queues and some m2m devices)
requires that the order we queue buffers to the driver do not change
in relation to the order they were queued from userspace. So that is
marked in the code by q->is_output or q->ordered_in_vb2 and this
implementation can work with both ordered and not-ordered queues at the
vb2 level. (See patch 09/17)
The in-fences are communicated at the VIDIOC_QBUF ioctl using the
V4L2_BUF_FLAG_IN_FENCE buffer flag and the fence_fd field. If an
in-fence needs to be passed to the kernel, fence_fd should be set to the
fence file descriptor number and the V4L2_BUF_FLAG_IN_FENCE should be
set as well. If ordered in vb2 needs to be kept we use fence arrays (see
patch 12 and 13)
Out-fence for a given buffer is enabled if the V4L2_BUF_FLAG_OUT_FENCE
flag is passed in the QBUF() call and then communicated to userspace via
the newly created V4L2_EVENT_OUT_FENCE event. The event is queued for
userspace pick up when the buffer is queued to the driver, or if the
queue is ordered both in the driver and vb2 we send the event right
away. (see patch 14 to 16)
Use cases
---------
I believe use cases like camera app preview, photoshoot, video record,
m2m color converters and scalers (with ordered queues) and output
devices are covered by this RFC, but others like synchronized
Audio/Video transmission doesn't seem to benefit too much from fences.
Talking to Nicolas he expressed that fences are not really useful for
such cases because we still need to sync with the Audio side, so may
be we need wait at least for a start of streaming signal or something
like that. For m2m encoders, we would be in a similar situation
needing to know bytesused beforehand. I want to hear what others think
here.
WIP on V4L2 event
-----------------
As the V4L2 event part is not the crucial part here, the current
changes there to allow install the fence_fd at DQ_EVENT() are to
be considered PoC. Suggestions are welcome. (See patch 4)
Main Changes on v4
-------------------
* Keep ordering in vb2 of buffers queued from userspace with
in-fence attached. This is necessary for OUTPUT and some m2m
queues.
* V4L2_EVENT_BUF_QUEUED event was renamed to
V4L2_EVENT_OUT_FENCE and is only sent when an out-fence
exists.
* For queues that are both ordered in vb2 and in driver the
OUT_FENCE event is sent out immediately without waiting for
the buffer to be queued on the driver
* The out-fence FD is only installed at DQ_EVENT(), but this
implementation needs improvement. See 'Open Questions'
* Refactor slightly event subscription for v4l2 driver, now
there is a v4l2_event_subscribe_v4l2() that will subscribe for
event that all devices need.
* Protect dma_fence_remove_callback() with the fence lock.
Only the most important changes are detailed here, for a full
list of them, see each individual patch.
Open Questions
--------------
* synchronized Audio/Video transmission and m2m encoders use
cases discussed above
* V4L2 event: Improve solution to install the fd and figure how
to clean up a event if userspace don't pick it up.
Test tool
---------
Test tool I've been using can be found at:
https://git.collabora.com/cgit/user/padovan/v4l2-test.git/
Regards,
Gustavo
--
[1] https://lkml.org/lkml/2017/9/7/489
Gustavo Padovan (16):
[media] v4l: create v4l2_event_subscribe_v4l2()
[media] v4l: use v4l2_subscribe_event_v4l2() on vtables
[media] v4l: use v4l2_subscribe_event_v4l2() on drivers
WIP: [media] v4l2: add v4l2_event_queue_fh_with_cb()
[media] v4l: add V4L2_EVENT_OUT_FENCE event
[media] vb2: add .send_out_fence() to notify userspace of out_fence_fd
[media] vivid: assign the specific device to the vb2_queue->dev
[media] vb2: add 'ordered_in_driver' property to queues
[media] vb2: add 'ordered_in_vb2' property to queues
[media] vivid: mark vivid queues as ordered_in_driver
[media] vb2: check earlier if stream can be started
[media] vb2: add explicit fence user API
[media] vb2: add in-fence support to QBUF
[media] vb2: add infrastructure to support out-fences
[media] vb2: add out-fence support to QBUF
[media] v4l: Document explicit synchronization behaviour
Javier Martinez Canillas (1):
[media] vb2: add videobuf2 dma-buf fence helpers
Documentation/media/uapi/v4l/buffer.rst | 15 ++
Documentation/media/uapi/v4l/vidioc-dqevent.rst | 23 ++
Documentation/media/uapi/v4l/vidioc-qbuf.rst | 31 +++
Documentation/media/videodev2.h.rst.exceptions | 1 +
drivers/media/common/saa7146/saa7146_video.c | 4 +-
drivers/media/dvb-frontends/rtl2832_sdr.c | 2 +-
drivers/media/pci/bt8xx/bttv-driver.c | 4 +-
drivers/media/pci/cobalt/cobalt-v4l2.c | 2 +-
drivers/media/pci/cx18/cx18-ioctl.c | 2 +-
drivers/media/pci/cx23885/cx23885-video.c | 2 +-
drivers/media/pci/cx25821/cx25821-video.c | 2 +-
drivers/media/pci/cx88/cx88-blackbird.c | 2 +-
drivers/media/pci/cx88/cx88-video.c | 4 +-
drivers/media/pci/ivtv/ivtv-ioctl.c | 2 +-
drivers/media/pci/meye/meye.c | 2 +-
drivers/media/pci/saa7134/saa7134-empress.c | 2 +-
drivers/media/pci/saa7134/saa7134-video.c | 4 +-
drivers/media/pci/saa7164/saa7164-encoder.c | 2 +-
drivers/media/pci/solo6x10/solo6x10-v4l2.c | 2 +-
drivers/media/pci/sta2x11/sta2x11_vip.c | 2 +-
drivers/media/pci/tw5864/tw5864-video.c | 2 +-
drivers/media/pci/tw68/tw68-video.c | 2 +-
drivers/media/pci/tw686x/tw686x-video.c | 2 +-
drivers/media/pci/zoran/zoran_driver.c | 2 +-
drivers/media/platform/am437x/am437x-vpfe.c | 2 +-
drivers/media/platform/atmel/atmel-isc.c | 2 +-
drivers/media/platform/atmel/atmel-isi.c | 2 +-
drivers/media/platform/coda/coda-common.c | 2 +-
drivers/media/platform/fsl-viu.c | 2 +-
drivers/media/platform/marvell-ccic/mcam-core.c | 2 +-
drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c | 2 +-
drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 2 +-
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 2 +-
drivers/media/platform/pxa_camera.c | 2 +-
drivers/media/platform/qcom/venus/vdec.c | 4 +-
drivers/media/platform/qcom/venus/venc.c | 2 +-
drivers/media/platform/rcar-vin/rcar-v4l2.c | 2 +-
drivers/media/platform/rcar_drif.c | 2 +-
drivers/media/platform/rcar_fdp1.c | 2 +-
drivers/media/platform/rcar_jpu.c | 2 +-
drivers/media/platform/s3c-camif/camif-capture.c | 2 +-
drivers/media/platform/sti/bdisp/bdisp-v4l2.c | 2 +-
drivers/media/platform/sti/hva/hva-v4l2.c | 2 +-
drivers/media/platform/stm32/stm32-dcmi.c | 2 +-
drivers/media/platform/ti-vpe/cal.c | 2 +-
drivers/media/platform/ti-vpe/vpe.c | 2 +-
drivers/media/platform/vim2m.c | 2 +-
drivers/media/platform/vivid/vivid-core.c | 15 +-
drivers/media/platform/vivid/vivid-vid-out.c | 2 +-
drivers/media/radio/dsbr100.c | 2 +-
drivers/media/radio/radio-cadet.c | 2 +-
drivers/media/radio/radio-isa.c | 2 +-
drivers/media/radio/radio-keene.c | 2 +-
drivers/media/radio/radio-ma901.c | 2 +-
drivers/media/radio/radio-miropcm20.c | 2 +-
drivers/media/radio/radio-mr800.c | 2 +-
drivers/media/radio/radio-sf16fmi.c | 2 +-
drivers/media/radio/radio-si476x.c | 2 +-
drivers/media/radio/radio-tea5764.c | 2 +-
drivers/media/radio/radio-tea5777.c | 2 +-
drivers/media/radio/radio-timb.c | 2 +-
drivers/media/radio/si470x/radio-si470x-common.c | 2 +-
drivers/media/radio/si4713/radio-platform-si4713.c | 2 +-
drivers/media/radio/si4713/radio-usb-si4713.c | 2 +-
drivers/media/radio/tea575x.c | 2 +-
drivers/media/usb/airspy/airspy.c | 2 +-
drivers/media/usb/au0828/au0828-video.c | 2 +-
drivers/media/usb/cpia2/cpia2_v4l.c | 4 +-
drivers/media/usb/cx231xx/cx231xx-417.c | 2 +-
drivers/media/usb/cx231xx/cx231xx-video.c | 4 +-
drivers/media/usb/em28xx/em28xx-video.c | 4 +-
drivers/media/usb/go7007/go7007-v4l2.c | 2 +-
drivers/media/usb/gspca/gspca.c | 2 +-
drivers/media/usb/hackrf/hackrf.c | 2 +-
drivers/media/usb/hdpvr/hdpvr-video.c | 2 +-
drivers/media/usb/msi2500/msi2500.c | 2 +-
drivers/media/usb/pwc/pwc-v4l.c | 2 +-
drivers/media/usb/s2255/s2255drv.c | 2 +-
drivers/media/usb/stk1160/stk1160-v4l.c | 2 +-
drivers/media/usb/stkwebcam/stk-webcam.c | 2 +-
drivers/media/usb/tm6000/tm6000-video.c | 4 +-
drivers/media/usb/usbvision/usbvision-video.c | 4 +-
drivers/media/usb/uvc/uvc_v4l2.c | 2 +-
drivers/media/usb/zr364xx/zr364xx.c | 2 +-
drivers/media/v4l2-core/Kconfig | 1 +
drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 4 +-
drivers/media/v4l2-core/v4l2-event.c | 45 +++-
drivers/media/v4l2-core/videobuf2-core.c | 239 +++++++++++++++++++--
drivers/media/v4l2-core/videobuf2-v4l2.c | 84 +++++++-
.../vc04_services/bcm2835-camera/bcm2835-camera.c | 2 +-
include/media/v4l2-event.h | 15 ++
include/media/videobuf2-core.h | 49 ++++-
include/media/videobuf2-fence.h | 48 +++++
include/uapi/linux/videodev2.h | 16 +-
samples/v4l/v4l2-pci-skeleton.c | 2 +-
95 files changed, 643 insertions(+), 125 deletions(-)
create mode 100644 include/media/videobuf2-fence.h
--
2.13.6
^ permalink raw reply [flat|nested] 32+ messages in thread
* [RFC v4 01/17] [media] v4l: create v4l2_event_subscribe_v4l2()
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
@ 2017-10-20 21:49 ` Gustavo Padovan
2017-10-24 12:51 ` Hans Verkuil
2017-10-20 21:49 ` [RFC v4 02/17] [media] v4l: use v4l2_subscribe_event_v4l2() on vtables Gustavo Padovan
` (15 subsequent siblings)
16 siblings, 1 reply; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:49 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
We need a common function to subscribe all the common events in drivers,
so far we had only V4L2_EVENT_CTRL, so such a function wasn't necessary,
but we are about to introduce a new event for the upcoming explicit fences
implementation, thus a common place is needed.
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
drivers/media/v4l2-core/v4l2-event.c | 12 ++++++++++++
include/media/v4l2-event.h | 8 ++++++++
2 files changed, 20 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-event.c b/drivers/media/v4l2-core/v4l2-event.c
index 968c2eb08b5a..313ee9d1f9ee 100644
--- a/drivers/media/v4l2-core/v4l2-event.c
+++ b/drivers/media/v4l2-core/v4l2-event.c
@@ -20,6 +20,7 @@
#include <media/v4l2-dev.h>
#include <media/v4l2-fh.h>
#include <media/v4l2-event.h>
+#include <media/v4l2-ctrls.h>
#include <linux/mm.h>
#include <linux/sched.h>
@@ -354,3 +355,14 @@ int v4l2_src_change_event_subdev_subscribe(struct v4l2_subdev *sd,
return v4l2_src_change_event_subscribe(fh, sub);
}
EXPORT_SYMBOL_GPL(v4l2_src_change_event_subdev_subscribe);
+
+int v4l2_subscribe_event_v4l2(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *sub)
+{
+ switch (sub->type) {
+ case V4L2_EVENT_CTRL:
+ return v4l2_ctrl_subscribe_event(fh, sub);
+ }
+ return -EINVAL;
+}
+EXPORT_SYMBOL(v4l2_subscribe_event_v4l2);
diff --git a/include/media/v4l2-event.h b/include/media/v4l2-event.h
index 6741910c3a18..2b794f2ad824 100644
--- a/include/media/v4l2-event.h
+++ b/include/media/v4l2-event.h
@@ -236,4 +236,12 @@ int v4l2_src_change_event_subscribe(struct v4l2_fh *fh,
int v4l2_src_change_event_subdev_subscribe(struct v4l2_subdev *sd,
struct v4l2_fh *fh,
struct v4l2_event_subscription *sub);
+/**
+ * v4l2_subscribe_event_v4l2 - helper function that subscribe all v4l2 events
+ *
+ * @fh: pointer to struct v4l2_fh
+ * @sub: pointer to &struct v4l2_event_subscription
+ */
+int v4l2_subscribe_event_v4l2(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *sub);
#endif /* V4L2_EVENT_H */
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 02/17] [media] v4l: use v4l2_subscribe_event_v4l2() on vtables
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
2017-10-20 21:49 ` [RFC v4 01/17] [media] v4l: create v4l2_event_subscribe_v4l2() Gustavo Padovan
@ 2017-10-20 21:49 ` Gustavo Padovan
2017-10-20 21:49 ` [RFC v4 03/17] [media] v4l: use v4l2_subscribe_event_v4l2() on drivers Gustavo Padovan
` (14 subsequent siblings)
16 siblings, 0 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:49 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
In all places that we were calling v4l2_ctrl_subscribe event() we now call
v4l2_subscribe_event_v4l2() that embed v4l2_ctrl_subscribe event() and can
be expanded to support more events.
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
drivers/media/common/saa7146/saa7146_video.c | 4 ++--
drivers/media/dvb-frontends/rtl2832_sdr.c | 2 +-
drivers/media/pci/bt8xx/bttv-driver.c | 4 ++--
drivers/media/pci/cx18/cx18-ioctl.c | 2 +-
drivers/media/pci/cx23885/cx23885-video.c | 2 +-
drivers/media/pci/cx25821/cx25821-video.c | 2 +-
drivers/media/pci/cx88/cx88-blackbird.c | 2 +-
drivers/media/pci/cx88/cx88-video.c | 4 ++--
drivers/media/pci/meye/meye.c | 2 +-
drivers/media/pci/saa7134/saa7134-empress.c | 2 +-
drivers/media/pci/saa7134/saa7134-video.c | 4 ++--
drivers/media/pci/saa7164/saa7164-encoder.c | 2 +-
drivers/media/pci/solo6x10/solo6x10-v4l2.c | 2 +-
drivers/media/pci/sta2x11/sta2x11_vip.c | 2 +-
drivers/media/pci/tw68/tw68-video.c | 2 +-
drivers/media/pci/tw686x/tw686x-video.c | 2 +-
drivers/media/pci/zoran/zoran_driver.c | 2 +-
drivers/media/platform/am437x/am437x-vpfe.c | 2 +-
drivers/media/platform/atmel/atmel-isc.c | 2 +-
drivers/media/platform/atmel/atmel-isi.c | 2 +-
drivers/media/platform/fsl-viu.c | 2 +-
drivers/media/platform/marvell-ccic/mcam-core.c | 2 +-
drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c | 2 +-
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 2 +-
drivers/media/platform/pxa_camera.c | 2 +-
drivers/media/platform/qcom/venus/venc.c | 2 +-
drivers/media/platform/rcar_drif.c | 2 +-
drivers/media/platform/rcar_fdp1.c | 2 +-
drivers/media/platform/rcar_jpu.c | 2 +-
drivers/media/platform/s3c-camif/camif-capture.c | 2 +-
drivers/media/platform/sti/bdisp/bdisp-v4l2.c | 2 +-
drivers/media/platform/sti/hva/hva-v4l2.c | 2 +-
drivers/media/platform/stm32/stm32-dcmi.c | 2 +-
drivers/media/platform/ti-vpe/cal.c | 2 +-
drivers/media/platform/ti-vpe/vpe.c | 2 +-
drivers/media/platform/vim2m.c | 2 +-
drivers/media/radio/dsbr100.c | 2 +-
drivers/media/radio/radio-cadet.c | 2 +-
drivers/media/radio/radio-isa.c | 2 +-
drivers/media/radio/radio-keene.c | 2 +-
drivers/media/radio/radio-ma901.c | 2 +-
drivers/media/radio/radio-miropcm20.c | 2 +-
drivers/media/radio/radio-mr800.c | 2 +-
drivers/media/radio/radio-sf16fmi.c | 2 +-
drivers/media/radio/radio-si476x.c | 2 +-
drivers/media/radio/radio-tea5764.c | 2 +-
drivers/media/radio/radio-tea5777.c | 2 +-
drivers/media/radio/radio-timb.c | 2 +-
drivers/media/radio/si470x/radio-si470x-common.c | 2 +-
drivers/media/radio/si4713/radio-platform-si4713.c | 2 +-
drivers/media/radio/si4713/radio-usb-si4713.c | 2 +-
drivers/media/radio/tea575x.c | 2 +-
drivers/media/usb/airspy/airspy.c | 2 +-
drivers/media/usb/au0828/au0828-video.c | 2 +-
drivers/media/usb/cpia2/cpia2_v4l.c | 2 +-
drivers/media/usb/cx231xx/cx231xx-417.c | 2 +-
drivers/media/usb/cx231xx/cx231xx-video.c | 4 ++--
drivers/media/usb/em28xx/em28xx-video.c | 4 ++--
drivers/media/usb/gspca/gspca.c | 2 +-
drivers/media/usb/hackrf/hackrf.c | 2 +-
drivers/media/usb/hdpvr/hdpvr-video.c | 2 +-
drivers/media/usb/msi2500/msi2500.c | 2 +-
drivers/media/usb/pwc/pwc-v4l.c | 2 +-
drivers/media/usb/s2255/s2255drv.c | 2 +-
drivers/media/usb/stk1160/stk1160-v4l.c | 2 +-
drivers/media/usb/stkwebcam/stk-webcam.c | 2 +-
drivers/media/usb/tm6000/tm6000-video.c | 4 ++--
drivers/media/usb/usbvision/usbvision-video.c | 4 ++--
drivers/media/usb/zr364xx/zr364xx.c | 2 +-
drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c | 2 +-
samples/v4l/v4l2-pci-skeleton.c | 2 +-
71 files changed, 79 insertions(+), 79 deletions(-)
diff --git a/drivers/media/common/saa7146/saa7146_video.c b/drivers/media/common/saa7146/saa7146_video.c
index 51eeed830de4..a2b692e84530 100644
--- a/drivers/media/common/saa7146/saa7146_video.c
+++ b/drivers/media/common/saa7146/saa7146_video.c
@@ -1014,7 +1014,7 @@ const struct v4l2_ioctl_ops saa7146_video_ioctl_ops = {
.vidioc_streamon = vidioc_streamon,
.vidioc_streamoff = vidioc_streamoff,
.vidioc_g_parm = vidioc_g_parm,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
@@ -1031,7 +1031,7 @@ const struct v4l2_ioctl_ops saa7146_vbi_ioctl_ops = {
.vidioc_streamon = vidioc_streamon,
.vidioc_streamoff = vidioc_streamoff,
.vidioc_g_parm = vidioc_g_parm,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/dvb-frontends/rtl2832_sdr.c b/drivers/media/dvb-frontends/rtl2832_sdr.c
index c6e78d870ccd..2af05249b65c 100644
--- a/drivers/media/dvb-frontends/rtl2832_sdr.c
+++ b/drivers/media/dvb-frontends/rtl2832_sdr.c
@@ -1236,7 +1236,7 @@ static const struct v4l2_ioctl_ops rtl2832_sdr_ioctl_ops = {
.vidioc_g_frequency = rtl2832_sdr_g_frequency,
.vidioc_s_frequency = rtl2832_sdr_s_frequency,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_log_status = v4l2_ctrl_log_status,
};
diff --git a/drivers/media/pci/bt8xx/bttv-driver.c b/drivers/media/pci/bt8xx/bttv-driver.c
index 227086a2e99c..718be81a20f6 100644
--- a/drivers/media/pci/bt8xx/bttv-driver.c
+++ b/drivers/media/pci/bt8xx/bttv-driver.c
@@ -3187,7 +3187,7 @@ static const struct v4l2_ioctl_ops bttv_ioctl_ops = {
.vidioc_s_frequency = bttv_s_frequency,
.vidioc_log_status = bttv_log_status,
.vidioc_querystd = bttv_querystd,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
#ifdef CONFIG_VIDEO_ADV_DEBUG
.vidioc_g_register = bttv_g_register,
@@ -3369,7 +3369,7 @@ static const struct v4l2_ioctl_ops radio_ioctl_ops = {
.vidioc_s_frequency = bttv_s_frequency,
.vidioc_s_hw_freq_seek = radio_s_hw_freq_seek,
.vidioc_enum_freq_bands = radio_enum_freq_bands,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/pci/cx18/cx18-ioctl.c b/drivers/media/pci/cx18/cx18-ioctl.c
index 80b902b12a78..b0f1dcf9b508 100644
--- a/drivers/media/pci/cx18/cx18-ioctl.c
+++ b/drivers/media/pci/cx18/cx18-ioctl.c
@@ -1115,7 +1115,7 @@ static const struct v4l2_ioctl_ops cx18_ioctl_ops = {
.vidioc_querybuf = cx18_querybuf,
.vidioc_qbuf = cx18_qbuf,
.vidioc_dqbuf = cx18_dqbuf,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/pci/cx23885/cx23885-video.c b/drivers/media/pci/cx23885/cx23885-video.c
index ecc580af0148..940bbae9b284 100644
--- a/drivers/media/pci/cx23885/cx23885-video.c
+++ b/drivers/media/pci/cx23885/cx23885-video.c
@@ -1138,7 +1138,7 @@ static const struct v4l2_ioctl_ops video_ioctl_ops = {
.vidioc_enumaudio = vidioc_enum_audinput,
.vidioc_g_audio = vidioc_g_audinput,
.vidioc_s_audio = vidioc_s_audinput,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/pci/cx25821/cx25821-video.c b/drivers/media/pci/cx25821/cx25821-video.c
index dbaf42ec26cd..9802ae365b08 100644
--- a/drivers/media/pci/cx25821/cx25821-video.c
+++ b/drivers/media/pci/cx25821/cx25821-video.c
@@ -624,7 +624,7 @@ static const struct v4l2_ioctl_ops video_ioctl_ops = {
.vidioc_g_input = cx25821_vidioc_g_input,
.vidioc_s_input = cx25821_vidioc_s_input,
.vidioc_log_status = vidioc_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/pci/cx88/cx88-blackbird.c b/drivers/media/pci/cx88/cx88-blackbird.c
index e3101f04941c..3837f80ea1d5 100644
--- a/drivers/media/pci/cx88/cx88-blackbird.c
+++ b/drivers/media/pci/cx88/cx88-blackbird.c
@@ -1071,7 +1071,7 @@ static const struct v4l2_ioctl_ops mpeg_ioctl_ops = {
.vidioc_s_tuner = vidioc_s_tuner,
.vidioc_g_std = vidioc_g_std,
.vidioc_s_std = vidioc_s_std,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/pci/cx88/cx88-video.c b/drivers/media/pci/cx88/cx88-video.c
index 7d25ecd4404b..64f6f18850fa 100644
--- a/drivers/media/pci/cx88/cx88-video.c
+++ b/drivers/media/pci/cx88/cx88-video.c
@@ -1195,7 +1195,7 @@ static const struct v4l2_ioctl_ops video_ioctl_ops = {
.vidioc_s_tuner = vidioc_s_tuner,
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
#ifdef CONFIG_VIDEO_ADV_DEBUG
.vidioc_g_register = vidioc_g_register,
@@ -1257,7 +1257,7 @@ static const struct v4l2_ioctl_ops radio_ioctl_ops = {
.vidioc_s_tuner = radio_s_tuner,
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
#ifdef CONFIG_VIDEO_ADV_DEBUG
.vidioc_g_register = vidioc_g_register,
diff --git a/drivers/media/pci/meye/meye.c b/drivers/media/pci/meye/meye.c
index 23999a8cef37..2ba588ca5ffa 100644
--- a/drivers/media/pci/meye/meye.c
+++ b/drivers/media/pci/meye/meye.c
@@ -1528,7 +1528,7 @@ static const struct v4l2_ioctl_ops meye_ioctl_ops = {
.vidioc_streamon = vidioc_streamon,
.vidioc_streamoff = vidioc_streamoff,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_default = vidioc_default,
};
diff --git a/drivers/media/pci/saa7134/saa7134-empress.c b/drivers/media/pci/saa7134/saa7134-empress.c
index 66acfd35ffc6..72f84cb8b809 100644
--- a/drivers/media/pci/saa7134/saa7134-empress.c
+++ b/drivers/media/pci/saa7134/saa7134-empress.c
@@ -199,7 +199,7 @@ static const struct v4l2_ioctl_ops ts_ioctl_ops = {
.vidioc_g_std = saa7134_g_std,
.vidioc_querystd = saa7134_querystd,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/pci/saa7134/saa7134-video.c b/drivers/media/pci/saa7134/saa7134-video.c
index 51d42bbf969e..fbe3424d16ad 100644
--- a/drivers/media/pci/saa7134/saa7134-video.c
+++ b/drivers/media/pci/saa7134/saa7134-video.c
@@ -2012,7 +2012,7 @@ static const struct v4l2_ioctl_ops video_ioctl_ops = {
.vidioc_s_register = vidioc_s_register,
#endif
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
@@ -2031,7 +2031,7 @@ static const struct v4l2_ioctl_ops radio_ioctl_ops = {
.vidioc_s_tuner = radio_s_tuner,
.vidioc_g_frequency = saa7134_g_frequency,
.vidioc_s_frequency = saa7134_s_frequency,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/pci/saa7164/saa7164-encoder.c b/drivers/media/pci/saa7164/saa7164-encoder.c
index f21c245a54f7..0009a1132db4 100644
--- a/drivers/media/pci/saa7164/saa7164-encoder.c
+++ b/drivers/media/pci/saa7164/saa7164-encoder.c
@@ -973,7 +973,7 @@ static const struct v4l2_ioctl_ops mpeg_ioctl_ops = {
.vidioc_try_fmt_vid_cap = vidioc_fmt_vid_cap,
.vidioc_s_fmt_vid_cap = vidioc_fmt_vid_cap,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/pci/solo6x10/solo6x10-v4l2.c b/drivers/media/pci/solo6x10/solo6x10-v4l2.c
index 99ffd1ed4a73..5f8b5979ed39 100644
--- a/drivers/media/pci/solo6x10/solo6x10-v4l2.c
+++ b/drivers/media/pci/solo6x10/solo6x10-v4l2.c
@@ -626,7 +626,7 @@ static const struct v4l2_ioctl_ops solo_v4l2_ioctl_ops = {
.vidioc_streamoff = vb2_ioctl_streamoff,
/* Logging and events */
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/pci/sta2x11/sta2x11_vip.c b/drivers/media/pci/sta2x11/sta2x11_vip.c
index eb5a9eae7c8e..33e03dfa62c6 100644
--- a/drivers/media/pci/sta2x11/sta2x11_vip.c
+++ b/drivers/media/pci/sta2x11/sta2x11_vip.c
@@ -750,7 +750,7 @@ static const struct v4l2_ioctl_ops vip_ioctl_ops = {
/* Log status ioctl */
.vidioc_log_status = v4l2_ctrl_log_status,
/* Event handling */
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/pci/tw68/tw68-video.c b/drivers/media/pci/tw68/tw68-video.c
index 8c1f4a049764..8b47a49e3e2c 100644
--- a/drivers/media/pci/tw68/tw68-video.c
+++ b/drivers/media/pci/tw68/tw68-video.c
@@ -908,7 +908,7 @@ static const struct v4l2_ioctl_ops video_ioctl_ops = {
.vidioc_try_fmt_vid_cap = tw68_try_fmt_vid_cap,
.vidioc_s_fmt_vid_cap = tw68_s_fmt_vid_cap,
.vidioc_log_status = vidioc_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
#ifdef CONFIG_VIDEO_ADV_DEBUG
.vidioc_g_register = vidioc_g_register,
diff --git a/drivers/media/pci/tw686x/tw686x-video.c b/drivers/media/pci/tw686x/tw686x-video.c
index c3fafa97b2d0..66dd22cc0515 100644
--- a/drivers/media/pci/tw686x/tw686x-video.c
+++ b/drivers/media/pci/tw686x/tw686x-video.c
@@ -1087,7 +1087,7 @@ static const struct v4l2_ioctl_ops tw686x_video_ioctl_ops = {
.vidioc_prepare_buf = vb2_ioctl_prepare_buf,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/pci/zoran/zoran_driver.c b/drivers/media/pci/zoran/zoran_driver.c
index a11cb501c550..127030f9a955 100644
--- a/drivers/media/pci/zoran/zoran_driver.c
+++ b/drivers/media/pci/zoran/zoran_driver.c
@@ -2826,7 +2826,7 @@ static const struct v4l2_ioctl_ops zoran_ioctl_ops = {
.vidioc_try_fmt_vid_cap = zoran_try_fmt_vid_cap,
.vidioc_try_fmt_vid_out = zoran_try_fmt_vid_out,
.vidioc_try_fmt_vid_overlay = zoran_try_fmt_vid_overlay,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/am437x/am437x-vpfe.c b/drivers/media/platform/am437x/am437x-vpfe.c
index dfcc484cab89..8a71624cf241 100644
--- a/drivers/media/platform/am437x/am437x-vpfe.c
+++ b/drivers/media/platform/am437x/am437x-vpfe.c
@@ -2279,7 +2279,7 @@ static const struct v4l2_ioctl_ops vpfe_ioctl_ops = {
.vidioc_streamoff = vb2_ioctl_streamoff,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_cropcap = vpfe_cropcap,
diff --git a/drivers/media/platform/atmel/atmel-isc.c b/drivers/media/platform/atmel/atmel-isc.c
index 2f8e345d297e..c0d0a66c5b7a 100644
--- a/drivers/media/platform/atmel/atmel-isc.c
+++ b/drivers/media/platform/atmel/atmel-isc.c
@@ -1174,7 +1174,7 @@ static const struct v4l2_ioctl_ops isc_ioctl_ops = {
.vidioc_enum_frameintervals = isc_enum_frameintervals,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/atmel/atmel-isi.c b/drivers/media/platform/atmel/atmel-isi.c
index 463c0146915e..19d6b49d7baa 100644
--- a/drivers/media/platform/atmel/atmel-isi.c
+++ b/drivers/media/platform/atmel/atmel-isi.c
@@ -925,7 +925,7 @@ static const struct v4l2_ioctl_ops isi_ioctl_ops = {
.vidioc_streamoff = vb2_ioctl_streamoff,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/fsl-viu.c b/drivers/media/platform/fsl-viu.c
index fb43025df573..f339f82926a8 100644
--- a/drivers/media/platform/fsl-viu.c
+++ b/drivers/media/platform/fsl-viu.c
@@ -1376,7 +1376,7 @@ static const struct v4l2_ioctl_ops viu_ioctl_ops = {
.vidioc_streamon = vidioc_streamon,
.vidioc_streamoff = vidioc_streamoff,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/marvell-ccic/mcam-core.c b/drivers/media/platform/marvell-ccic/mcam-core.c
index b07a251e8857..ba4855b8763b 100644
--- a/drivers/media/platform/marvell-ccic/mcam-core.c
+++ b/drivers/media/platform/marvell-ccic/mcam-core.c
@@ -1569,7 +1569,7 @@ static const struct v4l2_ioctl_ops mcam_v4l_ioctl_ops = {
.vidioc_s_parm = mcam_vidioc_s_parm,
.vidioc_enum_framesizes = mcam_vidioc_enum_framesizes,
.vidioc_enum_frameintervals = mcam_vidioc_enum_frameintervals,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
#ifdef CONFIG_VIDEO_ADV_DEBUG
.vidioc_g_register = mcam_vidioc_g_register,
diff --git a/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c b/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
index 583d47724ee8..27835f05d415 100644
--- a/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
+++ b/drivers/media/platform/mtk-mdp/mtk_mdp_m2m.c
@@ -970,7 +970,7 @@ static const struct v4l2_ioctl_ops mtk_mdp_m2m_ioctl_ops = {
.vidioc_reqbufs = mtk_mdp_m2m_reqbufs,
.vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
.vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_querybuf = v4l2_m2m_ioctl_querybuf,
.vidioc_qbuf = v4l2_m2m_ioctl_qbuf,
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
index 1b1a28abbf1f..d4d472d2ef40 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c
@@ -732,7 +732,7 @@ const struct v4l2_ioctl_ops mtk_venc_ioctl_ops = {
.vidioc_try_fmt_vid_cap_mplane = vidioc_try_fmt_vid_cap_mplane,
.vidioc_try_fmt_vid_out_mplane = vidioc_try_fmt_vid_out_mplane,
.vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_s_parm = vidioc_venc_s_parm,
diff --git a/drivers/media/platform/pxa_camera.c b/drivers/media/platform/pxa_camera.c
index edca993c2b1f..0d544dd68fd6 100644
--- a/drivers/media/platform/pxa_camera.c
+++ b/drivers/media/platform/pxa_camera.c
@@ -2096,7 +2096,7 @@ static const struct v4l2_ioctl_ops pxa_camera_ioctl_ops = {
.vidioc_g_register = pxac_vidioc_g_register,
.vidioc_s_register = pxac_vidioc_s_register,
#endif
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/venus/venc.c
index 6f123a387cf9..e93e01551acd 100644
--- a/drivers/media/platform/qcom/venus/venc.c
+++ b/drivers/media/platform/qcom/venus/venc.c
@@ -616,7 +616,7 @@ static const struct v4l2_ioctl_ops venc_ioctl_ops = {
.vidioc_g_parm = venc_g_parm,
.vidioc_enum_framesizes = venc_enum_framesizes,
.vidioc_enum_frameintervals = venc_enum_frameintervals,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/rcar_drif.c b/drivers/media/platform/rcar_drif.c
index 2c6afd38b78a..0fdc33132bbd 100644
--- a/drivers/media/platform/rcar_drif.c
+++ b/drivers/media/platform/rcar_drif.c
@@ -1045,7 +1045,7 @@ static const struct v4l2_ioctl_ops rcar_drif_ioctl_ops = {
.vidioc_s_tuner = rcar_drif_s_tuner,
.vidioc_g_tuner = rcar_drif_g_tuner,
.vidioc_enum_freq_bands = rcar_drif_enum_freq_bands,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_log_status = v4l2_ctrl_log_status,
};
diff --git a/drivers/media/platform/rcar_fdp1.c b/drivers/media/platform/rcar_fdp1.c
index 3245bc45f4a0..c567645871d1 100644
--- a/drivers/media/platform/rcar_fdp1.c
+++ b/drivers/media/platform/rcar_fdp1.c
@@ -1754,7 +1754,7 @@ static const struct v4l2_ioctl_ops fdp1_ioctl_ops = {
.vidioc_streamon = v4l2_m2m_ioctl_streamon,
.vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/rcar_jpu.c b/drivers/media/platform/rcar_jpu.c
index 070bac36d766..dc5bf7dcf0b7 100644
--- a/drivers/media/platform/rcar_jpu.c
+++ b/drivers/media/platform/rcar_jpu.c
@@ -972,7 +972,7 @@ static const struct v4l2_ioctl_ops jpu_ioctl_ops = {
.vidioc_streamon = jpu_streamon,
.vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe
};
diff --git a/drivers/media/platform/s3c-camif/camif-capture.c b/drivers/media/platform/s3c-camif/camif-capture.c
index 25c7a7d42292..0ced65d6527f 100644
--- a/drivers/media/platform/s3c-camif/camif-capture.c
+++ b/drivers/media/platform/s3c-camif/camif-capture.c
@@ -1065,7 +1065,7 @@ static const struct v4l2_ioctl_ops s3c_camif_ioctl_ops = {
.vidioc_dqbuf = s3c_camif_dqbuf,
.vidioc_streamon = s3c_camif_streamon,
.vidioc_streamoff = s3c_camif_streamoff,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_log_status = v4l2_ctrl_log_status,
};
diff --git a/drivers/media/platform/sti/bdisp/bdisp-v4l2.c b/drivers/media/platform/sti/bdisp/bdisp-v4l2.c
index 939da6da7644..db5a8d73ec32 100644
--- a/drivers/media/platform/sti/bdisp/bdisp-v4l2.c
+++ b/drivers/media/platform/sti/bdisp/bdisp-v4l2.c
@@ -1042,7 +1042,7 @@ static const struct v4l2_ioctl_ops bdisp_ioctl_ops = {
.vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
.vidioc_streamon = bdisp_streamon,
.vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/sti/hva/hva-v4l2.c b/drivers/media/platform/sti/hva/hva-v4l2.c
index 1c4fc33cbcb5..540570ed3f5a 100644
--- a/drivers/media/platform/sti/hva/hva-v4l2.c
+++ b/drivers/media/platform/sti/hva/hva-v4l2.c
@@ -602,7 +602,7 @@ static const struct v4l2_ioctl_ops hva_ioctl_ops = {
.vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
.vidioc_streamon = v4l2_m2m_ioctl_streamon,
.vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/stm32/stm32-dcmi.c b/drivers/media/platform/stm32/stm32-dcmi.c
index 35ba6f211b79..daffbf97eeaa 100644
--- a/drivers/media/platform/stm32/stm32-dcmi.c
+++ b/drivers/media/platform/stm32/stm32-dcmi.c
@@ -1283,7 +1283,7 @@ static const struct v4l2_ioctl_ops dcmi_ioctl_ops = {
.vidioc_streamoff = vb2_ioctl_streamoff,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 42e383a48ffe..4944bfe50f7b 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -1416,7 +1416,7 @@ static const struct v4l2_ioctl_ops cal_ioctl_ops = {
.vidioc_streamon = vb2_ioctl_streamon,
.vidioc_streamoff = vb2_ioctl_streamoff,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/ti-vpe/vpe.c b/drivers/media/platform/ti-vpe/vpe.c
index 45bd10544189..d73ca22d7871 100644
--- a/drivers/media/platform/ti-vpe/vpe.c
+++ b/drivers/media/platform/ti-vpe/vpe.c
@@ -2011,7 +2011,7 @@ static const struct v4l2_ioctl_ops vpe_ioctl_ops = {
.vidioc_streamon = v4l2_m2m_ioctl_streamon,
.vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/platform/vim2m.c b/drivers/media/platform/vim2m.c
index b01fba020d5f..9a630f3a4ae2 100644
--- a/drivers/media/platform/vim2m.c
+++ b/drivers/media/platform/vim2m.c
@@ -713,7 +713,7 @@ static const struct v4l2_ioctl_ops vim2m_ioctl_ops = {
.vidioc_streamon = v4l2_m2m_ioctl_streamon,
.vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/dsbr100.c b/drivers/media/radio/dsbr100.c
index 8521bb2825e8..b6d96870a4ab 100644
--- a/drivers/media/radio/dsbr100.c
+++ b/drivers/media/radio/dsbr100.c
@@ -337,7 +337,7 @@ static const struct v4l2_ioctl_ops usb_dsbr100_ioctl_ops = {
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/radio-cadet.c b/drivers/media/radio/radio-cadet.c
index 6888b7db449d..9c206783b1fd 100644
--- a/drivers/media/radio/radio-cadet.c
+++ b/drivers/media/radio/radio-cadet.c
@@ -518,7 +518,7 @@ static const struct v4l2_ioctl_ops cadet_ioctl_ops = {
.vidioc_s_frequency = vidioc_s_frequency,
.vidioc_enum_freq_bands = vidioc_enum_freq_bands,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/radio-isa.c b/drivers/media/radio/radio-isa.c
index 7312e469e850..a2b57e532ef0 100644
--- a/drivers/media/radio/radio-isa.c
+++ b/drivers/media/radio/radio-isa.c
@@ -165,7 +165,7 @@ static const struct v4l2_ioctl_ops radio_isa_ioctl_ops = {
.vidioc_g_frequency = radio_isa_g_frequency,
.vidioc_s_frequency = radio_isa_s_frequency,
.vidioc_log_status = radio_isa_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/radio-keene.c b/drivers/media/radio/radio-keene.c
index f2ea8bc5f5ee..83c8b7242d87 100644
--- a/drivers/media/radio/radio-keene.c
+++ b/drivers/media/radio/radio-keene.c
@@ -287,7 +287,7 @@ static const struct v4l2_ioctl_ops usb_keene_ioctl_ops = {
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/radio-ma901.c b/drivers/media/radio/radio-ma901.c
index fdc481257efd..601bfa76550f 100644
--- a/drivers/media/radio/radio-ma901.c
+++ b/drivers/media/radio/radio-ma901.c
@@ -325,7 +325,7 @@ static const struct v4l2_ioctl_ops usb_ma901radio_ioctl_ops = {
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/radio-miropcm20.c b/drivers/media/radio/radio-miropcm20.c
index 7b35e633118d..5f41fdcce53b 100644
--- a/drivers/media/radio/radio-miropcm20.c
+++ b/drivers/media/radio/radio-miropcm20.c
@@ -422,7 +422,7 @@ static const struct v4l2_ioctl_ops pcm20_ioctl_ops = {
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/radio-mr800.c b/drivers/media/radio/radio-mr800.c
index c9f59129af79..7b175f7d6723 100644
--- a/drivers/media/radio/radio-mr800.c
+++ b/drivers/media/radio/radio-mr800.c
@@ -491,7 +491,7 @@ static const struct v4l2_ioctl_ops usb_amradio_ioctl_ops = {
.vidioc_s_frequency = vidioc_s_frequency,
.vidioc_s_hw_freq_seek = vidioc_s_hw_freq_seek,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/radio-sf16fmi.c b/drivers/media/radio/radio-sf16fmi.c
index 28a89466cddc..3b71d0ba28eb 100644
--- a/drivers/media/radio/radio-sf16fmi.c
+++ b/drivers/media/radio/radio-sf16fmi.c
@@ -223,7 +223,7 @@ static const struct v4l2_ioctl_ops fmi_ioctl_ops = {
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/radio-si476x.c b/drivers/media/radio/radio-si476x.c
index 271f725b17e8..40dd19c49abd 100644
--- a/drivers/media/radio/radio-si476x.c
+++ b/drivers/media/radio/radio-si476x.c
@@ -1190,7 +1190,7 @@ static const struct v4l2_ioctl_ops si4761_ioctl_ops = {
.vidioc_s_hw_freq_seek = si476x_radio_s_hw_freq_seek,
.vidioc_enum_freq_bands = si476x_radio_enum_freq_bands,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
#ifdef CONFIG_VIDEO_ADV_DEBUG
diff --git a/drivers/media/radio/radio-tea5764.c b/drivers/media/radio/radio-tea5764.c
index bc7e69e7e32e..5470a609a200 100644
--- a/drivers/media/radio/radio-tea5764.c
+++ b/drivers/media/radio/radio-tea5764.c
@@ -409,7 +409,7 @@ static const struct v4l2_ioctl_ops tea5764_ioctl_ops = {
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/radio-tea5777.c b/drivers/media/radio/radio-tea5777.c
index 04ed1a5d1177..eaa28b63574a 100644
--- a/drivers/media/radio/radio-tea5777.c
+++ b/drivers/media/radio/radio-tea5777.c
@@ -527,7 +527,7 @@ static const struct v4l2_ioctl_ops tea575x_ioctl_ops = {
.vidioc_s_hw_freq_seek = vidioc_s_hw_freq_seek,
.vidioc_enum_freq_bands = vidioc_enum_freq_bands,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/radio-timb.c b/drivers/media/radio/radio-timb.c
index fc4d9a73ab17..bb13e3bc18cc 100644
--- a/drivers/media/radio/radio-timb.c
+++ b/drivers/media/radio/radio-timb.c
@@ -82,7 +82,7 @@ static const struct v4l2_ioctl_ops timbradio_ioctl_ops = {
.vidioc_g_frequency = timbradio_vidioc_g_frequency,
.vidioc_s_frequency = timbradio_vidioc_s_frequency,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/si470x/radio-si470x-common.c b/drivers/media/radio/si470x/radio-si470x-common.c
index cd76facc22f5..3117d3817690 100644
--- a/drivers/media/radio/si470x/radio-si470x-common.c
+++ b/drivers/media/radio/si470x/radio-si470x-common.c
@@ -741,7 +741,7 @@ static const struct v4l2_ioctl_ops si470x_ioctl_ops = {
.vidioc_s_frequency = si470x_vidioc_s_frequency,
.vidioc_s_hw_freq_seek = si470x_vidioc_s_hw_freq_seek,
.vidioc_enum_freq_bands = si470x_vidioc_enum_freq_bands,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/si4713/radio-platform-si4713.c b/drivers/media/radio/si4713/radio-platform-si4713.c
index 27339ec495f6..3fcaf1abac34 100644
--- a/drivers/media/radio/si4713/radio-platform-si4713.c
+++ b/drivers/media/radio/si4713/radio-platform-si4713.c
@@ -129,7 +129,7 @@ static struct v4l2_ioctl_ops radio_si4713_ioctl_ops = {
.vidioc_g_frequency = radio_si4713_g_frequency,
.vidioc_s_frequency = radio_si4713_s_frequency,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_default = radio_si4713_default,
};
diff --git a/drivers/media/radio/si4713/radio-usb-si4713.c b/drivers/media/radio/si4713/radio-usb-si4713.c
index a115db24667b..c02e86728839 100644
--- a/drivers/media/radio/si4713/radio-usb-si4713.c
+++ b/drivers/media/radio/si4713/radio-usb-si4713.c
@@ -127,7 +127,7 @@ static const struct v4l2_ioctl_ops usb_si4713_ioctl_ops = {
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/radio/tea575x.c b/drivers/media/radio/tea575x.c
index 4dc2067bce14..7b2197b2fc8e 100644
--- a/drivers/media/radio/tea575x.c
+++ b/drivers/media/radio/tea575x.c
@@ -493,7 +493,7 @@ static const struct v4l2_ioctl_ops tea575x_ioctl_ops = {
.vidioc_s_hw_freq_seek = vidioc_s_hw_freq_seek,
.vidioc_enum_freq_bands = vidioc_enum_freq_bands,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/airspy/airspy.c b/drivers/media/usb/airspy/airspy.c
index e70c9e2f3798..00b9f16b4357 100644
--- a/drivers/media/usb/airspy/airspy.c
+++ b/drivers/media/usb/airspy/airspy.c
@@ -844,7 +844,7 @@ static const struct v4l2_ioctl_ops airspy_ioctl_ops = {
.vidioc_s_frequency = airspy_s_frequency,
.vidioc_enum_freq_bands = airspy_enum_freq_bands,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_log_status = v4l2_ctrl_log_status,
};
diff --git a/drivers/media/usb/au0828/au0828-video.c b/drivers/media/usb/au0828/au0828-video.c
index 9342402b92f7..f70f1a60b122 100644
--- a/drivers/media/usb/au0828/au0828-video.c
+++ b/drivers/media/usb/au0828/au0828-video.c
@@ -1790,7 +1790,7 @@ static const struct v4l2_ioctl_ops video_ioctl_ops = {
.vidioc_s_register = vidioc_s_register,
#endif
.vidioc_log_status = vidioc_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/cpia2/cpia2_v4l.c b/drivers/media/usb/cpia2/cpia2_v4l.c
index 3dedd83f0b19..4666a642b2f6 100644
--- a/drivers/media/usb/cpia2/cpia2_v4l.c
+++ b/drivers/media/usb/cpia2/cpia2_v4l.c
@@ -1058,7 +1058,7 @@ static const struct v4l2_ioctl_ops cpia2_ioctl_ops = {
.vidioc_g_parm = cpia2_g_parm,
.vidioc_enum_framesizes = cpia2_enum_framesizes,
.vidioc_enum_frameintervals = cpia2_enum_frameintervals,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/cx231xx/cx231xx-417.c b/drivers/media/usb/cx231xx/cx231xx-417.c
index d538fa407742..0c49ad29be9a 100644
--- a/drivers/media/usb/cx231xx/cx231xx-417.c
+++ b/drivers/media/usb/cx231xx/cx231xx-417.c
@@ -1881,7 +1881,7 @@ static const struct v4l2_ioctl_ops mpeg_ioctl_ops = {
.vidioc_g_register = cx231xx_g_register,
.vidioc_s_register = cx231xx_s_register,
#endif
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/cx231xx/cx231xx-video.c b/drivers/media/usb/cx231xx/cx231xx-video.c
index 179b8481a870..51d827d95682 100644
--- a/drivers/media/usb/cx231xx/cx231xx-video.c
+++ b/drivers/media/usb/cx231xx/cx231xx-video.c
@@ -2112,7 +2112,7 @@ static const struct v4l2_ioctl_ops video_ioctl_ops = {
.vidioc_g_register = cx231xx_g_register,
.vidioc_s_register = cx231xx_s_register,
#endif
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
@@ -2144,7 +2144,7 @@ static const struct v4l2_ioctl_ops radio_ioctl_ops = {
.vidioc_g_register = cx231xx_g_register,
.vidioc_s_register = cx231xx_s_register,
#endif
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/em28xx/em28xx-video.c b/drivers/media/usb/em28xx/em28xx-video.c
index 8d253a5df0a9..d6f245c90d35 100644
--- a/drivers/media/usb/em28xx/em28xx-video.c
+++ b/drivers/media/usb/em28xx/em28xx-video.c
@@ -2271,7 +2271,7 @@ static const struct v4l2_ioctl_ops video_ioctl_ops = {
.vidioc_s_tuner = vidioc_s_tuner,
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
#ifdef CONFIG_VIDEO_ADV_DEBUG
.vidioc_g_chip_info = vidioc_g_chip_info,
@@ -2300,7 +2300,7 @@ static const struct v4l2_ioctl_ops radio_ioctl_ops = {
.vidioc_s_tuner = radio_s_tuner,
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
#ifdef CONFIG_VIDEO_ADV_DEBUG
.vidioc_g_chip_info = vidioc_g_chip_info,
diff --git a/drivers/media/usb/gspca/gspca.c b/drivers/media/usb/gspca/gspca.c
index 0f141762abf1..0a15c8d11b75 100644
--- a/drivers/media/usb/gspca/gspca.c
+++ b/drivers/media/usb/gspca/gspca.c
@@ -2000,7 +2000,7 @@ static const struct v4l2_ioctl_ops dev_ioctl_ops = {
.vidioc_g_register = vidioc_g_register,
.vidioc_s_register = vidioc_s_register,
#endif
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/hackrf/hackrf.c b/drivers/media/usb/hackrf/hackrf.c
index 7eb53517a82f..b3e8cb8f24ba 100644
--- a/drivers/media/usb/hackrf/hackrf.c
+++ b/drivers/media/usb/hackrf/hackrf.c
@@ -1247,7 +1247,7 @@ static const struct v4l2_ioctl_ops hackrf_ioctl_ops = {
.vidioc_g_frequency = hackrf_g_frequency,
.vidioc_enum_freq_bands = hackrf_enum_freq_bands,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_log_status = v4l2_ctrl_log_status,
};
diff --git a/drivers/media/usb/hdpvr/hdpvr-video.c b/drivers/media/usb/hdpvr/hdpvr-video.c
index 7fb036d6a86e..95f3650dfb48 100644
--- a/drivers/media/usb/hdpvr/hdpvr-video.c
+++ b/drivers/media/usb/hdpvr/hdpvr-video.c
@@ -1124,7 +1124,7 @@ static const struct v4l2_ioctl_ops hdpvr_ioctl_ops = {
.vidioc_encoder_cmd = vidioc_encoder_cmd,
.vidioc_try_encoder_cmd = vidioc_try_encoder_cmd,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/msi2500/msi2500.c b/drivers/media/usb/msi2500/msi2500.c
index a097d3dbc141..d04165f5c39e 100644
--- a/drivers/media/usb/msi2500/msi2500.c
+++ b/drivers/media/usb/msi2500/msi2500.c
@@ -1128,7 +1128,7 @@ static const struct v4l2_ioctl_ops msi2500_ioctl_ops = {
.vidioc_s_frequency = msi2500_s_frequency,
.vidioc_enum_freq_bands = msi2500_enum_freq_bands,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
.vidioc_log_status = v4l2_ctrl_log_status,
};
diff --git a/drivers/media/usb/pwc/pwc-v4l.c b/drivers/media/usb/pwc/pwc-v4l.c
index 043b2b97cee6..54ce52ade9fe 100644
--- a/drivers/media/usb/pwc/pwc-v4l.c
+++ b/drivers/media/usb/pwc/pwc-v4l.c
@@ -1046,6 +1046,6 @@ const struct v4l2_ioctl_ops pwc_ioctl_ops = {
.vidioc_enum_frameintervals = pwc_enum_frameintervals,
.vidioc_g_parm = pwc_g_parm,
.vidioc_s_parm = pwc_s_parm,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/s2255/s2255drv.c b/drivers/media/usb/s2255/s2255drv.c
index b2f239c4ba42..46d83ad37339 100644
--- a/drivers/media/usb/s2255/s2255drv.c
+++ b/drivers/media/usb/s2255/s2255drv.c
@@ -1570,7 +1570,7 @@ static const struct v4l2_ioctl_ops s2255_ioctl_ops = {
.vidioc_enum_framesizes = vidioc_enum_framesizes,
.vidioc_enum_frameintervals = vidioc_enum_frameintervals,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/stk1160/stk1160-v4l.c b/drivers/media/usb/stk1160/stk1160-v4l.c
index 77b759a0bcd9..ec6cb0bb972c 100644
--- a/drivers/media/usb/stk1160/stk1160-v4l.c
+++ b/drivers/media/usb/stk1160/stk1160-v4l.c
@@ -650,7 +650,7 @@ static const struct v4l2_ioctl_ops stk1160_ioctl_ops = {
.vidioc_expbuf = vb2_ioctl_expbuf,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
#ifdef CONFIG_VIDEO_ADV_DEBUG
diff --git a/drivers/media/usb/stkwebcam/stk-webcam.c b/drivers/media/usb/stkwebcam/stk-webcam.c
index c0bba773db25..7c571ff11065 100644
--- a/drivers/media/usb/stkwebcam/stk-webcam.c
+++ b/drivers/media/usb/stkwebcam/stk-webcam.c
@@ -1230,7 +1230,7 @@ static const struct v4l2_ioctl_ops v4l_stk_ioctl_ops = {
.vidioc_g_parm = stk_vidioc_g_parm,
.vidioc_enum_framesizes = stk_vidioc_enum_framesizes,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/tm6000/tm6000-video.c b/drivers/media/usb/tm6000/tm6000-video.c
index ec8c4d2534dc..4034ae3f3c8a 100644
--- a/drivers/media/usb/tm6000/tm6000-video.c
+++ b/drivers/media/usb/tm6000/tm6000-video.c
@@ -1563,7 +1563,7 @@ static const struct v4l2_ioctl_ops video_ioctl_ops = {
.vidioc_querybuf = vidioc_querybuf,
.vidioc_qbuf = vidioc_qbuf,
.vidioc_dqbuf = vidioc_dqbuf,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
@@ -1589,7 +1589,7 @@ static const struct v4l2_ioctl_ops radio_ioctl_ops = {
.vidioc_s_tuner = radio_s_tuner,
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/usbvision/usbvision-video.c b/drivers/media/usb/usbvision/usbvision-video.c
index 960272d3c924..21870283e35e 100644
--- a/drivers/media/usb/usbvision/usbvision-video.c
+++ b/drivers/media/usb/usbvision/usbvision-video.c
@@ -1182,7 +1182,7 @@ static const struct v4l2_ioctl_ops usbvision_ioctl_ops = {
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
#ifdef CONFIG_VIDEO_ADV_DEBUG
.vidioc_g_register = vidioc_g_register,
@@ -1215,7 +1215,7 @@ static const struct v4l2_ioctl_ops usbvision_radio_ioctl_ops = {
.vidioc_g_frequency = vidioc_g_frequency,
.vidioc_s_frequency = vidioc_s_frequency,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/media/usb/zr364xx/zr364xx.c b/drivers/media/usb/zr364xx/zr364xx.c
index 1d888661fd03..328d77819dea 100644
--- a/drivers/media/usb/zr364xx/zr364xx.c
+++ b/drivers/media/usb/zr364xx/zr364xx.c
@@ -1329,7 +1329,7 @@ static const struct v4l2_ioctl_ops zr364xx_ioctl_ops = {
.vidioc_qbuf = zr364xx_vidioc_qbuf,
.vidioc_dqbuf = zr364xx_vidioc_dqbuf,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c b/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
index be936b8fe317..38dbac531db6 100644
--- a/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
+++ b/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
@@ -1437,7 +1437,7 @@ static const struct v4l2_ioctl_ops camera0_ioctl_ops = {
.vidioc_streamoff = vb2_ioctl_streamoff,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
diff --git a/samples/v4l/v4l2-pci-skeleton.c b/samples/v4l/v4l2-pci-skeleton.c
index f520e3aef9c6..a86cab952791 100644
--- a/samples/v4l/v4l2-pci-skeleton.c
+++ b/samples/v4l/v4l2-pci-skeleton.c
@@ -726,7 +726,7 @@ static const struct v4l2_ioctl_ops skel_ioctl_ops = {
.vidioc_streamoff = vb2_ioctl_streamoff,
.vidioc_log_status = v4l2_ctrl_log_status,
- .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+ .vidioc_subscribe_event = v4l2_subscribe_event_v4l2,
.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
};
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 03/17] [media] v4l: use v4l2_subscribe_event_v4l2() on drivers
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
2017-10-20 21:49 ` [RFC v4 01/17] [media] v4l: create v4l2_event_subscribe_v4l2() Gustavo Padovan
2017-10-20 21:49 ` [RFC v4 02/17] [media] v4l: use v4l2_subscribe_event_v4l2() on vtables Gustavo Padovan
@ 2017-10-20 21:49 ` Gustavo Padovan
2017-10-20 21:49 ` [RFC v4 04/17] WIP: [media] v4l2: add v4l2_event_queue_fh_with_cb() Gustavo Padovan
` (13 subsequent siblings)
16 siblings, 0 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:49 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
Driver that implement their own .vidioc_subscribe_event function should
be using v4l2_subscribe_event_v4l2() instead of
v4l2_ctrl_subscribe_event().
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
drivers/media/pci/cobalt/cobalt-v4l2.c | 2 +-
drivers/media/pci/ivtv/ivtv-ioctl.c | 2 +-
drivers/media/pci/tw5864/tw5864-video.c | 2 +-
drivers/media/platform/coda/coda-common.c | 2 +-
drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c | 2 +-
drivers/media/platform/qcom/venus/vdec.c | 4 +---
drivers/media/platform/rcar-vin/rcar-v4l2.c | 2 +-
drivers/media/platform/vivid/vivid-vid-out.c | 2 +-
drivers/media/usb/go7007/go7007-v4l2.c | 2 +-
drivers/media/usb/uvc/uvc_v4l2.c | 2 +-
10 files changed, 10 insertions(+), 12 deletions(-)
diff --git a/drivers/media/pci/cobalt/cobalt-v4l2.c b/drivers/media/pci/cobalt/cobalt-v4l2.c
index def4a3b37084..7b79daa09416 100644
--- a/drivers/media/pci/cobalt/cobalt-v4l2.c
+++ b/drivers/media/pci/cobalt/cobalt-v4l2.c
@@ -1071,7 +1071,7 @@ static int cobalt_subscribe_event(struct v4l2_fh *fh,
case V4L2_EVENT_SOURCE_CHANGE:
return v4l2_event_subscribe(fh, sub, 4, NULL);
}
- return v4l2_ctrl_subscribe_event(fh, sub);
+ return v4l2_subscribe_event_v4l2(fh, sub);
}
static int cobalt_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
diff --git a/drivers/media/pci/ivtv/ivtv-ioctl.c b/drivers/media/pci/ivtv/ivtv-ioctl.c
index 670462d195b5..4d76a433fcf3 100644
--- a/drivers/media/pci/ivtv/ivtv-ioctl.c
+++ b/drivers/media/pci/ivtv/ivtv-ioctl.c
@@ -1507,7 +1507,7 @@ static int ivtv_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subs
case V4L2_EVENT_EOS:
return v4l2_event_subscribe(fh, sub, 0, NULL);
default:
- return v4l2_ctrl_subscribe_event(fh, sub);
+ return v4l2_subscribe_event_v4l2(fh, sub);
}
}
diff --git a/drivers/media/pci/tw5864/tw5864-video.c b/drivers/media/pci/tw5864/tw5864-video.c
index e7bd2b8484e3..67fa883bed3c 100644
--- a/drivers/media/pci/tw5864/tw5864-video.c
+++ b/drivers/media/pci/tw5864/tw5864-video.c
@@ -671,7 +671,7 @@ static int tw5864_subscribe_event(struct v4l2_fh *fh,
*/
return v4l2_event_subscribe(fh, sub, 30, NULL);
default:
- return v4l2_ctrl_subscribe_event(fh, sub);
+ return v4l2_subscribe_event_v4l2(fh, sub);
}
}
diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 15eb5dc4dff9..897c6939ab0f 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1085,7 +1085,7 @@ static int coda_subscribe_event(struct v4l2_fh *fh,
case V4L2_EVENT_EOS:
return v4l2_event_subscribe(fh, sub, 0, NULL);
default:
- return v4l2_ctrl_subscribe_event(fh, sub);
+ return v4l2_subscribe_event_v4l2(fh, sub);
}
}
diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
index 843510979ad8..a9f80db14b47 100644
--- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
+++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_dec.c
@@ -629,7 +629,7 @@ static int vidioc_vdec_subscribe_evt(struct v4l2_fh *fh,
case V4L2_EVENT_SOURCE_CHANGE:
return v4l2_src_change_event_subscribe(fh, sub);
default:
- return v4l2_ctrl_subscribe_event(fh, sub);
+ return v4l2_subscribe_event_v4l2(fh, sub);
}
}
diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index da611a5eb670..ccf9b778dcf9 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -459,10 +459,8 @@ static int vdec_subscribe_event(struct v4l2_fh *fh,
return v4l2_event_subscribe(fh, sub, 2, NULL);
case V4L2_EVENT_SOURCE_CHANGE:
return v4l2_src_change_event_subscribe(fh, sub);
- case V4L2_EVENT_CTRL:
- return v4l2_ctrl_subscribe_event(fh, sub);
default:
- return -EINVAL;
+ return v4l2_subscribe_event_v4l2(fh, sub);
}
}
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index dd37ea811680..153622e0d42f 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -542,7 +542,7 @@ static int rvin_subscribe_event(struct v4l2_fh *fh,
case V4L2_EVENT_SOURCE_CHANGE:
return v4l2_event_subscribe(fh, sub, 4, NULL);
}
- return v4l2_ctrl_subscribe_event(fh, sub);
+ return v4l2_subscribe_event_v4l2(fh, sub);
}
static int rvin_enum_dv_timings(struct file *file, void *priv_fh,
diff --git a/drivers/media/platform/vivid/vivid-vid-out.c b/drivers/media/platform/vivid/vivid-vid-out.c
index 0b1b6218ede8..f420e9c9d1f6 100644
--- a/drivers/media/platform/vivid/vivid-vid-out.c
+++ b/drivers/media/platform/vivid/vivid-vid-out.c
@@ -1183,7 +1183,7 @@ int vidioc_subscribe_event(struct v4l2_fh *fh,
return v4l2_src_change_event_subscribe(fh, sub);
break;
default:
- return v4l2_ctrl_subscribe_event(fh, sub);
+ return v4l2_subscribe_event_v4l2(fh, sub);
}
return -EINVAL;
}
diff --git a/drivers/media/usb/go7007/go7007-v4l2.c b/drivers/media/usb/go7007/go7007-v4l2.c
index 98cd57eaf36a..a0343a4b4dbc 100644
--- a/drivers/media/usb/go7007/go7007-v4l2.c
+++ b/drivers/media/usb/go7007/go7007-v4l2.c
@@ -797,7 +797,7 @@ static int vidioc_subscribe_event(struct v4l2_fh *fh,
* stored. */
return v4l2_event_subscribe(fh, sub, 30, NULL);
default:
- return v4l2_ctrl_subscribe_event(fh, sub);
+ return v4l2_subscribe_event_v4l2(fh, sub);
}
}
diff --git a/drivers/media/usb/uvc/uvc_v4l2.c b/drivers/media/usb/uvc/uvc_v4l2.c
index 3e7e283a44a8..943c6bb5548b 100644
--- a/drivers/media/usb/uvc/uvc_v4l2.c
+++ b/drivers/media/usb/uvc/uvc_v4l2.c
@@ -1240,7 +1240,7 @@ static int uvc_ioctl_subscribe_event(struct v4l2_fh *fh,
case V4L2_EVENT_CTRL:
return v4l2_event_subscribe(fh, sub, 0, &uvc_ctrl_sub_ev_ops);
default:
- return -EINVAL;
+ return v4l2_subscribe_event_v4l2(fh, sub);
}
}
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 04/17] WIP: [media] v4l2: add v4l2_event_queue_fh_with_cb()
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (2 preceding siblings ...)
2017-10-20 21:49 ` [RFC v4 03/17] [media] v4l: use v4l2_subscribe_event_v4l2() on drivers Gustavo Padovan
@ 2017-10-20 21:49 ` Gustavo Padovan
2017-11-03 7:31 ` Hans Verkuil
2017-10-20 21:50 ` [RFC v4 05/17] [media] v4l: add V4L2_EVENT_OUT_FENCE event Gustavo Padovan
` (12 subsequent siblings)
16 siblings, 1 reply; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:49 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
For some type of events we may require the event user in the
kernel to run some operation when DQ_EVENT() is called.
V4L2_EVENT_OUT_FENCE is the first user of this mechanism as it needs
to call v4l2 core back to install a file descriptor.
This is WIP, I believe we are able to come up with better ways to do it,
but as that is not the main part of the patchset I'll keep it like
this for now.
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
drivers/media/v4l2-core/v4l2-event.c | 31 +++++++++++++++++++++++++++----
include/media/v4l2-event.h | 7 +++++++
2 files changed, 34 insertions(+), 4 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-event.c b/drivers/media/v4l2-core/v4l2-event.c
index 313ee9d1f9ee..6274e3e174e0 100644
--- a/drivers/media/v4l2-core/v4l2-event.c
+++ b/drivers/media/v4l2-core/v4l2-event.c
@@ -58,6 +58,9 @@ static int __v4l2_event_dequeue(struct v4l2_fh *fh, struct v4l2_event *event)
spin_unlock_irqrestore(&fh->vdev->fh_lock, flags);
+ if (kev->prepare)
+ kev->prepare(kev->data);
+
return 0;
}
@@ -104,8 +107,11 @@ static struct v4l2_subscribed_event *v4l2_event_subscribed(
return NULL;
}
-static void __v4l2_event_queue_fh(struct v4l2_fh *fh, const struct v4l2_event *ev,
- const struct timespec *ts)
+static void __v4l2_event_queue_fh_with_cb(struct v4l2_fh *fh,
+ const struct v4l2_event *ev,
+ const struct timespec *ts,
+ void (*prepare)(void *data),
+ void *data)
{
struct v4l2_subscribed_event *sev;
struct v4l2_kevent *kev;
@@ -155,6 +161,8 @@ static void __v4l2_event_queue_fh(struct v4l2_fh *fh, const struct v4l2_event *e
kev->event.id = ev->id;
kev->event.timestamp = *ts;
kev->event.sequence = fh->sequence;
+ kev->prepare = prepare;
+ kev->data = data;
sev->in_use++;
list_add_tail(&kev->list, &fh->available);
@@ -177,7 +185,7 @@ void v4l2_event_queue(struct video_device *vdev, const struct v4l2_event *ev)
spin_lock_irqsave(&vdev->fh_lock, flags);
list_for_each_entry(fh, &vdev->fh_list, list)
- __v4l2_event_queue_fh(fh, ev, ×tamp);
+ __v4l2_event_queue_fh_with_cb(fh, ev, ×tamp, NULL, NULL);
spin_unlock_irqrestore(&vdev->fh_lock, flags);
}
@@ -191,11 +199,26 @@ void v4l2_event_queue_fh(struct v4l2_fh *fh, const struct v4l2_event *ev)
ktime_get_ts(×tamp);
spin_lock_irqsave(&fh->vdev->fh_lock, flags);
- __v4l2_event_queue_fh(fh, ev, ×tamp);
+ __v4l2_event_queue_fh_with_cb(fh, ev, ×tamp, NULL, NULL);
spin_unlock_irqrestore(&fh->vdev->fh_lock, flags);
}
EXPORT_SYMBOL_GPL(v4l2_event_queue_fh);
+void v4l2_event_queue_fh_with_cb(struct v4l2_fh *fh,
+ const struct v4l2_event *ev,
+ void (*prepare)(void *data), void *data)
+{
+ unsigned long flags;
+ struct timespec timestamp;
+
+ ktime_get_ts(×tamp);
+
+ spin_lock_irqsave(&fh->vdev->fh_lock, flags);
+ __v4l2_event_queue_fh_with_cb(fh, ev, ×tamp, prepare, data);
+ spin_unlock_irqrestore(&fh->vdev->fh_lock, flags);
+}
+EXPORT_SYMBOL_GPL(v4l2_event_queue_fh_with_cb);
+
int v4l2_event_pending(struct v4l2_fh *fh)
{
return fh->navailable;
diff --git a/include/media/v4l2-event.h b/include/media/v4l2-event.h
index 2b794f2ad824..dc770257811e 100644
--- a/include/media/v4l2-event.h
+++ b/include/media/v4l2-event.h
@@ -68,11 +68,14 @@ struct video_device;
* @list: List node for the v4l2_fh->available list.
* @sev: Pointer to parent v4l2_subscribed_event.
* @event: The event itself.
+ * @prepare: Callback to prepare the event only at DQ_EVENT() time.
*/
struct v4l2_kevent {
struct list_head list;
struct v4l2_subscribed_event *sev;
struct v4l2_event event;
+ void (*prepare)(void *data);
+ void *data;
};
/**
@@ -160,6 +163,10 @@ void v4l2_event_queue(struct video_device *vdev, const struct v4l2_event *ev);
*/
void v4l2_event_queue_fh(struct v4l2_fh *fh, const struct v4l2_event *ev);
+void v4l2_event_queue_fh_with_cb(struct v4l2_fh *fh,
+ const struct v4l2_event *ev,
+ void (*prepare)(void *data), void *data);
+
/**
* v4l2_event_pending - Check if an event is available
*
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 05/17] [media] v4l: add V4L2_EVENT_OUT_FENCE event
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (3 preceding siblings ...)
2017-10-20 21:49 ` [RFC v4 04/17] WIP: [media] v4l2: add v4l2_event_queue_fh_with_cb() Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-10-24 13:09 ` Hans Verkuil
2017-10-20 21:50 ` [RFC v4 06/17] [media] vb2: add .send_out_fence() to notify userspace of out_fence_fd Gustavo Padovan
` (11 subsequent siblings)
16 siblings, 1 reply; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
Add a new event the userspace can subscribe to receive notifications
of the out_fence_fd when a buffer is queued onto the driver.
The event provides the index of the queued buffer and the out_fence_fd.
v3: - Rename event to V4L2_EVENT_OUT_FENCE
v2: - Add missing Documentation (Mauro)
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
Documentation/media/uapi/v4l/vidioc-dqevent.rst | 23 +++++++++++++++++++++++
Documentation/media/videodev2.h.rst.exceptions | 1 +
include/uapi/linux/videodev2.h | 12 ++++++++++++
3 files changed, 36 insertions(+)
diff --git a/Documentation/media/uapi/v4l/vidioc-dqevent.rst b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
index cb3565f36793..2143df63aeb1 100644
--- a/Documentation/media/uapi/v4l/vidioc-dqevent.rst
+++ b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
@@ -79,6 +79,10 @@ call.
- ``src_change``
- Event data for event V4L2_EVENT_SOURCE_CHANGE.
* -
+ - struct :c:type:`v4l2_event_buf_queued`
+ - ``buf_queued``
+ - Event data for event V4L2_EVENT_OUT_FENCE.
+ * -
- __u8
- ``data``\ [64]
- Event data. Defined by the event type. The union should be used to
@@ -338,6 +342,25 @@ call.
each cell in the motion detection grid, then that all cells are
automatically assigned to the default region 0.
+.. c:type:: v4l2_event_out_fence
+
+.. flat-table:: struct v4l2_event_out_fence
+ :header-rows: 0
+ :stub-columns: 0
+ :widths: 1 1 2
+
+ * - __u32
+ - ``index``
+ - The index of the buffer that was queued to the driver.
+ * - __s32
+ - ``out_fence_fd``
+ - The out-fence file descriptor of the buffer that was queued to
+ the driver. It will signal when the buffer is ready, or if an
+ error happens.
+
+
+
+.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions
index a5cb0a8686ac..32f3d5b37e3a 100644
--- a/Documentation/media/videodev2.h.rst.exceptions
+++ b/Documentation/media/videodev2.h.rst.exceptions
@@ -462,6 +462,7 @@ replace define V4L2_EVENT_CTRL event-type
replace define V4L2_EVENT_FRAME_SYNC event-type
replace define V4L2_EVENT_SOURCE_CHANGE event-type
replace define V4L2_EVENT_MOTION_DET event-type
+replace define V4L2_EVENT_OUT_FENCE event-type
replace define V4L2_EVENT_PRIVATE_START event-type
replace define V4L2_EVENT_CTRL_CH_VALUE ctrl-changes-flags
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 185d6a0acc06..2a432e8c18e3 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -2156,6 +2156,7 @@ struct v4l2_streamparm {
#define V4L2_EVENT_FRAME_SYNC 4
#define V4L2_EVENT_SOURCE_CHANGE 5
#define V4L2_EVENT_MOTION_DET 6
+#define V4L2_EVENT_OUT_FENCE 7
#define V4L2_EVENT_PRIVATE_START 0x08000000
/* Payload for V4L2_EVENT_VSYNC */
@@ -2208,6 +2209,16 @@ struct v4l2_event_motion_det {
__u32 region_mask;
};
+/**
+ * struct v4l2_event_out_fence - out fence fd event
+ * @index: index of the buffer queued in the driver
+ * @out_fence_fd: out-fence fd of the buffer queued
+ */
+struct v4l2_event_out_fence {
+ __u32 index;
+ __s32 out_fence_fd;
+};
+
struct v4l2_event {
__u32 type;
union {
@@ -2216,6 +2227,7 @@ struct v4l2_event {
struct v4l2_event_frame_sync frame_sync;
struct v4l2_event_src_change src_change;
struct v4l2_event_motion_det motion_det;
+ struct v4l2_event_out_fence out_fence;
__u8 data[64];
} u;
__u32 pending;
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 06/17] [media] vb2: add .send_out_fence() to notify userspace of out_fence_fd
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (4 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 05/17] [media] v4l: add V4L2_EVENT_OUT_FENCE event Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 07/17] [media] vivid: assign the specific device to the vb2_queue->dev Gustavo Padovan
` (10 subsequent siblings)
16 siblings, 0 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
With the upcoming explicit synchronization support to V4L2 we need a
way to notify userspace of the out_fence_fd when buffers are queued to the
driver - buffers with in-fences attached to it can only be queued once the
fence signal, so the queueing to the driver might be deferred.
v2: rename to .send_out_fence()
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
include/media/videobuf2-core.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index ef9b64398c8c..96af4eb49e52 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -408,6 +408,7 @@ struct vb2_ops {
* will return an error.
* @copy_timestamp: copy the timestamp from a userspace structure to
* the vb2_buffer struct.
+ * @send_out_fence: send an out_fence fd of the buffer queued to userspace.
*/
struct vb2_buf_ops {
int (*verify_planes_array)(struct vb2_buffer *vb, const void *pb);
@@ -415,6 +416,7 @@ struct vb2_buf_ops {
int (*fill_vb2_buffer)(struct vb2_buffer *vb, const void *pb,
struct vb2_plane *planes);
void (*copy_timestamp)(struct vb2_buffer *vb, const void *pb);
+ void (*send_out_fence)(struct vb2_buffer *vb);
};
/**
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 07/17] [media] vivid: assign the specific device to the vb2_queue->dev
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (5 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 06/17] [media] vb2: add .send_out_fence() to notify userspace of out_fence_fd Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 08/17] [media] vb2: add 'ordered_in_driver' property to queues Gustavo Padovan
` (9 subsequent siblings)
16 siblings, 0 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
Instead of assigning the global v4l2 device, assign the specific device.
This was causing trouble when using V4L2 events with vivid devices.
The device's queue should be the same we opened in userspace.
This is needed for the upcoming V4L2_EVENT_OUT_FENCE support. The current
vivid code isn't wrong, it just needs to be changed so V4L2_EVENT_OUT_FENCE
can be supported. The change doesn't affect any other behaviour of vivid.
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
---
drivers/media/platform/vivid/vivid-core.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/drivers/media/platform/vivid/vivid-core.c b/drivers/media/platform/vivid/vivid-core.c
index 5f316a5e38db..608bcceed463 100644
--- a/drivers/media/platform/vivid/vivid-core.c
+++ b/drivers/media/platform/vivid/vivid-core.c
@@ -1070,7 +1070,7 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
q->min_buffers_needed = 2;
q->lock = &dev->mutex;
- q->dev = dev->v4l2_dev.dev;
+ q->dev = &dev->vid_cap_dev.dev;
ret = vb2_queue_init(q);
if (ret)
@@ -1090,7 +1090,7 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
q->min_buffers_needed = 2;
q->lock = &dev->mutex;
- q->dev = dev->v4l2_dev.dev;
+ q->dev = &dev->vid_out_dev.dev;
ret = vb2_queue_init(q);
if (ret)
@@ -1110,7 +1110,7 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
q->min_buffers_needed = 2;
q->lock = &dev->mutex;
- q->dev = dev->v4l2_dev.dev;
+ q->dev = &dev->vbi_cap_dev.dev;
ret = vb2_queue_init(q);
if (ret)
@@ -1130,7 +1130,7 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
q->min_buffers_needed = 2;
q->lock = &dev->mutex;
- q->dev = dev->v4l2_dev.dev;
+ q->dev = &dev->vbi_out_dev.dev;
ret = vb2_queue_init(q);
if (ret)
@@ -1149,7 +1149,7 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
q->min_buffers_needed = 8;
q->lock = &dev->mutex;
- q->dev = dev->v4l2_dev.dev;
+ q->dev = &dev->sdr_cap_dev.dev;
ret = vb2_queue_init(q);
if (ret)
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 08/17] [media] vb2: add 'ordered_in_driver' property to queues
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (6 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 07/17] [media] vivid: assign the specific device to the vb2_queue->dev Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-10-24 13:11 ` Hans Verkuil
2017-10-20 21:50 ` [RFC v4 09/17] [media] vb2: add 'ordered_in_vb2' " Gustavo Padovan
` (8 subsequent siblings)
16 siblings, 1 reply; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
For explicit synchronization (and soon for HAL3/Request API) we need
the v4l2-driver to guarantee the ordering in which the buffers were queued
by userspace. This is already true for many drivers, but we never needed
to say it.
v2: rename property to 'ordered_in_driver' to avoid confusion
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
include/media/videobuf2-core.h | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 96af4eb49e52..6dd3f0181107 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -500,6 +500,11 @@ struct vb2_buf_ops {
* @last_buffer_dequeued: used in poll() and DQBUF to immediately return if the
* last decoded buffer was already dequeued. Set for capture queues
* when a buffer with the V4L2_BUF_FLAG_LAST is dequeued.
+ * @ordered_in_driver: if the driver can guarantee that the queue will be
+ * ordered or not, i.e., the buffers are queued to the driver in
+ * the same order they are dequeued from the driver. The default
+ * is not ordered unless the driver sets this flag. As of now it
+ * is mandatory for using explicit fences.
* @fileio: file io emulator internal data, used only if emulator is active
* @threadio: thread io internal data, used only if thread is active
*/
@@ -552,6 +557,7 @@ struct vb2_queue {
unsigned int is_output:1;
unsigned int copy_timestamp:1;
unsigned int last_buffer_dequeued:1;
+ unsigned int ordered_in_driver:1;
struct vb2_fileio_data *fileio;
struct vb2_threadio_data *threadio;
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 09/17] [media] vb2: add 'ordered_in_vb2' property to queues
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (7 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 08/17] [media] vb2: add 'ordered_in_driver' property to queues Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-11-03 7:23 ` Hans Verkuil
2017-10-20 21:50 ` [RFC v4 10/17] [media] vivid: mark vivid queues as ordered_in_driver Gustavo Padovan
` (7 subsequent siblings)
16 siblings, 1 reply; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
By setting this member on vb2_queue the driver tell vb2 core that
it requires the buffers queued in QBUF to be queued with same order to the
driver.
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
include/media/videobuf2-core.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 6dd3f0181107..fc333e10e7d8 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -505,6 +505,9 @@ struct vb2_buf_ops {
* the same order they are dequeued from the driver. The default
* is not ordered unless the driver sets this flag. As of now it
* is mandatory for using explicit fences.
+ * @ordered_in_vb2: set by the driver to tell vb2 te guarantee the order
+ * of buffer queue from userspace with QBUF() until they are
+ * queued to the driver.
* @fileio: file io emulator internal data, used only if emulator is active
* @threadio: thread io internal data, used only if thread is active
*/
@@ -558,6 +561,7 @@ struct vb2_queue {
unsigned int copy_timestamp:1;
unsigned int last_buffer_dequeued:1;
unsigned int ordered_in_driver:1;
+ unsigned int ordered_in_vb2:1;
struct vb2_fileio_data *fileio;
struct vb2_threadio_data *threadio;
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 10/17] [media] vivid: mark vivid queues as ordered_in_driver
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (8 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 09/17] [media] vb2: add 'ordered_in_vb2' " Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 11/17] [media] vb2: check earlier if stream can be started Gustavo Padovan
` (6 subsequent siblings)
16 siblings, 0 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
To enable vivid to be used with explicit synchronization we need
to mark its queues as ordered. vivid queues are already ordered by
default so we no changes are needed.
v2: rename 'ordered' to 'ordered_in_driver' to avoid confusion.
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com> (v1)
---
drivers/media/platform/vivid/vivid-core.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/drivers/media/platform/vivid/vivid-core.c b/drivers/media/platform/vivid/vivid-core.c
index 608bcceed463..1f7702faea7d 100644
--- a/drivers/media/platform/vivid/vivid-core.c
+++ b/drivers/media/platform/vivid/vivid-core.c
@@ -1063,6 +1063,7 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
q->type = dev->multiplanar ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE :
V4L2_BUF_TYPE_VIDEO_CAPTURE;
q->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF | VB2_READ;
+ q->ordered_in_driver = 1;
q->drv_priv = dev;
q->buf_struct_size = sizeof(struct vivid_buffer);
q->ops = &vivid_vid_cap_qops;
@@ -1083,6 +1084,7 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
q->type = dev->multiplanar ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE :
V4L2_BUF_TYPE_VIDEO_OUTPUT;
q->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF | VB2_WRITE;
+ q->ordered_in_driver = 1;
q->drv_priv = dev;
q->buf_struct_size = sizeof(struct vivid_buffer);
q->ops = &vivid_vid_out_qops;
@@ -1103,6 +1105,7 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
q->type = dev->has_raw_vbi_cap ? V4L2_BUF_TYPE_VBI_CAPTURE :
V4L2_BUF_TYPE_SLICED_VBI_CAPTURE;
q->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF | VB2_READ;
+ q->ordered_in_driver = 1;
q->drv_priv = dev;
q->buf_struct_size = sizeof(struct vivid_buffer);
q->ops = &vivid_vbi_cap_qops;
@@ -1123,6 +1126,7 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
q->type = dev->has_raw_vbi_out ? V4L2_BUF_TYPE_VBI_OUTPUT :
V4L2_BUF_TYPE_SLICED_VBI_OUTPUT;
q->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF | VB2_WRITE;
+ q->ordered_in_driver = 1;
q->drv_priv = dev;
q->buf_struct_size = sizeof(struct vivid_buffer);
q->ops = &vivid_vbi_out_qops;
@@ -1142,6 +1146,7 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
q = &dev->vb_sdr_cap_q;
q->type = V4L2_BUF_TYPE_SDR_CAPTURE;
q->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF | VB2_READ;
+ q->ordered_in_driver = 1;
q->drv_priv = dev;
q->buf_struct_size = sizeof(struct vivid_buffer);
q->ops = &vivid_sdr_cap_qops;
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 11/17] [media] vb2: check earlier if stream can be started
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (9 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 10/17] [media] vivid: mark vivid queues as ordered_in_driver Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 12/17] [media] vb2: add explicit fence user API Gustavo Padovan
` (5 subsequent siblings)
16 siblings, 0 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
To support explicit synchronization we need to run all operations that can
fail before we queue the buffer to the driver. With fences the queueing
will be delayed if the fence is not signaled yet and it will be better if
such callback do not fail.
For that we move the vb2_start_streaming() before the queuing for the
buffer may happen.
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
drivers/media/v4l2-core/videobuf2-core.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
index cb115ba6a1d2..60f8b582396a 100644
--- a/drivers/media/v4l2-core/videobuf2-core.c
+++ b/drivers/media/v4l2-core/videobuf2-core.c
@@ -1399,29 +1399,27 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
trace_vb2_qbuf(q, vb);
/*
- * If already streaming, give the buffer to driver for processing.
- * If not, the buffer will be given to driver on next streamon.
- */
- if (q->start_streaming_called)
- __enqueue_in_driver(vb);
-
- /* Fill buffer information for the userspace */
- if (pb)
- call_void_bufop(q, fill_user_buffer, vb, pb);
-
- /*
* If streamon has been called, and we haven't yet called
* start_streaming() since not enough buffers were queued, and
* we now have reached the minimum number of queued buffers,
* then we can finally call start_streaming().
+ *
+ * If already streaming, give the buffer to driver for processing.
+ * If not, the buffer will be given to driver on next streamon.
*/
if (q->streaming && !q->start_streaming_called &&
q->queued_count >= q->min_buffers_needed) {
ret = vb2_start_streaming(q);
if (ret)
return ret;
+ } else if (q->start_streaming_called) {
+ __enqueue_in_driver(vb);
}
+ /* Fill buffer information for the userspace */
+ if (pb)
+ call_void_bufop(q, fill_user_buffer, vb, pb);
+
dprintk(2, "qbuf of buffer %d succeeded\n", vb->index);
return 0;
}
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 12/17] [media] vb2: add explicit fence user API
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (10 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 11/17] [media] vb2: check earlier if stream can be started Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-11-03 7:47 ` Hans Verkuil
2017-10-20 21:50 ` [RFC v4 13/17] [media] vb2: add in-fence support to QBUF Gustavo Padovan
` (4 subsequent siblings)
16 siblings, 1 reply; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
Turn the reserved2 field into fence_fd that we will use to send
an in-fence to the kernel and return an out-fence from the kernel to
userspace.
Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
when sending a fence to the kernel to be waited on, and
V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.
v3:
- make the out_fence refer to the current buffer (Hans)
v2: add documentation
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
Documentation/media/uapi/v4l/buffer.rst | 15 +++++++++++++++
drivers/media/usb/cpia2/cpia2_v4l.c | 2 +-
drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 4 ++--
drivers/media/v4l2-core/videobuf2-v4l2.c | 2 +-
include/uapi/linux/videodev2.h | 4 +++-
5 files changed, 22 insertions(+), 5 deletions(-)
diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
index ae6ee73f151c..99afc0837671 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -648,6 +648,21 @@ Buffer Flags
- Start Of Exposure. The buffer timestamp has been taken when the
exposure of the frame has begun. This is only valid for the
``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
+ * .. _`V4L2-BUF-FLAG-IN-FENCE`:
+
+ - ``V4L2_BUF_FLAG_IN_FENCE``
+ - 0x00200000
+ - Ask V4L2 to wait on fence passed in ``fence_fd`` field. The buffer
+ won't be queued to the driver until the fence signals.
+
+ * .. _`V4L2-BUF-FLAG-OUT-FENCE`:
+
+ - ``V4L2_BUF_FLAG_OUT_FENCE``
+ - 0x00400000
+ - Request a fence to be attached to the buffer. One should listen for
+ the ``V4L2_EVENT_OUT_FENCE`` event to receive the buffer ``index``
+ and the ``out_fence_fd`` attached to the buffer. The event is
+ triggered at the moment the buffer is queued to the V4L2 driver.
diff --git a/drivers/media/usb/cpia2/cpia2_v4l.c b/drivers/media/usb/cpia2/cpia2_v4l.c
index 4666a642b2f6..d9d6b6a31751 100644
--- a/drivers/media/usb/cpia2/cpia2_v4l.c
+++ b/drivers/media/usb/cpia2/cpia2_v4l.c
@@ -948,7 +948,7 @@ static int cpia2_dqbuf(struct file *file, void *fh, struct v4l2_buffer *buf)
buf->sequence = cam->buffers[buf->index].seq;
buf->m.offset = cam->buffers[buf->index].data - cam->frame_buffer;
buf->length = cam->frame_size;
- buf->reserved2 = 0;
+ buf->fence_fd = -1;
buf->reserved = 0;
memset(&buf->timecode, 0, sizeof(buf->timecode));
diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
index 821f2aa299ae..3a31d318df2a 100644
--- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
+++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
@@ -370,7 +370,7 @@ struct v4l2_buffer32 {
__s32 fd;
} m;
__u32 length;
- __u32 reserved2;
+ __s32 fence_fd;
__u32 reserved;
};
@@ -533,7 +533,7 @@ static int put_v4l2_buffer32(struct v4l2_buffer *kp, struct v4l2_buffer32 __user
put_user(kp->timestamp.tv_usec, &up->timestamp.tv_usec) ||
copy_to_user(&up->timecode, &kp->timecode, sizeof(struct v4l2_timecode)) ||
put_user(kp->sequence, &up->sequence) ||
- put_user(kp->reserved2, &up->reserved2) ||
+ put_user(kp->fence_fd, &up->fence_fd) ||
put_user(kp->reserved, &up->reserved) ||
put_user(kp->length, &up->length))
return -EFAULT;
diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c b/drivers/media/v4l2-core/videobuf2-v4l2.c
index 0c0669976bdc..110fb45fef6f 100644
--- a/drivers/media/v4l2-core/videobuf2-v4l2.c
+++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
@@ -203,7 +203,7 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
b->timestamp = ns_to_timeval(vb->timestamp);
b->timecode = vbuf->timecode;
b->sequence = vbuf->sequence;
- b->reserved2 = 0;
+ b->fence_fd = -1;
b->reserved = 0;
if (q->is_multiplanar) {
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 2a432e8c18e3..8975fec4ab68 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -924,7 +924,7 @@ struct v4l2_buffer {
__s32 fd;
} m;
__u32 length;
- __u32 reserved2;
+ __s32 fence_fd;
__u32 reserved;
};
@@ -961,6 +961,8 @@ struct v4l2_buffer {
#define V4L2_BUF_FLAG_TSTAMP_SRC_SOE 0x00010000
/* mem2mem encoder/decoder */
#define V4L2_BUF_FLAG_LAST 0x00100000
+#define V4L2_BUF_FLAG_IN_FENCE 0x00200000
+#define V4L2_BUF_FLAG_OUT_FENCE 0x00400000
/**
* struct v4l2_exportbuffer - export of video buffer as DMABUF file descriptor
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 13/17] [media] vb2: add in-fence support to QBUF
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (11 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 12/17] [media] vb2: add explicit fence user API Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-10-25 14:49 ` Brian Starkey
2017-10-20 21:50 ` [RFC v4 14/17] [media] vb2: add videobuf2 dma-buf fence helpers Gustavo Padovan
` (3 subsequent siblings)
16 siblings, 1 reply; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
Receive in-fence from userspace and add support for waiting on them
before queueing the buffer to the driver. Buffers are only queued
to the driver once they are ready. A buffer is ready when its
in-fence signals.
For queues that require vb2 to queue buffers to the v4l2 driver in same
order they are received from userspace we use fence_array to keep that
ordering. Basically we create a fence_array that contains both the current
fence and the fence from the previous buffer (which might be a fence array
as well). The base fence class for the fence_array becomes the new buffer
fence, waiting on that one guarantees that it won't be queued out of
order.
v5: - use fence_array to keep buffers ordered in vb2 core when
needed (Brian Stark)
- keep backward compatibility on the reserved2 field (Brian Stark)
- protect fence callback removal with lock (Brian Stark)
v4:
- Add a comment about dma_fence_add_callback() not returning a
error (Hans)
- Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
- select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
- Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
- Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
- Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
vb2_start_streaming() (Hans)
- set IN_FENCE flags on __fill_v4l2_buffer (Hans)
- Queue buffers to the driver as soon as they are ready (Hans)
- call fill_user_buffer() after queuing the buffer (Hans)
- add err: label to clean up fence
- add dma_fence_wait() before calling vb2_start_streaming()
v3: - document fence parameter
- remove ternary if at vb2_qbuf() return (Mauro)
- do not change if conditions behaviour (Mauro)
v2:
- fix vb2_queue_or_prepare_buf() ret check
- remove check for VB2_MEMORY_DMABUF only (Javier)
- check num of ready buffers to start streaming
- when queueing, start from the first ready buffer
- handle queue cancel
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
drivers/media/v4l2-core/Kconfig | 1 +
drivers/media/v4l2-core/videobuf2-core.c | 179 ++++++++++++++++++++++++++++---
drivers/media/v4l2-core/videobuf2-v4l2.c | 29 ++++-
include/media/videobuf2-core.h | 17 ++-
4 files changed, 208 insertions(+), 18 deletions(-)
diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
index a35c33686abf..3f988c407c80 100644
--- a/drivers/media/v4l2-core/Kconfig
+++ b/drivers/media/v4l2-core/Kconfig
@@ -83,6 +83,7 @@ config VIDEOBUF_DVB
# Used by drivers that need Videobuf2 modules
config VIDEOBUF2_CORE
select DMA_SHARED_BUFFER
+ select SYNC_FILE
tristate
config VIDEOBUF2_MEMOPS
diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
index 60f8b582396a..78f369dba3e3 100644
--- a/drivers/media/v4l2-core/videobuf2-core.c
+++ b/drivers/media/v4l2-core/videobuf2-core.c
@@ -346,6 +346,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
vb->index = q->num_buffers + buffer;
vb->type = q->type;
vb->memory = memory;
+ spin_lock_init(&vb->fence_cb_lock);
for (plane = 0; plane < num_planes; ++plane) {
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
@@ -1222,6 +1223,9 @@ static void __enqueue_in_driver(struct vb2_buffer *vb)
{
struct vb2_queue *q = vb->vb2_queue;
+ if (vb->in_fence && !dma_fence_is_signaled(vb->in_fence))
+ return;
+
vb->state = VB2_BUF_STATE_ACTIVE;
atomic_inc(&q->owned_by_drv_count);
@@ -1273,6 +1277,20 @@ static int __buf_prepare(struct vb2_buffer *vb, const void *pb)
return 0;
}
+static int __get_num_ready_buffers(struct vb2_queue *q)
+{
+ struct vb2_buffer *vb;
+ int ready_count = 0;
+
+ /* count num of buffers ready in front of the queued_list */
+ list_for_each_entry(vb, &q->queued_list, queued_entry) {
+ if (!vb->in_fence || dma_fence_is_signaled(vb->in_fence))
+ ready_count++;
+ }
+
+ return ready_count;
+}
+
int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
{
struct vb2_buffer *vb;
@@ -1361,7 +1379,74 @@ static int vb2_start_streaming(struct vb2_queue *q)
return ret;
}
-int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
+static struct dma_fence *__set_in_fence(struct vb2_queue *q,
+ struct vb2_buffer *vb,
+ struct dma_fence *fence)
+{
+ if (q->last_fence && dma_fence_is_signaled(q->last_fence)) {
+ dma_fence_put(q->last_fence);
+ q->last_fence = NULL;
+ }
+
+ /* For drivers that require the ordering of buffers queued from
+ * userspace to be the same it is queued to the driver - output
+ * some m2m devices - we create a fence array with the fence from
+ * the last queued buffer and this one, that way the fence for this
+ * buffer can't signal before the last one.
+ */
+ if (q->is_output || q->ordered_in_vb2) {
+ if (fence && q->last_fence) {
+ struct dma_fence **fences;
+ struct dma_fence_array *arr;
+
+ fences = kcalloc(2, sizeof(*fences), GFP_KERNEL);
+ if (!fences)
+ return ERR_PTR(-ENOMEM);
+
+ fences[0] = fence;
+ fences[1] = q->last_fence;
+
+ arr = dma_fence_array_create(2, fences,
+ dma_fence_context_alloc(1),
+ 1, false);
+ if (!arr) {
+ kfree(fences);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ fence = &arr->base;
+ } else if (!fence && q->last_fence) {
+ fence = dma_fence_get(q->last_fence);
+ }
+
+ q->last_fence = dma_fence_get(fence);
+ }
+
+ return fence;
+}
+
+static void vb2_qbuf_fence_cb(struct dma_fence *f, struct dma_fence_cb *cb)
+{
+ struct vb2_buffer *vb = container_of(cb, struct vb2_buffer, fence_cb);
+ struct vb2_queue *q = vb->vb2_queue;
+ unsigned long flags;
+
+ spin_lock_irqsave(&vb->fence_cb_lock, flags);
+ if (!vb->in_fence) {
+ spin_unlock_irqrestore(&vb->fence_cb_lock, flags);
+ return;
+ }
+
+ dma_fence_put(vb->in_fence);
+ vb->in_fence = NULL;
+ spin_unlock_irqrestore(&vb->fence_cb_lock, flags);
+
+ if (q->start_streaming_called)
+ __enqueue_in_driver(vb);
+}
+
+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
+ struct dma_fence *fence)
{
struct vb2_buffer *vb;
int ret;
@@ -1372,16 +1457,18 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
case VB2_BUF_STATE_DEQUEUED:
ret = __buf_prepare(vb, pb);
if (ret)
- return ret;
+ goto err;
break;
case VB2_BUF_STATE_PREPARED:
break;
case VB2_BUF_STATE_PREPARING:
dprintk(1, "buffer still being prepared\n");
- return -EINVAL;
+ ret = -EINVAL;
+ goto err;
default:
dprintk(1, "invalid buffer state %d\n", vb->state);
- return -EINVAL;
+ ret = -EINVAL;
+ goto err;
}
/*
@@ -1398,30 +1485,75 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
trace_vb2_qbuf(q, vb);
+ vb->in_fence = __set_in_fence(q, vb, fence);
+ if (IS_ERR(vb->in_fence)) {
+ ret = PTR_ERR(vb->in_fence);
+ goto err;
+ }
+
+ /*
+ * If it is time to call vb2_start_streaming() wait for the fence
+ * to signal first. Of course, this happens only once per streaming.
+ * We want to run any step that might fail before we set the callback
+ * to queue the fence when it signals.
+ */
+ if (fence && !q->start_streaming_called &&
+ __get_num_ready_buffers(q) == q->min_buffers_needed - 1)
+ dma_fence_wait(fence, true);
+
/*
* If streamon has been called, and we haven't yet called
* start_streaming() since not enough buffers were queued, and
* we now have reached the minimum number of queued buffers,
* then we can finally call start_streaming().
- *
- * If already streaming, give the buffer to driver for processing.
- * If not, the buffer will be given to driver on next streamon.
*/
if (q->streaming && !q->start_streaming_called &&
- q->queued_count >= q->min_buffers_needed) {
+ __get_num_ready_buffers(q) >= q->min_buffers_needed) {
ret = vb2_start_streaming(q);
if (ret)
- return ret;
- } else if (q->start_streaming_called) {
- __enqueue_in_driver(vb);
+ goto err;
+ }
+
+ /*
+ * For explicit synchronization: If the fence didn't signal
+ * yet we setup a callback to queue the buffer once the fence
+ * signals, and then, return successfully. But if the fence
+ * already signaled we lose the reference we held and queue the
+ * buffer to the driver.
+ */
+ if (fence) {
+ ret = dma_fence_add_callback(fence, &vb->fence_cb,
+ vb2_qbuf_fence_cb);
+ if (!ret)
+ goto fill;
+
+ dma_fence_put(fence);
+ vb->in_fence = NULL;
}
+fill:
+ /*
+ * If already streaming and there is no fence to wait on
+ * give the buffer to driver for processing.
+ */
+ if (q->start_streaming_called && !vb->in_fence)
+ __enqueue_in_driver(vb);
+
/* Fill buffer information for the userspace */
if (pb)
call_void_bufop(q, fill_user_buffer, vb, pb);
dprintk(2, "qbuf of buffer %d succeeded\n", vb->index);
return 0;
+
+err:
+ if (vb->in_fence) {
+ dma_fence_put(vb->in_fence);
+ vb->in_fence = NULL;
+ }
+
+ return ret;
+
}
EXPORT_SYMBOL_GPL(vb2_core_qbuf);
@@ -1632,6 +1764,8 @@ EXPORT_SYMBOL_GPL(vb2_core_dqbuf);
static void __vb2_queue_cancel(struct vb2_queue *q)
{
unsigned int i;
+ struct vb2_buffer *vb;
+ unsigned long flags;
/*
* Tell driver to stop all transactions and release all queued
@@ -1659,6 +1793,21 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
q->queued_count = 0;
q->error = 0;
+ list_for_each_entry(vb, &q->queued_list, queued_entry) {
+ spin_lock_irqsave(&vb->fence_cb_lock, flags);
+ if (vb->in_fence) {
+ dma_fence_remove_callback(vb->in_fence, &vb->fence_cb);
+ dma_fence_put(vb->in_fence);
+ vb->in_fence = NULL;
+ }
+ spin_unlock_irqrestore(&vb->fence_cb_lock, flags);
+ }
+
+ if (q->last_fence) {
+ dma_fence_put(q->last_fence);
+ q->last_fence = NULL;
+ }
+
/*
* Remove all buffers from videobuf's list...
*/
@@ -1720,7 +1869,7 @@ int vb2_core_streamon(struct vb2_queue *q, unsigned int type)
* Tell driver to start streaming provided sufficient buffers
* are available.
*/
- if (q->queued_count >= q->min_buffers_needed) {
+ if (__get_num_ready_buffers(q) >= q->min_buffers_needed) {
ret = v4l_vb2q_enable_media_source(q);
if (ret)
return ret;
@@ -2240,7 +2389,7 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
* Queue all buffers.
*/
for (i = 0; i < q->num_buffers; i++) {
- ret = vb2_core_qbuf(q, i, NULL);
+ ret = vb2_core_qbuf(q, i, NULL, NULL);
if (ret)
goto err_reqbufs;
fileio->bufs[i].queued = 1;
@@ -2419,7 +2568,7 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
if (copy_timestamp)
b->timestamp = ktime_get_ns();
- ret = vb2_core_qbuf(q, index, NULL);
+ ret = vb2_core_qbuf(q, index, NULL, NULL);
dprintk(5, "vb2_dbuf result: %d\n", ret);
if (ret)
return ret;
@@ -2522,7 +2671,7 @@ static int vb2_thread(void *data)
if (copy_timestamp)
vb->timestamp = ktime_get_ns();;
if (!threadio->stop)
- ret = vb2_core_qbuf(q, vb->index, NULL);
+ ret = vb2_core_qbuf(q, vb->index, NULL, NULL);
call_void_qop(q, wait_prepare, q);
if (ret || threadio->stop)
break;
diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c b/drivers/media/v4l2-core/videobuf2-v4l2.c
index 110fb45fef6f..4c09ea007d90 100644
--- a/drivers/media/v4l2-core/videobuf2-v4l2.c
+++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
@@ -23,6 +23,7 @@
#include <linux/sched.h>
#include <linux/freezer.h>
#include <linux/kthread.h>
+#include <linux/sync_file.h>
#include <media/v4l2-dev.h>
#include <media/v4l2-fh.h>
@@ -178,6 +179,12 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
return -EINVAL;
}
+ if ((b->fence_fd != 0 && b->fence_fd != -1) &&
+ !(b->flags & V4L2_BUF_FLAG_IN_FENCE)) {
+ dprintk(1, "%s: fence_fd set without IN_FENCE flag\n", opname);
+ return -EINVAL;
+ }
+
return __verify_planes_array(q->bufs[b->index], b);
}
@@ -203,9 +210,14 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
b->timestamp = ns_to_timeval(vb->timestamp);
b->timecode = vbuf->timecode;
b->sequence = vbuf->sequence;
- b->fence_fd = -1;
b->reserved = 0;
+ b->fence_fd = -1;
+ if (vb->in_fence)
+ b->flags |= V4L2_BUF_FLAG_IN_FENCE;
+ else
+ b->flags &= ~V4L2_BUF_FLAG_IN_FENCE;
+
if (q->is_multiplanar) {
/*
* Fill in plane-related data if userspace provided an array
@@ -560,6 +572,7 @@ EXPORT_SYMBOL_GPL(vb2_create_bufs);
int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
{
+ struct dma_fence *fence = NULL;
int ret;
if (vb2_fileio_is_active(q)) {
@@ -568,7 +581,19 @@ int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
}
ret = vb2_queue_or_prepare_buf(q, b, "qbuf");
- return ret ? ret : vb2_core_qbuf(q, b->index, b);
+ if (ret)
+ return ret;
+
+ if (b->flags & V4L2_BUF_FLAG_IN_FENCE) {
+ fence = sync_file_get_fence(b->fence_fd);
+ if (!fence) {
+ dprintk(1, "failed to get in-fence from fd %d\n",
+ b->fence_fd);
+ return -EINVAL;
+ }
+ }
+
+ return vb2_core_qbuf(q, b->index, b, fence);
}
EXPORT_SYMBOL_GPL(vb2_qbuf);
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index fc333e10e7d8..624ca2dce9ea 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -16,6 +16,7 @@
#include <linux/mutex.h>
#include <linux/poll.h>
#include <linux/dma-buf.h>
+#include <linux/dma-fence.h>
#define VB2_MAX_FRAME (32)
#define VB2_MAX_PLANES (8)
@@ -254,11 +255,20 @@ struct vb2_buffer {
* all buffers queued from userspace
* done_entry: entry on the list that stores all buffers ready
* to be dequeued to userspace
+ * in_fence: fence receive from vb2 client to wait on before
+ * using the buffer (queueing to the driver)
+ * fence_cb: fence callback information
+ * fence_cb_lock: protect callback signal/remove
*/
enum vb2_buffer_state state;
struct list_head queued_entry;
struct list_head done_entry;
+
+ struct dma_fence *in_fence;
+ struct dma_fence_cb fence_cb;
+ spinlock_t fence_cb_lock;
+
#ifdef CONFIG_VIDEO_ADV_DEBUG
/*
* Counters for how often these buffer-related ops are
@@ -508,6 +518,7 @@ struct vb2_buf_ops {
* @ordered_in_vb2: set by the driver to tell vb2 te guarantee the order
* of buffer queue from userspace with QBUF() until they are
* queued to the driver.
+ * @last_fence: last in-fence received. Used to keep ordering.
* @fileio: file io emulator internal data, used only if emulator is active
* @threadio: thread io internal data, used only if thread is active
*/
@@ -563,6 +574,8 @@ struct vb2_queue {
unsigned int ordered_in_driver:1;
unsigned int ordered_in_vb2:1;
+ struct dma_fence *last_fence;
+
struct vb2_fileio_data *fileio;
struct vb2_threadio_data *threadio;
@@ -738,6 +751,7 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
* @index: id number of the buffer
* @pb: buffer structure passed from userspace to vidioc_qbuf handler
* in driver
+ * @fence: in-fence to wait on before queueing the buffer
*
* Should be called from vidioc_qbuf ioctl handler of a driver.
* The passed buffer should have been verified.
@@ -752,7 +766,8 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
* The return values from this function are intended to be directly returned
* from vidioc_qbuf handler in driver.
*/
-int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb);
+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
+ struct dma_fence *fence);
/**
* vb2_core_dqbuf() - Dequeue a buffer to the userspace
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 14/17] [media] vb2: add videobuf2 dma-buf fence helpers
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (12 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 13/17] [media] vb2: add in-fence support to QBUF Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 15/17] [media] vb2: add infrastructure to support out-fences Gustavo Padovan
` (2 subsequent siblings)
16 siblings, 0 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Javier Martinez Canillas
From: Javier Martinez Canillas <javier@osg.samsung.com>
Add a videobuf2-fence.h header file that contains different helpers
for DMA buffer sharing explicit fence support in videobuf2.
v2: - use fence context provided by the caller in vb2_fence_alloc()
Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
include/media/videobuf2-fence.h | 48 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
create mode 100644 include/media/videobuf2-fence.h
diff --git a/include/media/videobuf2-fence.h b/include/media/videobuf2-fence.h
new file mode 100644
index 000000000000..b49cc1bf6bb4
--- /dev/null
+++ b/include/media/videobuf2-fence.h
@@ -0,0 +1,48 @@
+/*
+ * videobuf2-fence.h - DMA buffer sharing fence helpers for videobuf 2
+ *
+ * Copyright (C) 2016 Samsung Electronics
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation.
+ */
+
+#include <linux/dma-fence.h>
+#include <linux/slab.h>
+
+static DEFINE_SPINLOCK(vb2_fence_lock);
+
+static inline const char *vb2_fence_get_driver_name(struct dma_fence *fence)
+{
+ return "vb2_fence";
+}
+
+static inline const char *vb2_fence_get_timeline_name(struct dma_fence *fence)
+{
+ return "vb2_fence_timeline";
+}
+
+static inline bool vb2_fence_enable_signaling(struct dma_fence *fence)
+{
+ return true;
+}
+
+static const struct dma_fence_ops vb2_fence_ops = {
+ .get_driver_name = vb2_fence_get_driver_name,
+ .get_timeline_name = vb2_fence_get_timeline_name,
+ .enable_signaling = vb2_fence_enable_signaling,
+ .wait = dma_fence_default_wait,
+};
+
+static inline struct dma_fence *vb2_fence_alloc(u64 context)
+{
+ struct dma_fence *vb2_fence = kzalloc(sizeof(*vb2_fence), GFP_KERNEL);
+
+ if (!vb2_fence)
+ return NULL;
+
+ dma_fence_init(vb2_fence, &vb2_fence_ops, &vb2_fence_lock, context, 1);
+
+ return vb2_fence;
+}
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 15/17] [media] vb2: add infrastructure to support out-fences
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (13 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 14/17] [media] vb2: add videobuf2 dma-buf fence helpers Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-10-27 9:41 ` Brian Starkey
2017-10-20 21:50 ` [RFC v4 16/17] [media] vb2: add out-fence support to QBUF Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 17/17] [media] v4l: Document explicit synchronization behaviour Gustavo Padovan
16 siblings, 1 reply; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
Add vb2_setup_out_fence() and the needed members to struct vb2_buffer.
v2: - change it to reflect fd_install at DQEVENT
- add fence context for out-fences
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
drivers/media/v4l2-core/videobuf2-core.c | 31 +++++++++++++++++++++++++++++++
include/media/videobuf2-core.h | 20 ++++++++++++++++++++
2 files changed, 51 insertions(+)
diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
index 78f369dba3e3..c7ba67bda5ac 100644
--- a/drivers/media/v4l2-core/videobuf2-core.c
+++ b/drivers/media/v4l2-core/videobuf2-core.c
@@ -23,8 +23,11 @@
#include <linux/sched.h>
#include <linux/freezer.h>
#include <linux/kthread.h>
+#include <linux/sync_file.h>
+#include <linux/dma-fence.h>
#include <media/videobuf2-core.h>
+#include <media/videobuf2-fence.h>
#include <media/v4l2-mc.h>
#include <trace/events/vb2.h>
@@ -1316,6 +1319,34 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
}
EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
+int vb2_setup_out_fence(struct vb2_queue *q, unsigned int index)
+{
+ struct vb2_buffer *vb;
+ struct dma_fence *fence;
+
+ vb = q->bufs[index];
+
+ vb->out_fence_fd = get_unused_fd_flags(O_CLOEXEC);
+
+ fence = vb2_fence_alloc(q->out_fence_context);
+ if (!fence) {
+ put_unused_fd(vb->out_fence_fd);
+ return -ENOMEM;
+ }
+
+ vb->sync_file = sync_file_create(fence);
+ if (!vb->sync_file) {
+ dma_fence_put(fence);
+ put_unused_fd(vb->out_fence_fd);
+ return -ENOMEM;
+ }
+
+ vb->out_fence = dma_fence_get(fence);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(vb2_setup_out_fence);
+
/**
* vb2_start_streaming() - Attempt to start streaming.
* @q: videobuf2 queue
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 624ca2dce9ea..1925ede73804 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -259,6 +259,10 @@ struct vb2_buffer {
* using the buffer (queueing to the driver)
* fence_cb: fence callback information
* fence_cb_lock: protect callback signal/remove
+ * out_fence_fd: the out_fence_fd to be shared with userspace.
+ * out_fence: the out-fence associated with the buffer once
+ * it is queued to the driver.
+ * sync_file: the sync file to wrap the out fence
*/
enum vb2_buffer_state state;
@@ -269,6 +273,10 @@ struct vb2_buffer {
struct dma_fence_cb fence_cb;
spinlock_t fence_cb_lock;
+ int out_fence_fd;
+ struct dma_fence *out_fence;
+ struct sync_file *sync_file;
+
#ifdef CONFIG_VIDEO_ADV_DEBUG
/*
* Counters for how often these buffer-related ops are
@@ -518,6 +526,7 @@ struct vb2_buf_ops {
* @ordered_in_vb2: set by the driver to tell vb2 te guarantee the order
* of buffer queue from userspace with QBUF() until they are
* queued to the driver.
+ * @out_fence_context: the fence context for the out fences
* @last_fence: last in-fence received. Used to keep ordering.
* @fileio: file io emulator internal data, used only if emulator is active
* @threadio: thread io internal data, used only if thread is active
@@ -574,6 +583,7 @@ struct vb2_queue {
unsigned int ordered_in_driver:1;
unsigned int ordered_in_vb2:1;
+ u64 out_fence_context;
struct dma_fence *last_fence;
struct vb2_fileio_data *fileio;
@@ -745,6 +755,16 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
/**
+ * vb2_setup_out_fence() - setup new out-fence
+ * @q: The vb2_queue where to setup it
+ * @index: index of the buffer
+ *
+ * Setup the file descriptor, the fence and the sync_file for the next
+ * buffer to be queued and add everything to the tail of the q->out_fence_list.
+ */
+int vb2_setup_out_fence(struct vb2_queue *q, unsigned int index);
+
+/**
* vb2_core_qbuf() - Queue a buffer from userspace
*
* @q: videobuf2 queue
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 16/17] [media] vb2: add out-fence support to QBUF
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (14 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 15/17] [media] vb2: add infrastructure to support out-fences Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-10-27 10:01 ` Brian Starkey
2017-10-20 21:50 ` [RFC v4 17/17] [media] v4l: Document explicit synchronization behaviour Gustavo Padovan
16 siblings, 1 reply; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
an out_fence and send to userspace on the V4L2_EVENT_OUT_FENCE when
the buffer is queued to the driver, or right away if the queue is ordered
both in VB2 and in the driver.
The fence is signaled on buffer_done(), when the job on the buffer is
finished.
v5:
- delay fd_install to DQ_EVENT (Hans)
- if queue is fully ordered send OUT_FENCE event right away
(Brian)
- rename 'q->ordered' to 'q->ordered_in_driver'
- merge change to implement OUT_FENCE event here
v4:
- return the out_fence_fd in the BUF_QUEUED event(Hans)
v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
- set the OUT_FENCE flag if there is a fence pending (Hans)
- call fd_install() after vb2_core_qbuf() (Hans)
- clean up fence if vb2_core_qbuf() fails (Hans)
- add list to store sync_file and fence for the next queued buffer
v2: check if the queue is ordered.
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
drivers/media/v4l2-core/v4l2-event.c | 2 ++
drivers/media/v4l2-core/videobuf2-core.c | 25 +++++++++++++++
drivers/media/v4l2-core/videobuf2-v4l2.c | 55 ++++++++++++++++++++++++++++++++
3 files changed, 82 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-event.c b/drivers/media/v4l2-core/v4l2-event.c
index 6274e3e174e0..275da224ace4 100644
--- a/drivers/media/v4l2-core/v4l2-event.c
+++ b/drivers/media/v4l2-core/v4l2-event.c
@@ -385,6 +385,8 @@ int v4l2_subscribe_event_v4l2(struct v4l2_fh *fh,
switch (sub->type) {
case V4L2_EVENT_CTRL:
return v4l2_ctrl_subscribe_event(fh, sub);
+ case V4L2_EVENT_OUT_FENCE:
+ return v4l2_event_subscribe(fh, sub, VIDEO_MAX_FRAME, NULL);
}
return -EINVAL;
}
diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
index c7ba67bda5ac..21e2052776c1 100644
--- a/drivers/media/v4l2-core/videobuf2-core.c
+++ b/drivers/media/v4l2-core/videobuf2-core.c
@@ -354,6 +354,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
vb->planes[plane].length = plane_sizes[plane];
vb->planes[plane].min_length = plane_sizes[plane];
}
+ vb->out_fence_fd = -1;
q->bufs[vb->index] = vb;
/* Allocate video buffer memory for the MMAP type */
@@ -934,10 +935,24 @@ void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
case VB2_BUF_STATE_QUEUED:
return;
case VB2_BUF_STATE_REQUEUEING:
+ /*
+ * Explicit synchronization requires ordered queues for now,
+ * so WARN_ON if we are requeuing on an ordered queue.
+ */
+ if (vb->out_fence)
+ WARN_ON_ONCE(q->ordered_in_driver);
+
if (q->start_streaming_called)
__enqueue_in_driver(vb);
return;
default:
+ if (state == VB2_BUF_STATE_ERROR)
+ dma_fence_set_error(vb->out_fence, -ENOENT);
+ dma_fence_signal(vb->out_fence);
+ dma_fence_put(vb->out_fence);
+ vb->out_fence = NULL;
+ vb->out_fence_fd = -1;
+
/* Inform any processes that may be waiting for buffers */
wake_up(&q->done_wq);
break;
@@ -1235,6 +1250,9 @@ static void __enqueue_in_driver(struct vb2_buffer *vb)
trace_vb2_buf_queue(q, vb);
call_void_vb_qop(vb, buf_queue, vb);
+
+ if (!(q->is_output || q->ordered_in_vb2))
+ call_void_bufop(q, send_out_fence, vb);
}
static int __buf_prepare(struct vb2_buffer *vb, const void *pb)
@@ -1451,6 +1469,7 @@ static struct dma_fence *__set_in_fence(struct vb2_queue *q,
}
q->last_fence = dma_fence_get(fence);
+ call_void_bufop(q, send_out_fence, vb);
}
return fence;
@@ -1840,6 +1859,11 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
}
/*
+ * Renew out-fence context.
+ */
+ q->out_fence_context = dma_fence_context_alloc(1);
+
+ /*
* Remove all buffers from videobuf's list...
*/
INIT_LIST_HEAD(&q->queued_list);
@@ -2171,6 +2195,7 @@ int vb2_core_queue_init(struct vb2_queue *q)
spin_lock_init(&q->done_lock);
mutex_init(&q->mmap_lock);
init_waitqueue_head(&q->done_wq);
+ q->out_fence_context = dma_fence_context_alloc(1);
if (q->buf_struct_size == 0)
q->buf_struct_size = sizeof(struct vb2_buffer);
diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c b/drivers/media/v4l2-core/videobuf2-v4l2.c
index 4c09ea007d90..9fb01ddefdc9 100644
--- a/drivers/media/v4l2-core/videobuf2-v4l2.c
+++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
@@ -32,6 +32,11 @@
#include <media/videobuf2-v4l2.h>
+struct out_fence_data {
+ int fence_fd;
+ struct file *file;
+};
+
static int debug;
module_param(debug, int, 0644);
@@ -138,6 +143,38 @@ static void __copy_timestamp(struct vb2_buffer *vb, const void *pb)
}
};
+static void __fd_install_at_dequeue_cb(void *data)
+{
+ struct out_fence_data *of = data;
+
+ fd_install(of->fence_fd, of->file);
+ kfree(of);
+}
+
+static void __send_out_fence(struct vb2_buffer *vb)
+{
+ struct video_device *vdev = to_video_device(vb->vb2_queue->dev);
+ struct v4l2_fh *fh = vdev->queue->owner;
+ struct v4l2_event event;
+ struct out_fence_data *of;
+
+ if (vb->out_fence_fd < 0)
+ return;
+
+ memset(&event, 0, sizeof(event));
+ event.type = V4L2_EVENT_OUT_FENCE;
+ event.u.out_fence.index = vb->index;
+ event.u.out_fence.out_fence_fd = vb->out_fence_fd;
+
+ of = kmalloc(sizeof(*of), GFP_KERNEL);
+ of->fence_fd = vb->out_fence_fd;
+ of->file = vb->sync_file->file;
+
+ v4l2_event_queue_fh_with_cb(fh, &event, __fd_install_at_dequeue_cb, of);
+
+ vb->sync_file = NULL;
+}
+
static void vb2_warn_zero_bytesused(struct vb2_buffer *vb)
{
static bool check_once;
@@ -185,6 +222,12 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
return -EINVAL;
}
+ if (!q->ordered_in_driver && (b->flags & V4L2_BUF_FLAG_OUT_FENCE)) {
+ dprintk(1, "%s: out-fence doesn't work on unordered queues\n",
+ opname);
+ return -EINVAL;
+ }
+
return __verify_planes_array(q->bufs[b->index], b);
}
@@ -213,6 +256,8 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
b->reserved = 0;
b->fence_fd = -1;
+ if (vb->out_fence)
+ b->flags |= V4L2_BUF_FLAG_OUT_FENCE;
if (vb->in_fence)
b->flags |= V4L2_BUF_FLAG_IN_FENCE;
else
@@ -456,6 +501,7 @@ static const struct vb2_buf_ops v4l2_buf_ops = {
.fill_user_buffer = __fill_v4l2_buffer,
.fill_vb2_buffer = __fill_vb2_buffer,
.copy_timestamp = __copy_timestamp,
+ .send_out_fence = __send_out_fence,
};
/**
@@ -593,6 +639,15 @@ int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
}
}
+ if (b->flags & V4L2_BUF_FLAG_OUT_FENCE) {
+ ret = vb2_setup_out_fence(q, b->index);
+ if (ret) {
+ dprintk(1, "failed to set up out-fence\n");
+ dma_fence_put(fence);
+ return ret;
+ }
+ }
+
return vb2_core_qbuf(q, b->index, b, fence);
}
EXPORT_SYMBOL_GPL(vb2_qbuf);
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [RFC v4 17/17] [media] v4l: Document explicit synchronization behaviour
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
` (15 preceding siblings ...)
2017-10-20 21:50 ` [RFC v4 16/17] [media] vb2: add out-fence support to QBUF Gustavo Padovan
@ 2017-10-20 21:50 ` Gustavo Padovan
2017-10-27 10:08 ` Brian Starkey
2017-11-03 7:53 ` Hans Verkuil
16 siblings, 2 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-10-20 21:50 UTC (permalink / raw)
To: linux-media
Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
From: Gustavo Padovan <gustavo.padovan@collabora.com>
Add section to VIDIOC_QBUF about it
v3:
- make the out_fence refer to the current buffer (Hans)
- Note what happens when the IN_FENCE is not set (Hans)
v2:
- mention that fences are files (Hans)
- rework for the new API
Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
---
Documentation/media/uapi/v4l/vidioc-qbuf.rst | 31 ++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/Documentation/media/uapi/v4l/vidioc-qbuf.rst b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
index 9e448a4aa3aa..a65a50578bad 100644
--- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst
+++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
@@ -118,6 +118,37 @@ immediately with an ``EAGAIN`` error code when no buffer is available.
The struct :c:type:`v4l2_buffer` structure is specified in
:ref:`buffer`.
+Explicit Synchronization
+------------------------
+
+Explicit Synchronization allows us to control the synchronization of
+shared buffers from userspace by passing fences to the kernel and/or
+receiving them from it. Fences passed to the kernel are named in-fences and
+the kernel should wait on them to signal before using the buffer, i.e., queueing
+it to the driver. On the other side, the kernel can create out-fences for the
+buffers it queues to the drivers. Out-fences signal when the driver is
+finished with buffer, i.e., the buffer is ready. The fences are represented
+as a file and passed as a file descriptor to userspace.
+
+The in-fences are communicated to the kernel at the ``VIDIOC_QBUF`` ioctl
+using the ``V4L2_BUF_FLAG_IN_FENCE`` buffer
+flags and the `fence_fd` field. If an in-fence needs to be passed to the kernel,
+`fence_fd` should be set to the fence file descriptor number and the
+``V4L2_BUF_FLAG_IN_FENCE`` should be set as well Setting one but not the other
+will cause ``VIDIOC_QBUF`` to return with error.
+
+The fence_fd field (formely the reserved2 field) will be ignored if the
+``V4L2_BUF_FLAG_IN_FENCE`` is not set.
+
+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE`` flag should
+be set to ask for a fence to be attached to the buffer. To become aware of
+the out-fence created one should listen for the ``V4L2_EVENT_OUT_FENCE`` event.
+An event will be triggered for every buffer queued to the V4L2 driver with the
+``V4L2_BUF_FLAG_OUT_FENCE``.
+
+At streamoff the out-fences will either signal normally if the drivers waits
+for the operations on the buffers to finish or signal with error if the
+driver cancels the pending operations.
Return Value
============
--
2.13.6
^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [RFC v4 01/17] [media] v4l: create v4l2_event_subscribe_v4l2()
2017-10-20 21:49 ` [RFC v4 01/17] [media] v4l: create v4l2_event_subscribe_v4l2() Gustavo Padovan
@ 2017-10-24 12:51 ` Hans Verkuil
0 siblings, 0 replies; 32+ messages in thread
From: Hans Verkuil @ 2017-10-24 12:51 UTC (permalink / raw)
To: Gustavo Padovan, linux-media
Cc: Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
On 10/20/2017 11:49 PM, Gustavo Padovan wrote:
> From: Gustavo Padovan <gustavo.padovan@collabora.com>
>
> We need a common function to subscribe all the common events in drivers,
> so far we had only V4L2_EVENT_CTRL, so such a function wasn't necessary,
> but we are about to introduce a new event for the upcoming explicit fences
> implementation, thus a common place is needed.
>
> Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
> ---
> drivers/media/v4l2-core/v4l2-event.c | 12 ++++++++++++
> include/media/v4l2-event.h | 8 ++++++++
> 2 files changed, 20 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-event.c b/drivers/media/v4l2-core/v4l2-event.c
> index 968c2eb08b5a..313ee9d1f9ee 100644
> --- a/drivers/media/v4l2-core/v4l2-event.c
> +++ b/drivers/media/v4l2-core/v4l2-event.c
> @@ -20,6 +20,7 @@
> #include <media/v4l2-dev.h>
> #include <media/v4l2-fh.h>
> #include <media/v4l2-event.h>
> +#include <media/v4l2-ctrls.h>
>
> #include <linux/mm.h>
> #include <linux/sched.h>
> @@ -354,3 +355,14 @@ int v4l2_src_change_event_subdev_subscribe(struct v4l2_subdev *sd,
> return v4l2_src_change_event_subscribe(fh, sub);
> }
> EXPORT_SYMBOL_GPL(v4l2_src_change_event_subdev_subscribe);
> +
> +int v4l2_subscribe_event_v4l2(struct v4l2_fh *fh,
> + const struct v4l2_event_subscription *sub)
> +{
> + switch (sub->type) {
> + case V4L2_EVENT_CTRL:
> + return v4l2_ctrl_subscribe_event(fh, sub);
> + }
> + return -EINVAL;
> +}
> +EXPORT_SYMBOL(v4l2_subscribe_event_v4l2);
> diff --git a/include/media/v4l2-event.h b/include/media/v4l2-event.h
> index 6741910c3a18..2b794f2ad824 100644
> --- a/include/media/v4l2-event.h
> +++ b/include/media/v4l2-event.h
> @@ -236,4 +236,12 @@ int v4l2_src_change_event_subscribe(struct v4l2_fh *fh,
> int v4l2_src_change_event_subdev_subscribe(struct v4l2_subdev *sd,
> struct v4l2_fh *fh,
> struct v4l2_event_subscription *sub);
> +/**
> + * v4l2_subscribe_event_v4l2 - helper function that subscribe all v4l2 events
> + *
> + * @fh: pointer to struct v4l2_fh
> + * @sub: pointer to &struct v4l2_event_subscription
> + */
> +int v4l2_subscribe_event_v4l2(struct v4l2_fh *fh,
Why the _v4l2 suffix? It's weird and I think you can just drop it. Otherwise this looks good.
Regards,
Hans
> + const struct v4l2_event_subscription *sub);
> #endif /* V4L2_EVENT_H */
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 05/17] [media] v4l: add V4L2_EVENT_OUT_FENCE event
2017-10-20 21:50 ` [RFC v4 05/17] [media] v4l: add V4L2_EVENT_OUT_FENCE event Gustavo Padovan
@ 2017-10-24 13:09 ` Hans Verkuil
0 siblings, 0 replies; 32+ messages in thread
From: Hans Verkuil @ 2017-10-24 13:09 UTC (permalink / raw)
To: Gustavo Padovan, linux-media
Cc: Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
On 10/20/2017 11:50 PM, Gustavo Padovan wrote:
> From: Gustavo Padovan <gustavo.padovan@collabora.com>
>
> Add a new event the userspace can subscribe to receive notifications
> of the out_fence_fd when a buffer is queued onto the driver.
> The event provides the index of the queued buffer and the out_fence_fd.
>
> v3: - Rename event to V4L2_EVENT_OUT_FENCE
>
> v2: - Add missing Documentation (Mauro)
>
> Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
> ---
> Documentation/media/uapi/v4l/vidioc-dqevent.rst | 23 +++++++++++++++++++++++
> Documentation/media/videodev2.h.rst.exceptions | 1 +
> include/uapi/linux/videodev2.h | 12 ++++++++++++
> 3 files changed, 36 insertions(+)
>
> diff --git a/Documentation/media/uapi/v4l/vidioc-dqevent.rst b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
> index cb3565f36793..2143df63aeb1 100644
> --- a/Documentation/media/uapi/v4l/vidioc-dqevent.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
> @@ -79,6 +79,10 @@ call.
> - ``src_change``
> - Event data for event V4L2_EVENT_SOURCE_CHANGE.
> * -
> + - struct :c:type:`v4l2_event_buf_queued`
> + - ``buf_queued``
> + - Event data for event V4L2_EVENT_OUT_FENCE.
> + * -
> - __u8
> - ``data``\ [64]
> - Event data. Defined by the event type. The union should be used to
> @@ -338,6 +342,25 @@ call.
> each cell in the motion detection grid, then that all cells are
> automatically assigned to the default region 0.
>
> +.. c:type:: v4l2_event_out_fence
> +
> +.. flat-table:: struct v4l2_event_out_fence
> + :header-rows: 0
> + :stub-columns: 0
> + :widths: 1 1 2
> +
> + * - __u32
> + - ``index``
> + - The index of the buffer that was queued to the driver.
> + * - __s32
> + - ``out_fence_fd``
> + - The out-fence file descriptor of the buffer that was queued to
> + the driver. It will signal when the buffer is ready, or if an
> + error happens.
What happens when the buffer has already been dequeued? This needs to be defined.
Perhaps set out_fence_fd to 0 or -1 in that case?
Regards,
Hans
> +
> +
> +
> +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
>
>
> .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
> diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions
> index a5cb0a8686ac..32f3d5b37e3a 100644
> --- a/Documentation/media/videodev2.h.rst.exceptions
> +++ b/Documentation/media/videodev2.h.rst.exceptions
> @@ -462,6 +462,7 @@ replace define V4L2_EVENT_CTRL event-type
> replace define V4L2_EVENT_FRAME_SYNC event-type
> replace define V4L2_EVENT_SOURCE_CHANGE event-type
> replace define V4L2_EVENT_MOTION_DET event-type
> +replace define V4L2_EVENT_OUT_FENCE event-type
> replace define V4L2_EVENT_PRIVATE_START event-type
>
> replace define V4L2_EVENT_CTRL_CH_VALUE ctrl-changes-flags
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 185d6a0acc06..2a432e8c18e3 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -2156,6 +2156,7 @@ struct v4l2_streamparm {
> #define V4L2_EVENT_FRAME_SYNC 4
> #define V4L2_EVENT_SOURCE_CHANGE 5
> #define V4L2_EVENT_MOTION_DET 6
> +#define V4L2_EVENT_OUT_FENCE 7
> #define V4L2_EVENT_PRIVATE_START 0x08000000
>
> /* Payload for V4L2_EVENT_VSYNC */
> @@ -2208,6 +2209,16 @@ struct v4l2_event_motion_det {
> __u32 region_mask;
> };
>
> +/**
> + * struct v4l2_event_out_fence - out fence fd event
> + * @index: index of the buffer queued in the driver
> + * @out_fence_fd: out-fence fd of the buffer queued
> + */
> +struct v4l2_event_out_fence {
> + __u32 index;
> + __s32 out_fence_fd;
> +};
> +
> struct v4l2_event {
> __u32 type;
> union {
> @@ -2216,6 +2227,7 @@ struct v4l2_event {
> struct v4l2_event_frame_sync frame_sync;
> struct v4l2_event_src_change src_change;
> struct v4l2_event_motion_det motion_det;
> + struct v4l2_event_out_fence out_fence;
> __u8 data[64];
> } u;
> __u32 pending;
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 08/17] [media] vb2: add 'ordered_in_driver' property to queues
2017-10-20 21:50 ` [RFC v4 08/17] [media] vb2: add 'ordered_in_driver' property to queues Gustavo Padovan
@ 2017-10-24 13:11 ` Hans Verkuil
0 siblings, 0 replies; 32+ messages in thread
From: Hans Verkuil @ 2017-10-24 13:11 UTC (permalink / raw)
To: Gustavo Padovan, linux-media
Cc: Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
On 10/20/2017 11:50 PM, Gustavo Padovan wrote:
> From: Gustavo Padovan <gustavo.padovan@collabora.com>
>
> For explicit synchronization (and soon for HAL3/Request API) we need
> the v4l2-driver to guarantee the ordering in which the buffers were queued
> by userspace. This is already true for many drivers, but we never needed
> to say it.
>
> v2: rename property to 'ordered_in_driver' to avoid confusion
>
> Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
> ---
> include/media/videobuf2-core.h | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 96af4eb49e52..6dd3f0181107 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -500,6 +500,11 @@ struct vb2_buf_ops {
> * @last_buffer_dequeued: used in poll() and DQBUF to immediately return if the
> * last decoded buffer was already dequeued. Set for capture queues
> * when a buffer with the V4L2_BUF_FLAG_LAST is dequeued.
> + * @ordered_in_driver: if the driver can guarantee that the queue will be
> + * ordered or not, i.e., the buffers are queued to the driver in
> + * the same order they are dequeued from the driver. The default
I'd swap the words 'queued' and 'dequeued'. It's more logical that way.
Regards,
Hans
> + * is not ordered unless the driver sets this flag. As of now it
> + * is mandatory for using explicit fences.
> * @fileio: file io emulator internal data, used only if emulator is active
> * @threadio: thread io internal data, used only if thread is active
> */
> @@ -552,6 +557,7 @@ struct vb2_queue {
> unsigned int is_output:1;
> unsigned int copy_timestamp:1;
> unsigned int last_buffer_dequeued:1;
> + unsigned int ordered_in_driver:1;
>
> struct vb2_fileio_data *fileio;
> struct vb2_threadio_data *threadio;
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 13/17] [media] vb2: add in-fence support to QBUF
2017-10-20 21:50 ` [RFC v4 13/17] [media] vb2: add in-fence support to QBUF Gustavo Padovan
@ 2017-10-25 14:49 ` Brian Starkey
2017-11-10 12:38 ` Gustavo Padovan
0 siblings, 1 reply; 32+ messages in thread
From: Brian Starkey @ 2017-10-25 14:49 UTC (permalink / raw)
To: Gustavo Padovan
Cc: linux-media, Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan,
Pawel Osciak, Alexandre Courbot, Sakari Ailus, linux-kernel,
Gustavo Padovan
Hi Gustavo,
On Fri, Oct 20, 2017 at 07:50:08PM -0200, Gustavo Padovan wrote:
>From: Gustavo Padovan <gustavo.padovan@collabora.com>
>
>Receive in-fence from userspace and add support for waiting on them
>before queueing the buffer to the driver. Buffers are only queued
>to the driver once they are ready. A buffer is ready when its
>in-fence signals.
>
>For queues that require vb2 to queue buffers to the v4l2 driver in same
>order they are received from userspace we use fence_array to keep that
>ordering. Basically we create a fence_array that contains both the current
>fence and the fence from the previous buffer (which might be a fence array
>as well). The base fence class for the fence_array becomes the new buffer
>fence, waiting on that one guarantees that it won't be queued out of
>order.
The API sounds/looks good to me, makes sense to let driver opt in/out
via the ordered_in_vb2 thing. Thanks for implementing it!
>
>v5: - use fence_array to keep buffers ordered in vb2 core when
> needed (Brian Stark)
> - keep backward compatibility on the reserved2 field (Brian Stark)
> - protect fence callback removal with lock (Brian Stark)
Brian Starkey, but close ;-)
>
>v4:
> - Add a comment about dma_fence_add_callback() not returning a
> error (Hans)
> - Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
> - select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
> - Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
> - Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
> - Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
> vb2_start_streaming() (Hans)
> - set IN_FENCE flags on __fill_v4l2_buffer (Hans)
> - Queue buffers to the driver as soon as they are ready (Hans)
> - call fill_user_buffer() after queuing the buffer (Hans)
> - add err: label to clean up fence
> - add dma_fence_wait() before calling vb2_start_streaming()
>
>v3: - document fence parameter
> - remove ternary if at vb2_qbuf() return (Mauro)
> - do not change if conditions behaviour (Mauro)
>
>v2:
> - fix vb2_queue_or_prepare_buf() ret check
> - remove check for VB2_MEMORY_DMABUF only (Javier)
> - check num of ready buffers to start streaming
> - when queueing, start from the first ready buffer
> - handle queue cancel
>
>Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
>---
> drivers/media/v4l2-core/Kconfig | 1 +
> drivers/media/v4l2-core/videobuf2-core.c | 179 ++++++++++++++++++++++++++++---
> drivers/media/v4l2-core/videobuf2-v4l2.c | 29 ++++-
> include/media/videobuf2-core.h | 17 ++-
> 4 files changed, 208 insertions(+), 18 deletions(-)
>
>diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
>index a35c33686abf..3f988c407c80 100644
>--- a/drivers/media/v4l2-core/Kconfig
>+++ b/drivers/media/v4l2-core/Kconfig
>@@ -83,6 +83,7 @@ config VIDEOBUF_DVB
> # Used by drivers that need Videobuf2 modules
> config VIDEOBUF2_CORE
> select DMA_SHARED_BUFFER
>+ select SYNC_FILE
> tristate
>
> config VIDEOBUF2_MEMOPS
>diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
>index 60f8b582396a..78f369dba3e3 100644
>--- a/drivers/media/v4l2-core/videobuf2-core.c
>+++ b/drivers/media/v4l2-core/videobuf2-core.c
>@@ -346,6 +346,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
> vb->index = q->num_buffers + buffer;
> vb->type = q->type;
> vb->memory = memory;
>+ spin_lock_init(&vb->fence_cb_lock);
> for (plane = 0; plane < num_planes; ++plane) {
> vb->planes[plane].length = plane_sizes[plane];
> vb->planes[plane].min_length = plane_sizes[plane];
>@@ -1222,6 +1223,9 @@ static void __enqueue_in_driver(struct vb2_buffer *vb)
> {
> struct vb2_queue *q = vb->vb2_queue;
>
>+ if (vb->in_fence && !dma_fence_is_signaled(vb->in_fence))
>+ return;
>+
> vb->state = VB2_BUF_STATE_ACTIVE;
> atomic_inc(&q->owned_by_drv_count);
>
>@@ -1273,6 +1277,20 @@ static int __buf_prepare(struct vb2_buffer *vb, const void *pb)
> return 0;
> }
>
>+static int __get_num_ready_buffers(struct vb2_queue *q)
>+{
>+ struct vb2_buffer *vb;
>+ int ready_count = 0;
>+
>+ /* count num of buffers ready in front of the queued_list */
>+ list_for_each_entry(vb, &q->queued_list, queued_entry) {
>+ if (!vb->in_fence || dma_fence_is_signaled(vb->in_fence))
>+ ready_count++;
I think there's still some races on vb->in_fence. Couldn't the
callback on any of the buffers in the queued_list have their callback
run at any moment?
That could make this loop do bad things, and similarly the loop in
vb2_start_streaming() which calls __enqueue_in_driver().
>+ }
>+
>+ return ready_count;
>+}
>+
> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
> {
> struct vb2_buffer *vb;
>@@ -1361,7 +1379,74 @@ static int vb2_start_streaming(struct vb2_queue *q)
> return ret;
> }
>
>-int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
>+static struct dma_fence *__set_in_fence(struct vb2_queue *q,
>+ struct vb2_buffer *vb,
>+ struct dma_fence *fence)
>+{
>+ if (q->last_fence && dma_fence_is_signaled(q->last_fence)) {
>+ dma_fence_put(q->last_fence);
>+ q->last_fence = NULL;
>+ }
>+
>+ /* For drivers that require the ordering of buffers queued from
>+ * userspace to be the same it is queued to the driver - output
>+ * some m2m devices - we create a fence array with the fence from
>+ * the last queued buffer and this one, that way the fence for this
>+ * buffer can't signal before the last one.
>+ */
>+ if (q->is_output || q->ordered_in_vb2) {
Just a stylistic thing - you could invert this conditional and return
early, saving a level of indentation below.
>+ if (fence && q->last_fence) {
>+ struct dma_fence **fences;
>+ struct dma_fence_array *arr;
>+
>+ fences = kcalloc(2, sizeof(*fences), GFP_KERNEL);
>+ if (!fences)
>+ return ERR_PTR(-ENOMEM);
>+
>+ fences[0] = fence;
>+ fences[1] = q->last_fence;
>+
>+ arr = dma_fence_array_create(2, fences,
>+ dma_fence_context_alloc(1),
>+ 1, false);
I'm not sure if it's right to get a new fence_context every time here.
The comment on dma_fence_context_counter sounds like the vb2_queue
should have a single context of its own.
>+ if (!arr) {
>+ kfree(fences);
>+ return ERR_PTR(-ENOMEM);
>+ }
>+
>+ fence = &arr->base;
>+ } else if (!fence && q->last_fence) {
>+ fence = dma_fence_get(q->last_fence);
>+ }
>+
>+ q->last_fence = dma_fence_get(fence);
Seems like the refcounting isn't quite right here.
If you land in "else if (!fence && q->last_fence)", then "fence" gets
a new reference to q->last_fence, and you take another one here -
which looks like one too many.
I _think_ the right thing to do is just give the previous reference on
q->last_fence to "fence", without taking a new one:
} else if (!fence && q->last_fence) {
fence = q->last_fence;
}
That's analogous to the other path where fence is not-NULL, and the
fence-array takes ownership of the existing references on fence and
q->last_fence, without taking new references.
Please double check my logic though!
>+ }
>+
>+ return fence;
>+}
>+
>+static void vb2_qbuf_fence_cb(struct dma_fence *f, struct dma_fence_cb *cb)
>+{
>+ struct vb2_buffer *vb = container_of(cb, struct vb2_buffer, fence_cb);
>+ struct vb2_queue *q = vb->vb2_queue;
>+ unsigned long flags;
>+
>+ spin_lock_irqsave(&vb->fence_cb_lock, flags);
>+ if (!vb->in_fence) {
>+ spin_unlock_irqrestore(&vb->fence_cb_lock, flags);
>+ return;
>+ }
>+
>+ dma_fence_put(vb->in_fence);
>+ vb->in_fence = NULL;
>+ spin_unlock_irqrestore(&vb->fence_cb_lock, flags);
>+
>+ if (q->start_streaming_called)
>+ __enqueue_in_driver(vb);
>+}
>+
>+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
>+ struct dma_fence *fence)
> {
> struct vb2_buffer *vb;
> int ret;
>@@ -1372,16 +1457,18 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
> case VB2_BUF_STATE_DEQUEUED:
> ret = __buf_prepare(vb, pb);
> if (ret)
>- return ret;
>+ goto err;
> break;
> case VB2_BUF_STATE_PREPARED:
> break;
> case VB2_BUF_STATE_PREPARING:
> dprintk(1, "buffer still being prepared\n");
>- return -EINVAL;
>+ ret = -EINVAL;
>+ goto err;
> default:
> dprintk(1, "invalid buffer state %d\n", vb->state);
>- return -EINVAL;
>+ ret = -EINVAL;
>+ goto err;
> }
>
> /*
>@@ -1398,30 +1485,75 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
>
> trace_vb2_qbuf(q, vb);
>
>+ vb->in_fence = __set_in_fence(q, vb, fence);
>+ if (IS_ERR(vb->in_fence)) {
>+ ret = PTR_ERR(vb->in_fence);
>+ goto err;
Between here, __set_in_fence() and vb2_qbuf() there's a reference
leaked on "fence" in the case of error. I'm not sure what semantics
you'd like, but it should be clear whose responsibility it is to clear
up "fence" if __set_in_fence() fails (at the moment, looks like no-one
does).
>+ }
... and if __set_in_fence() succeeds, then vb->in_fence now owns the
reference on "fence" that was passed in, so I guess you should set
fence = NULL here.
>+
>+ /*
>+ * If it is time to call vb2_start_streaming() wait for the fence
>+ * to signal first. Of course, this happens only once per streaming.
>+ * We want to run any step that might fail before we set the callback
>+ * to queue the fence when it signals.
>+ */
>+ if (fence && !q->start_streaming_called &&
I think this should be vb->in_fence now, and the same for all further
uses of "fence" in this function.
>+ __get_num_ready_buffers(q) == q->min_buffers_needed - 1)
>+ dma_fence_wait(fence, true);
>+
> /*
> * If streamon has been called, and we haven't yet called
> * start_streaming() since not enough buffers were queued, and
> * we now have reached the minimum number of queued buffers,
> * then we can finally call start_streaming().
>- *
>- * If already streaming, give the buffer to driver for processing.
>- * If not, the buffer will be given to driver on next streamon.
> */
> if (q->streaming && !q->start_streaming_called &&
>- q->queued_count >= q->min_buffers_needed) {
>+ __get_num_ready_buffers(q) >= q->min_buffers_needed) {
> ret = vb2_start_streaming(q);
> if (ret)
>- return ret;
>- } else if (q->start_streaming_called) {
>- __enqueue_in_driver(vb);
>+ goto err;
>+ }
>+
>+ /*
>+ * For explicit synchronization: If the fence didn't signal
>+ * yet we setup a callback to queue the buffer once the fence
>+ * signals, and then, return successfully. But if the fence
>+ * already signaled we lose the reference we held and queue the
>+ * buffer to the driver.
>+ */
>+ if (fence) {
>+ ret = dma_fence_add_callback(fence, &vb->fence_cb,
>+ vb2_qbuf_fence_cb);
>+ if (!ret)
>+ goto fill;
Just spotted that you should probably handle the -EINVAL case here and
jump to err.
>+
>+ dma_fence_put(fence);
>+ vb->in_fence = NULL;
> }
>
>+fill:
>+ /*
>+ * If already streaming and there is no fence to wait on
>+ * give the buffer to driver for processing.
>+ */
>+ if (q->start_streaming_called && !vb->in_fence)
>+ __enqueue_in_driver(vb);
The callback could have run just before this, clearing vb->in_fence
and ending up with __enqueue_in_driver() getting called twice.
>+
> /* Fill buffer information for the userspace */
> if (pb)
> call_void_bufop(q, fill_user_buffer, vb, pb);
>
> dprintk(2, "qbuf of buffer %d succeeded\n", vb->index);
> return 0;
>+
>+err:
>+ if (vb->in_fence) {
>+ dma_fence_put(vb->in_fence);
>+ vb->in_fence = NULL;
>+ }
>+
>+ return ret;
>+
> }
> EXPORT_SYMBOL_GPL(vb2_core_qbuf);
>
>@@ -1632,6 +1764,8 @@ EXPORT_SYMBOL_GPL(vb2_core_dqbuf);
> static void __vb2_queue_cancel(struct vb2_queue *q)
> {
> unsigned int i;
>+ struct vb2_buffer *vb;
>+ unsigned long flags;
>
> /*
> * Tell driver to stop all transactions and release all queued
>@@ -1659,6 +1793,21 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
> q->queued_count = 0;
> q->error = 0;
>
>+ list_for_each_entry(vb, &q->queued_list, queued_entry) {
>+ spin_lock_irqsave(&vb->fence_cb_lock, flags);
>+ if (vb->in_fence) {
>+ dma_fence_remove_callback(vb->in_fence, &vb->fence_cb);
>+ dma_fence_put(vb->in_fence);
>+ vb->in_fence = NULL;
>+ }
>+ spin_unlock_irqrestore(&vb->fence_cb_lock, flags);
>+ }
>+
>+ if (q->last_fence) {
>+ dma_fence_put(q->last_fence);
>+ q->last_fence = NULL;
>+ }
>+
> /*
> * Remove all buffers from videobuf's list...
> */
>@@ -1720,7 +1869,7 @@ int vb2_core_streamon(struct vb2_queue *q, unsigned int type)
> * Tell driver to start streaming provided sufficient buffers
> * are available.
> */
>- if (q->queued_count >= q->min_buffers_needed) {
>+ if (__get_num_ready_buffers(q) >= q->min_buffers_needed) {
> ret = v4l_vb2q_enable_media_source(q);
> if (ret)
> return ret;
>@@ -2240,7 +2389,7 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
> * Queue all buffers.
> */
> for (i = 0; i < q->num_buffers; i++) {
>- ret = vb2_core_qbuf(q, i, NULL);
>+ ret = vb2_core_qbuf(q, i, NULL, NULL);
> if (ret)
> goto err_reqbufs;
> fileio->bufs[i].queued = 1;
>@@ -2419,7 +2568,7 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
>
> if (copy_timestamp)
> b->timestamp = ktime_get_ns();
>- ret = vb2_core_qbuf(q, index, NULL);
>+ ret = vb2_core_qbuf(q, index, NULL, NULL);
> dprintk(5, "vb2_dbuf result: %d\n", ret);
> if (ret)
> return ret;
>@@ -2522,7 +2671,7 @@ static int vb2_thread(void *data)
> if (copy_timestamp)
> vb->timestamp = ktime_get_ns();;
> if (!threadio->stop)
>- ret = vb2_core_qbuf(q, vb->index, NULL);
>+ ret = vb2_core_qbuf(q, vb->index, NULL, NULL);
> call_void_qop(q, wait_prepare, q);
> if (ret || threadio->stop)
> break;
>diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c b/drivers/media/v4l2-core/videobuf2-v4l2.c
>index 110fb45fef6f..4c09ea007d90 100644
>--- a/drivers/media/v4l2-core/videobuf2-v4l2.c
>+++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
>@@ -23,6 +23,7 @@
> #include <linux/sched.h>
> #include <linux/freezer.h>
> #include <linux/kthread.h>
>+#include <linux/sync_file.h>
>
> #include <media/v4l2-dev.h>
> #include <media/v4l2-fh.h>
>@@ -178,6 +179,12 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
> return -EINVAL;
> }
>
>+ if ((b->fence_fd != 0 && b->fence_fd != -1) &&
>+ !(b->flags & V4L2_BUF_FLAG_IN_FENCE)) {
>+ dprintk(1, "%s: fence_fd set without IN_FENCE flag\n", opname);
>+ return -EINVAL;
>+ }
>+
> return __verify_planes_array(q->bufs[b->index], b);
> }
>
>@@ -203,9 +210,14 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
> b->timestamp = ns_to_timeval(vb->timestamp);
> b->timecode = vbuf->timecode;
> b->sequence = vbuf->sequence;
>- b->fence_fd = -1;
> b->reserved = 0;
>
>+ b->fence_fd = -1;
>+ if (vb->in_fence)
>+ b->flags |= V4L2_BUF_FLAG_IN_FENCE;
>+ else
>+ b->flags &= ~V4L2_BUF_FLAG_IN_FENCE;
>+
> if (q->is_multiplanar) {
> /*
> * Fill in plane-related data if userspace provided an array
>@@ -560,6 +572,7 @@ EXPORT_SYMBOL_GPL(vb2_create_bufs);
>
> int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
> {
>+ struct dma_fence *fence = NULL;
> int ret;
>
> if (vb2_fileio_is_active(q)) {
>@@ -568,7 +581,19 @@ int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
> }
>
> ret = vb2_queue_or_prepare_buf(q, b, "qbuf");
>- return ret ? ret : vb2_core_qbuf(q, b->index, b);
>+ if (ret)
>+ return ret;
>+
>+ if (b->flags & V4L2_BUF_FLAG_IN_FENCE) {
>+ fence = sync_file_get_fence(b->fence_fd);
In DRM a fence_fd of -1 means "already signaled" - should that be
allowed here too?
Cheers,
-Brian
>+ if (!fence) {
>+ dprintk(1, "failed to get in-fence from fd %d\n",
>+ b->fence_fd);
>+ return -EINVAL;
>+ }
>+ }
>+
>+ return vb2_core_qbuf(q, b->index, b, fence);
> }
> EXPORT_SYMBOL_GPL(vb2_qbuf);
>
>diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>index fc333e10e7d8..624ca2dce9ea 100644
>--- a/include/media/videobuf2-core.h
>+++ b/include/media/videobuf2-core.h
>@@ -16,6 +16,7 @@
> #include <linux/mutex.h>
> #include <linux/poll.h>
> #include <linux/dma-buf.h>
>+#include <linux/dma-fence.h>
>
> #define VB2_MAX_FRAME (32)
> #define VB2_MAX_PLANES (8)
>@@ -254,11 +255,20 @@ struct vb2_buffer {
> * all buffers queued from userspace
> * done_entry: entry on the list that stores all buffers ready
> * to be dequeued to userspace
>+ * in_fence: fence receive from vb2 client to wait on before
>+ * using the buffer (queueing to the driver)
>+ * fence_cb: fence callback information
>+ * fence_cb_lock: protect callback signal/remove
> */
> enum vb2_buffer_state state;
>
> struct list_head queued_entry;
> struct list_head done_entry;
>+
>+ struct dma_fence *in_fence;
>+ struct dma_fence_cb fence_cb;
>+ spinlock_t fence_cb_lock;
>+
> #ifdef CONFIG_VIDEO_ADV_DEBUG
> /*
> * Counters for how often these buffer-related ops are
>@@ -508,6 +518,7 @@ struct vb2_buf_ops {
> * @ordered_in_vb2: set by the driver to tell vb2 te guarantee the order
> * of buffer queue from userspace with QBUF() until they are
> * queued to the driver.
>+ * @last_fence: last in-fence received. Used to keep ordering.
> * @fileio: file io emulator internal data, used only if emulator is active
> * @threadio: thread io internal data, used only if thread is active
> */
>@@ -563,6 +574,8 @@ struct vb2_queue {
> unsigned int ordered_in_driver:1;
> unsigned int ordered_in_vb2:1;
>
>+ struct dma_fence *last_fence;
>+
> struct vb2_fileio_data *fileio;
> struct vb2_threadio_data *threadio;
>
>@@ -738,6 +751,7 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
> * @index: id number of the buffer
> * @pb: buffer structure passed from userspace to vidioc_qbuf handler
> * in driver
>+ * @fence: in-fence to wait on before queueing the buffer
> *
> * Should be called from vidioc_qbuf ioctl handler of a driver.
> * The passed buffer should have been verified.
>@@ -752,7 +766,8 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
> * The return values from this function are intended to be directly returned
> * from vidioc_qbuf handler in driver.
> */
>-int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb);
>+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
>+ struct dma_fence *fence);
>
> /**
> * vb2_core_dqbuf() - Dequeue a buffer to the userspace
>--
>2.13.6
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 15/17] [media] vb2: add infrastructure to support out-fences
2017-10-20 21:50 ` [RFC v4 15/17] [media] vb2: add infrastructure to support out-fences Gustavo Padovan
@ 2017-10-27 9:41 ` Brian Starkey
0 siblings, 0 replies; 32+ messages in thread
From: Brian Starkey @ 2017-10-27 9:41 UTC (permalink / raw)
To: Gustavo Padovan
Cc: linux-media, Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan,
Pawel Osciak, Alexandre Courbot, Sakari Ailus, linux-kernel,
Gustavo Padovan
Hi Gustavo,
Sorry for the sporadic review... I'm struggling to get time to look at
these.
On Fri, Oct 20, 2017 at 07:50:10PM -0200, Gustavo Padovan wrote:
>From: Gustavo Padovan <gustavo.padovan@collabora.com>
>
>Add vb2_setup_out_fence() and the needed members to struct vb2_buffer.
>
>v2: - change it to reflect fd_install at DQEVENT
> - add fence context for out-fences
>
>Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
>---
> drivers/media/v4l2-core/videobuf2-core.c | 31 +++++++++++++++++++++++++++++++
> include/media/videobuf2-core.h | 20 ++++++++++++++++++++
> 2 files changed, 51 insertions(+)
>
>diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
>index 78f369dba3e3..c7ba67bda5ac 100644
>--- a/drivers/media/v4l2-core/videobuf2-core.c
>+++ b/drivers/media/v4l2-core/videobuf2-core.c
>@@ -23,8 +23,11 @@
> #include <linux/sched.h>
> #include <linux/freezer.h>
> #include <linux/kthread.h>
>+#include <linux/sync_file.h>
>+#include <linux/dma-fence.h>
>
> #include <media/videobuf2-core.h>
>+#include <media/videobuf2-fence.h>
> #include <media/v4l2-mc.h>
>
> #include <trace/events/vb2.h>
>@@ -1316,6 +1319,34 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
> }
> EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
>
>+int vb2_setup_out_fence(struct vb2_queue *q, unsigned int index)
>+{
>+ struct vb2_buffer *vb;
>+ struct dma_fence *fence;
>+
>+ vb = q->bufs[index];
>+
>+ vb->out_fence_fd = get_unused_fd_flags(O_CLOEXEC);
>+
>+ fence = vb2_fence_alloc(q->out_fence_context);
>+ if (!fence) {
>+ put_unused_fd(vb->out_fence_fd);
>+ return -ENOMEM;
>+ }
>+
>+ vb->sync_file = sync_file_create(fence);
>+ if (!vb->sync_file) {
>+ dma_fence_put(fence);
>+ put_unused_fd(vb->out_fence_fd);
>+ return -ENOMEM;
>+ }
sync_file_create() takes an extra reference of its own, so I think you
don't need the dma_fence_get() below, or you should drop a reference
before returning.
-Brian
>+
>+ vb->out_fence = dma_fence_get(fence);
>+
>+ return 0;
>+}
>+EXPORT_SYMBOL_GPL(vb2_setup_out_fence);
>+
> /**
> * vb2_start_streaming() - Attempt to start streaming.
> * @q: videobuf2 queue
>diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>index 624ca2dce9ea..1925ede73804 100644
>--- a/include/media/videobuf2-core.h
>+++ b/include/media/videobuf2-core.h
>@@ -259,6 +259,10 @@ struct vb2_buffer {
> * using the buffer (queueing to the driver)
> * fence_cb: fence callback information
> * fence_cb_lock: protect callback signal/remove
>+ * out_fence_fd: the out_fence_fd to be shared with userspace.
>+ * out_fence: the out-fence associated with the buffer once
>+ * it is queued to the driver.
>+ * sync_file: the sync file to wrap the out fence
> */
> enum vb2_buffer_state state;
>
>@@ -269,6 +273,10 @@ struct vb2_buffer {
> struct dma_fence_cb fence_cb;
> spinlock_t fence_cb_lock;
>
>+ int out_fence_fd;
>+ struct dma_fence *out_fence;
>+ struct sync_file *sync_file;
>+
> #ifdef CONFIG_VIDEO_ADV_DEBUG
> /*
> * Counters for how often these buffer-related ops are
>@@ -518,6 +526,7 @@ struct vb2_buf_ops {
> * @ordered_in_vb2: set by the driver to tell vb2 te guarantee the order
> * of buffer queue from userspace with QBUF() until they are
> * queued to the driver.
>+ * @out_fence_context: the fence context for the out fences
> * @last_fence: last in-fence received. Used to keep ordering.
> * @fileio: file io emulator internal data, used only if emulator is active
> * @threadio: thread io internal data, used only if thread is active
>@@ -574,6 +583,7 @@ struct vb2_queue {
> unsigned int ordered_in_driver:1;
> unsigned int ordered_in_vb2:1;
>
>+ u64 out_fence_context;
> struct dma_fence *last_fence;
>
> struct vb2_fileio_data *fileio;
>@@ -745,6 +755,16 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
> int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
>
> /**
>+ * vb2_setup_out_fence() - setup new out-fence
>+ * @q: The vb2_queue where to setup it
>+ * @index: index of the buffer
>+ *
>+ * Setup the file descriptor, the fence and the sync_file for the next
>+ * buffer to be queued and add everything to the tail of the q->out_fence_list.
>+ */
>+int vb2_setup_out_fence(struct vb2_queue *q, unsigned int index);
>+
>+/**
> * vb2_core_qbuf() - Queue a buffer from userspace
> *
> * @q: videobuf2 queue
>--
>2.13.6
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 16/17] [media] vb2: add out-fence support to QBUF
2017-10-20 21:50 ` [RFC v4 16/17] [media] vb2: add out-fence support to QBUF Gustavo Padovan
@ 2017-10-27 10:01 ` Brian Starkey
2017-11-03 0:03 ` Gustavo Padovan
0 siblings, 1 reply; 32+ messages in thread
From: Brian Starkey @ 2017-10-27 10:01 UTC (permalink / raw)
To: Gustavo Padovan
Cc: linux-media, Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan,
Pawel Osciak, Alexandre Courbot, Sakari Ailus, linux-kernel,
Gustavo Padovan
Hi Gustavo,
On Fri, Oct 20, 2017 at 07:50:11PM -0200, Gustavo Padovan wrote:
>From: Gustavo Padovan <gustavo.padovan@collabora.com>
>
>If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
>an out_fence and send to userspace on the V4L2_EVENT_OUT_FENCE when
>the buffer is queued to the driver, or right away if the queue is ordered
>both in VB2 and in the driver.
>
>The fence is signaled on buffer_done(), when the job on the buffer is
>finished.
>
>v5:
> - delay fd_install to DQ_EVENT (Hans)
> - if queue is fully ordered send OUT_FENCE event right away
> (Brian)
> - rename 'q->ordered' to 'q->ordered_in_driver'
> - merge change to implement OUT_FENCE event here
>
>v4:
> - return the out_fence_fd in the BUF_QUEUED event(Hans)
>
>v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
> - set the OUT_FENCE flag if there is a fence pending (Hans)
> - call fd_install() after vb2_core_qbuf() (Hans)
> - clean up fence if vb2_core_qbuf() fails (Hans)
> - add list to store sync_file and fence for the next queued buffer
>
>v2: check if the queue is ordered.
>
>Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
>---
> drivers/media/v4l2-core/v4l2-event.c | 2 ++
> drivers/media/v4l2-core/videobuf2-core.c | 25 +++++++++++++++
> drivers/media/v4l2-core/videobuf2-v4l2.c | 55 ++++++++++++++++++++++++++++++++
> 3 files changed, 82 insertions(+)
>
>diff --git a/drivers/media/v4l2-core/v4l2-event.c b/drivers/media/v4l2-core/v4l2-event.c
>index 6274e3e174e0..275da224ace4 100644
>--- a/drivers/media/v4l2-core/v4l2-event.c
>+++ b/drivers/media/v4l2-core/v4l2-event.c
>@@ -385,6 +385,8 @@ int v4l2_subscribe_event_v4l2(struct v4l2_fh *fh,
> switch (sub->type) {
> case V4L2_EVENT_CTRL:
> return v4l2_ctrl_subscribe_event(fh, sub);
>+ case V4L2_EVENT_OUT_FENCE:
>+ return v4l2_event_subscribe(fh, sub, VIDEO_MAX_FRAME, NULL);
> }
> return -EINVAL;
> }
>diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
>index c7ba67bda5ac..21e2052776c1 100644
>--- a/drivers/media/v4l2-core/videobuf2-core.c
>+++ b/drivers/media/v4l2-core/videobuf2-core.c
>@@ -354,6 +354,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
> vb->planes[plane].length = plane_sizes[plane];
> vb->planes[plane].min_length = plane_sizes[plane];
> }
>+ vb->out_fence_fd = -1;
> q->bufs[vb->index] = vb;
>
> /* Allocate video buffer memory for the MMAP type */
>@@ -934,10 +935,24 @@ void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
> case VB2_BUF_STATE_QUEUED:
> return;
> case VB2_BUF_STATE_REQUEUEING:
>+ /*
>+ * Explicit synchronization requires ordered queues for now,
>+ * so WARN_ON if we are requeuing on an ordered queue.
>+ */
>+ if (vb->out_fence)
>+ WARN_ON_ONCE(q->ordered_in_driver);
>+
> if (q->start_streaming_called)
> __enqueue_in_driver(vb);
> return;
> default:
>+ if (state == VB2_BUF_STATE_ERROR)
>+ dma_fence_set_error(vb->out_fence, -ENOENT);
>+ dma_fence_signal(vb->out_fence);
>+ dma_fence_put(vb->out_fence);
>+ vb->out_fence = NULL;
>+ vb->out_fence_fd = -1;
>+
> /* Inform any processes that may be waiting for buffers */
> wake_up(&q->done_wq);
> break;
>@@ -1235,6 +1250,9 @@ static void __enqueue_in_driver(struct vb2_buffer *vb)
> trace_vb2_buf_queue(q, vb);
>
> call_void_vb_qop(vb, buf_queue, vb);
>+
>+ if (!(q->is_output || q->ordered_in_vb2))
>+ call_void_bufop(q, send_out_fence, vb);
> }
>
> static int __buf_prepare(struct vb2_buffer *vb, const void *pb)
>@@ -1451,6 +1469,7 @@ static struct dma_fence *__set_in_fence(struct vb2_queue *q,
> }
>
> q->last_fence = dma_fence_get(fence);
>+ call_void_bufop(q, send_out_fence, vb);
> }
>
> return fence;
>@@ -1840,6 +1859,11 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
> }
>
> /*
>+ * Renew out-fence context.
>+ */
Why is that? I don't think I understand the nuances of fence contexts.
>+ q->out_fence_context = dma_fence_context_alloc(1);
>+
>+ /*
> * Remove all buffers from videobuf's list...
> */
> INIT_LIST_HEAD(&q->queued_list);
>@@ -2171,6 +2195,7 @@ int vb2_core_queue_init(struct vb2_queue *q)
> spin_lock_init(&q->done_lock);
> mutex_init(&q->mmap_lock);
> init_waitqueue_head(&q->done_wq);
>+ q->out_fence_context = dma_fence_context_alloc(1);
>
> if (q->buf_struct_size == 0)
> q->buf_struct_size = sizeof(struct vb2_buffer);
>diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c b/drivers/media/v4l2-core/videobuf2-v4l2.c
>index 4c09ea007d90..9fb01ddefdc9 100644
>--- a/drivers/media/v4l2-core/videobuf2-v4l2.c
>+++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
>@@ -32,6 +32,11 @@
>
> #include <media/videobuf2-v4l2.h>
>
>+struct out_fence_data {
>+ int fence_fd;
>+ struct file *file;
>+};
>+
> static int debug;
> module_param(debug, int, 0644);
>
>@@ -138,6 +143,38 @@ static void __copy_timestamp(struct vb2_buffer *vb, const void *pb)
> }
> };
>
>+static void __fd_install_at_dequeue_cb(void *data)
>+{
>+ struct out_fence_data *of = data;
>+
>+ fd_install(of->fence_fd, of->file);
>+ kfree(of);
Is it possible for the user to never dequeue the event? In that case
the fd and sync_file would leak I guess.
>+}
>+
>+static void __send_out_fence(struct vb2_buffer *vb)
>+{
>+ struct video_device *vdev = to_video_device(vb->vb2_queue->dev);
>+ struct v4l2_fh *fh = vdev->queue->owner;
>+ struct v4l2_event event;
>+ struct out_fence_data *of;
>+
>+ if (vb->out_fence_fd < 0)
>+ return;
>+
>+ memset(&event, 0, sizeof(event));
>+ event.type = V4L2_EVENT_OUT_FENCE;
>+ event.u.out_fence.index = vb->index;
>+ event.u.out_fence.out_fence_fd = vb->out_fence_fd;
>+
>+ of = kmalloc(sizeof(*of), GFP_KERNEL);
>+ of->fence_fd = vb->out_fence_fd;
>+ of->file = vb->sync_file->file;
>+
>+ v4l2_event_queue_fh_with_cb(fh, &event, __fd_install_at_dequeue_cb, of);
>+
>+ vb->sync_file = NULL;
See the comment above about never dequeuing the event - maybe you need
to keep hold of the sync file to be able to clean it up in-case the
user never dequeues. If not, then you could set vb->out_fence_fd = -1
here too.
I've been looking at your v4l2-fences branch on kernel.org, which
looks quite different. Is this the newer version?
Cheers,
-Brian
>+}
>+
> static void vb2_warn_zero_bytesused(struct vb2_buffer *vb)
> {
> static bool check_once;
>@@ -185,6 +222,12 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
> return -EINVAL;
> }
>
>+ if (!q->ordered_in_driver && (b->flags & V4L2_BUF_FLAG_OUT_FENCE)) {
>+ dprintk(1, "%s: out-fence doesn't work on unordered queues\n",
>+ opname);
>+ return -EINVAL;
>+ }
>+
> return __verify_planes_array(q->bufs[b->index], b);
> }
>
>@@ -213,6 +256,8 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
> b->reserved = 0;
>
> b->fence_fd = -1;
>+ if (vb->out_fence)
>+ b->flags |= V4L2_BUF_FLAG_OUT_FENCE;
> if (vb->in_fence)
> b->flags |= V4L2_BUF_FLAG_IN_FENCE;
> else
>@@ -456,6 +501,7 @@ static const struct vb2_buf_ops v4l2_buf_ops = {
> .fill_user_buffer = __fill_v4l2_buffer,
> .fill_vb2_buffer = __fill_vb2_buffer,
> .copy_timestamp = __copy_timestamp,
>+ .send_out_fence = __send_out_fence,
> };
>
> /**
>@@ -593,6 +639,15 @@ int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
> }
> }
>
>+ if (b->flags & V4L2_BUF_FLAG_OUT_FENCE) {
>+ ret = vb2_setup_out_fence(q, b->index);
>+ if (ret) {
>+ dprintk(1, "failed to set up out-fence\n");
>+ dma_fence_put(fence);
>+ return ret;
>+ }
>+ }
>+
> return vb2_core_qbuf(q, b->index, b, fence);
> }
> EXPORT_SYMBOL_GPL(vb2_qbuf);
>--
>2.13.6
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 17/17] [media] v4l: Document explicit synchronization behaviour
2017-10-20 21:50 ` [RFC v4 17/17] [media] v4l: Document explicit synchronization behaviour Gustavo Padovan
@ 2017-10-27 10:08 ` Brian Starkey
2017-11-03 7:53 ` Hans Verkuil
1 sibling, 0 replies; 32+ messages in thread
From: Brian Starkey @ 2017-10-27 10:08 UTC (permalink / raw)
To: Gustavo Padovan
Cc: linux-media, Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan,
Pawel Osciak, Alexandre Courbot, Sakari Ailus, linux-kernel,
Gustavo Padovan
On Fri, Oct 20, 2017 at 07:50:12PM -0200, Gustavo Padovan wrote:
>From: Gustavo Padovan <gustavo.padovan@collabora.com>
>
>Add section to VIDIOC_QBUF about it
>
>v3:
> - make the out_fence refer to the current buffer (Hans)
> - Note what happens when the IN_FENCE is not set (Hans)
>
>v2:
> - mention that fences are files (Hans)
> - rework for the new API
>
>Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
>---
> Documentation/media/uapi/v4l/vidioc-qbuf.rst | 31 ++++++++++++++++++++++++++++
> 1 file changed, 31 insertions(+)
>
>diff --git a/Documentation/media/uapi/v4l/vidioc-qbuf.rst b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
>index 9e448a4aa3aa..a65a50578bad 100644
>--- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst
>+++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
>@@ -118,6 +118,37 @@ immediately with an ``EAGAIN`` error code when no buffer is available.
> The struct :c:type:`v4l2_buffer` structure is specified in
> :ref:`buffer`.
>
>+Explicit Synchronization
>+------------------------
>+
>+Explicit Synchronization allows us to control the synchronization of
>+shared buffers from userspace by passing fences to the kernel and/or
>+receiving them from it. Fences passed to the kernel are named in-fences and
>+the kernel should wait on them to signal before using the buffer, i.e., queueing
>+it to the driver. On the other side, the kernel can create out-fences for the
>+buffers it queues to the drivers. Out-fences signal when the driver is
>+finished with buffer, i.e., the buffer is ready. The fences are represented
>+as a file and passed as a file descriptor to userspace.
>+
>+The in-fences are communicated to the kernel at the ``VIDIOC_QBUF`` ioctl
>+using the ``V4L2_BUF_FLAG_IN_FENCE`` buffer
>+flags and the `fence_fd` field. If an in-fence needs to be passed to the kernel,
>+`fence_fd` should be set to the fence file descriptor number and the
>+``V4L2_BUF_FLAG_IN_FENCE`` should be set as well Setting one but not the other
>+will cause ``VIDIOC_QBUF`` to return with error.
nit: full-stop before "Setting".
Also, depending what is decided about passing in a -1 in-fence, the
wording might want to include that behaviour.
>+
>+The fence_fd field (formely the reserved2 field) will be ignored if the
>+``V4L2_BUF_FLAG_IN_FENCE`` is not set.
>+
>+To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE`` flag should
>+be set to ask for a fence to be attached to the buffer. To become aware of
>+the out-fence created one should listen for the ``V4L2_EVENT_OUT_FENCE`` event.
>+An event will be triggered for every buffer queued to the V4L2 driver with the
>+``V4L2_BUF_FLAG_OUT_FENCE``.
Is it worth mentioning the immediate out-fence behaviour for ordered
queues? And/or clarifying that otherwise the V4L2_EVENT_OUT_FENCE
event will not be sent until all in-fence(s) on the buffer (or
previously queued ones) have signalled.
Cheers,
-Brian
>+
>+At streamoff the out-fences will either signal normally if the drivers waits
>+for the operations on the buffers to finish or signal with error if the
>+driver cancels the pending operations.
>
> Return Value
> ============
>--
>2.13.6
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 16/17] [media] vb2: add out-fence support to QBUF
2017-10-27 10:01 ` Brian Starkey
@ 2017-11-03 0:03 ` Gustavo Padovan
0 siblings, 0 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-11-03 0:03 UTC (permalink / raw)
To: Brian Starkey
Cc: linux-media, Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan,
Pawel Osciak, Alexandre Courbot, Sakari Ailus, linux-kernel,
Gustavo Padovan
Hi Brian,
2017-10-27 Brian Starkey <brian.starkey@arm.com>:
> Hi Gustavo,
>
> On Fri, Oct 20, 2017 at 07:50:11PM -0200, Gustavo Padovan wrote:
> > From: Gustavo Padovan <gustavo.padovan@collabora.com>
> >
> > If V4L2_BUF_FLAG_OUT_FENCE flag is present on the QBUF call we create
> > an out_fence and send to userspace on the V4L2_EVENT_OUT_FENCE when
> > the buffer is queued to the driver, or right away if the queue is ordered
> > both in VB2 and in the driver.
> >
> > The fence is signaled on buffer_done(), when the job on the buffer is
> > finished.
> >
> > v5:
> > - delay fd_install to DQ_EVENT (Hans)
> > - if queue is fully ordered send OUT_FENCE event right away
> > (Brian)
> > - rename 'q->ordered' to 'q->ordered_in_driver'
> > - merge change to implement OUT_FENCE event here
> >
> > v4:
> > - return the out_fence_fd in the BUF_QUEUED event(Hans)
> >
> > v3: - add WARN_ON_ONCE(q->ordered) on requeueing (Hans)
> > - set the OUT_FENCE flag if there is a fence pending (Hans)
> > - call fd_install() after vb2_core_qbuf() (Hans)
> > - clean up fence if vb2_core_qbuf() fails (Hans)
> > - add list to store sync_file and fence for the next queued buffer
> >
> > v2: check if the queue is ordered.
> >
> > Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
> > ---
> > drivers/media/v4l2-core/v4l2-event.c | 2 ++
> > drivers/media/v4l2-core/videobuf2-core.c | 25 +++++++++++++++
> > drivers/media/v4l2-core/videobuf2-v4l2.c | 55 ++++++++++++++++++++++++++++++++
> > 3 files changed, 82 insertions(+)
> >
> > diff --git a/drivers/media/v4l2-core/v4l2-event.c b/drivers/media/v4l2-core/v4l2-event.c
> > index 6274e3e174e0..275da224ace4 100644
> > --- a/drivers/media/v4l2-core/v4l2-event.c
> > +++ b/drivers/media/v4l2-core/v4l2-event.c
> > @@ -385,6 +385,8 @@ int v4l2_subscribe_event_v4l2(struct v4l2_fh *fh,
> > switch (sub->type) {
> > case V4L2_EVENT_CTRL:
> > return v4l2_ctrl_subscribe_event(fh, sub);
> > + case V4L2_EVENT_OUT_FENCE:
> > + return v4l2_event_subscribe(fh, sub, VIDEO_MAX_FRAME, NULL);
> > }
> > return -EINVAL;
> > }
> > diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
> > index c7ba67bda5ac..21e2052776c1 100644
> > --- a/drivers/media/v4l2-core/videobuf2-core.c
> > +++ b/drivers/media/v4l2-core/videobuf2-core.c
> > @@ -354,6 +354,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
> > vb->planes[plane].length = plane_sizes[plane];
> > vb->planes[plane].min_length = plane_sizes[plane];
> > }
> > + vb->out_fence_fd = -1;
> > q->bufs[vb->index] = vb;
> >
> > /* Allocate video buffer memory for the MMAP type */
> > @@ -934,10 +935,24 @@ void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
> > case VB2_BUF_STATE_QUEUED:
> > return;
> > case VB2_BUF_STATE_REQUEUEING:
> > + /*
> > + * Explicit synchronization requires ordered queues for now,
> > + * so WARN_ON if we are requeuing on an ordered queue.
> > + */
> > + if (vb->out_fence)
> > + WARN_ON_ONCE(q->ordered_in_driver);
> > +
> > if (q->start_streaming_called)
> > __enqueue_in_driver(vb);
> > return;
> > default:
> > + if (state == VB2_BUF_STATE_ERROR)
> > + dma_fence_set_error(vb->out_fence, -ENOENT);
> > + dma_fence_signal(vb->out_fence);
> > + dma_fence_put(vb->out_fence);
> > + vb->out_fence = NULL;
> > + vb->out_fence_fd = -1;
> > +
> > /* Inform any processes that may be waiting for buffers */
> > wake_up(&q->done_wq);
> > break;
> > @@ -1235,6 +1250,9 @@ static void __enqueue_in_driver(struct vb2_buffer *vb)
> > trace_vb2_buf_queue(q, vb);
> >
> > call_void_vb_qop(vb, buf_queue, vb);
> > +
> > + if (!(q->is_output || q->ordered_in_vb2))
> > + call_void_bufop(q, send_out_fence, vb);
> > }
> >
> > static int __buf_prepare(struct vb2_buffer *vb, const void *pb)
> > @@ -1451,6 +1469,7 @@ static struct dma_fence *__set_in_fence(struct vb2_queue *q,
> > }
> >
> > q->last_fence = dma_fence_get(fence);
> > + call_void_bufop(q, send_out_fence, vb);
> > }
> >
> > return fence;
> > @@ -1840,6 +1859,11 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
> > }
> >
> > /*
> > + * Renew out-fence context.
> > + */
>
> Why is that? I don't think I understand the nuances of fence contexts.
Because inside each context we should maintain ordering of the fences.
If we cancel the stream and restart it with we need a new context. Look
at ordering between two different streams doesn't make sense.
>
> > + q->out_fence_context = dma_fence_context_alloc(1);
> > +
> > + /*
> > * Remove all buffers from videobuf's list...
> > */
> > INIT_LIST_HEAD(&q->queued_list);
> > @@ -2171,6 +2195,7 @@ int vb2_core_queue_init(struct vb2_queue *q)
> > spin_lock_init(&q->done_lock);
> > mutex_init(&q->mmap_lock);
> > init_waitqueue_head(&q->done_wq);
> > + q->out_fence_context = dma_fence_context_alloc(1);
> >
> > if (q->buf_struct_size == 0)
> > q->buf_struct_size = sizeof(struct vb2_buffer);
> > diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c b/drivers/media/v4l2-core/videobuf2-v4l2.c
> > index 4c09ea007d90..9fb01ddefdc9 100644
> > --- a/drivers/media/v4l2-core/videobuf2-v4l2.c
> > +++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
> > @@ -32,6 +32,11 @@
> >
> > #include <media/videobuf2-v4l2.h>
> >
> > +struct out_fence_data {
> > + int fence_fd;
> > + struct file *file;
> > +};
> > +
> > static int debug;
> > module_param(debug, int, 0644);
> >
> > @@ -138,6 +143,38 @@ static void __copy_timestamp(struct vb2_buffer *vb, const void *pb)
> > }
> > };
> >
> > +static void __fd_install_at_dequeue_cb(void *data)
> > +{
> > + struct out_fence_data *of = data;
> > +
> > + fd_install(of->fence_fd, of->file);
> > + kfree(of);
>
> Is it possible for the user to never dequeue the event? In that case
> the fd and sync_file would leak I guess.
Yes. It is totally possible. That fell through...
>
> > +}
> > +
> > +static void __send_out_fence(struct vb2_buffer *vb)
> > +{
> > + struct video_device *vdev = to_video_device(vb->vb2_queue->dev);
> > + struct v4l2_fh *fh = vdev->queue->owner;
> > + struct v4l2_event event;
> > + struct out_fence_data *of;
> > +
> > + if (vb->out_fence_fd < 0)
> > + return;
> > +
> > + memset(&event, 0, sizeof(event));
> > + event.type = V4L2_EVENT_OUT_FENCE;
> > + event.u.out_fence.index = vb->index;
> > + event.u.out_fence.out_fence_fd = vb->out_fence_fd;
> > +
> > + of = kmalloc(sizeof(*of), GFP_KERNEL);
> > + of->fence_fd = vb->out_fence_fd;
> > + of->file = vb->sync_file->file;
> > +
> > + v4l2_event_queue_fh_with_cb(fh, &event, __fd_install_at_dequeue_cb, of);
> > +
> > + vb->sync_file = NULL;
>
> See the comment above about never dequeuing the event - maybe you need
> to keep hold of the sync file to be able to clean it up in-case the
> user never dequeues. If not, then you could set vb->out_fence_fd = -1
> here too.
That is a good suggestion.
> I've been looking at your v4l2-fences branch on kernel.org, which
> looks quite different. Is this the newer version?
Yes. I wrote this in a hurry to send it to the mailing list before
the Media Summit, so I probably forgot to push and left a few issues
around.
Regards,
Gustavo
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 09/17] [media] vb2: add 'ordered_in_vb2' property to queues
2017-10-20 21:50 ` [RFC v4 09/17] [media] vb2: add 'ordered_in_vb2' " Gustavo Padovan
@ 2017-11-03 7:23 ` Hans Verkuil
0 siblings, 0 replies; 32+ messages in thread
From: Hans Verkuil @ 2017-11-03 7:23 UTC (permalink / raw)
To: Gustavo Padovan, linux-media
Cc: Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
On 10/20/2017 11:50 PM, Gustavo Padovan wrote:
> From: Gustavo Padovan <gustavo.padovan@collabora.com>
>
> By setting this member on vb2_queue the driver tell vb2 core that
> it requires the buffers queued in QBUF to be queued with same order to the
> driver.
As we discussed in Prague this appears to be an unnecessary flag. In fact,
as far as I can tell it isn't set at all in this patch series.
Regards,
Hans
>
> Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
> ---
> include/media/videobuf2-core.h | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 6dd3f0181107..fc333e10e7d8 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -505,6 +505,9 @@ struct vb2_buf_ops {
> * the same order they are dequeued from the driver. The default
> * is not ordered unless the driver sets this flag. As of now it
> * is mandatory for using explicit fences.
> + * @ordered_in_vb2: set by the driver to tell vb2 te guarantee the order
> + * of buffer queue from userspace with QBUF() until they are
> + * queued to the driver.
> * @fileio: file io emulator internal data, used only if emulator is active
> * @threadio: thread io internal data, used only if thread is active
> */
> @@ -558,6 +561,7 @@ struct vb2_queue {
> unsigned int copy_timestamp:1;
> unsigned int last_buffer_dequeued:1;
> unsigned int ordered_in_driver:1;
> + unsigned int ordered_in_vb2:1;
>
> struct vb2_fileio_data *fileio;
> struct vb2_threadio_data *threadio;
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 04/17] WIP: [media] v4l2: add v4l2_event_queue_fh_with_cb()
2017-10-20 21:49 ` [RFC v4 04/17] WIP: [media] v4l2: add v4l2_event_queue_fh_with_cb() Gustavo Padovan
@ 2017-11-03 7:31 ` Hans Verkuil
0 siblings, 0 replies; 32+ messages in thread
From: Hans Verkuil @ 2017-11-03 7:31 UTC (permalink / raw)
To: Gustavo Padovan, linux-media
Cc: Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
On 10/20/2017 11:49 PM, Gustavo Padovan wrote:
> From: Gustavo Padovan <gustavo.padovan@collabora.com>
>
> For some type of events we may require the event user in the
> kernel to run some operation when DQ_EVENT() is called.
> V4L2_EVENT_OUT_FENCE is the first user of this mechanism as it needs
> to call v4l2 core back to install a file descriptor.
>
> This is WIP, I believe we are able to come up with better ways to do it,
> but as that is not the main part of the patchset I'll keep it like
> this for now.
I'm OK with this callback. I don't off-hand see a better way of doing this.
>
> Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
> ---
> drivers/media/v4l2-core/v4l2-event.c | 31 +++++++++++++++++++++++++++----
> include/media/v4l2-event.h | 7 +++++++
> 2 files changed, 34 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-event.c b/drivers/media/v4l2-core/v4l2-event.c
> index 313ee9d1f9ee..6274e3e174e0 100644
> --- a/drivers/media/v4l2-core/v4l2-event.c
> +++ b/drivers/media/v4l2-core/v4l2-event.c
> @@ -58,6 +58,9 @@ static int __v4l2_event_dequeue(struct v4l2_fh *fh, struct v4l2_event *event)
>
> spin_unlock_irqrestore(&fh->vdev->fh_lock, flags);
>
> + if (kev->prepare)
> + kev->prepare(kev->data);
> +
> return 0;
> }
>
> @@ -104,8 +107,11 @@ static struct v4l2_subscribed_event *v4l2_event_subscribed(
> return NULL;
> }
>
> -static void __v4l2_event_queue_fh(struct v4l2_fh *fh, const struct v4l2_event *ev,
> - const struct timespec *ts)
> +static void __v4l2_event_queue_fh_with_cb(struct v4l2_fh *fh,
I wouldn't rename this function, just add the two arguments.
> + const struct v4l2_event *ev,
> + const struct timespec *ts,
> + void (*prepare)(void *data),
> + void *data)
> {
> struct v4l2_subscribed_event *sev;
> struct v4l2_kevent *kev;
> @@ -155,6 +161,8 @@ static void __v4l2_event_queue_fh(struct v4l2_fh *fh, const struct v4l2_event *e
> kev->event.id = ev->id;
> kev->event.timestamp = *ts;
> kev->event.sequence = fh->sequence;
> + kev->prepare = prepare;
> + kev->data = data;
> sev->in_use++;
> list_add_tail(&kev->list, &fh->available);
>
> @@ -177,7 +185,7 @@ void v4l2_event_queue(struct video_device *vdev, const struct v4l2_event *ev)
> spin_lock_irqsave(&vdev->fh_lock, flags);
>
> list_for_each_entry(fh, &vdev->fh_list, list)
> - __v4l2_event_queue_fh(fh, ev, ×tamp);
> + __v4l2_event_queue_fh_with_cb(fh, ev, ×tamp, NULL, NULL);
>
> spin_unlock_irqrestore(&vdev->fh_lock, flags);
> }
> @@ -191,11 +199,26 @@ void v4l2_event_queue_fh(struct v4l2_fh *fh, const struct v4l2_event *ev)
> ktime_get_ts(×tamp);
>
> spin_lock_irqsave(&fh->vdev->fh_lock, flags);
> - __v4l2_event_queue_fh(fh, ev, ×tamp);
> + __v4l2_event_queue_fh_with_cb(fh, ev, ×tamp, NULL, NULL);
> spin_unlock_irqrestore(&fh->vdev->fh_lock, flags);
> }
> EXPORT_SYMBOL_GPL(v4l2_event_queue_fh);
I'd drop this function and replace it with a static inline in the v4l2-event.h
header that calls v4l2_event_queue_fh_with_cb.
>
> +void v4l2_event_queue_fh_with_cb(struct v4l2_fh *fh,
> + const struct v4l2_event *ev,
> + void (*prepare)(void *data), void *data)
> +{
> + unsigned long flags;
> + struct timespec timestamp;
> +
> + ktime_get_ts(×tamp);
> +
> + spin_lock_irqsave(&fh->vdev->fh_lock, flags);
> + __v4l2_event_queue_fh_with_cb(fh, ev, ×tamp, prepare, data);
> + spin_unlock_irqrestore(&fh->vdev->fh_lock, flags);
> +}
> +EXPORT_SYMBOL_GPL(v4l2_event_queue_fh_with_cb);
> +
> int v4l2_event_pending(struct v4l2_fh *fh)
> {
> return fh->navailable;
> diff --git a/include/media/v4l2-event.h b/include/media/v4l2-event.h
> index 2b794f2ad824..dc770257811e 100644
> --- a/include/media/v4l2-event.h
> +++ b/include/media/v4l2-event.h
> @@ -68,11 +68,14 @@ struct video_device;
> * @list: List node for the v4l2_fh->available list.
> * @sev: Pointer to parent v4l2_subscribed_event.
> * @event: The event itself.
> + * @prepare: Callback to prepare the event only at DQ_EVENT() time.
@data is not documented.
> */
> struct v4l2_kevent {
> struct list_head list;
> struct v4l2_subscribed_event *sev;
> struct v4l2_event event;
> + void (*prepare)(void *data);
> + void *data;
> };
>
> /**
> @@ -160,6 +163,10 @@ void v4l2_event_queue(struct video_device *vdev, const struct v4l2_event *ev);
> */
> void v4l2_event_queue_fh(struct v4l2_fh *fh, const struct v4l2_event *ev);
>
> +void v4l2_event_queue_fh_with_cb(struct v4l2_fh *fh,
> + const struct v4l2_event *ev,
> + void (*prepare)(void *data), void *data);
> +
Missing kerneldoc documentation.
> /**
> * v4l2_event_pending - Check if an event is available
> *
>
Regards,
Hans
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 12/17] [media] vb2: add explicit fence user API
2017-10-20 21:50 ` [RFC v4 12/17] [media] vb2: add explicit fence user API Gustavo Padovan
@ 2017-11-03 7:47 ` Hans Verkuil
0 siblings, 0 replies; 32+ messages in thread
From: Hans Verkuil @ 2017-11-03 7:47 UTC (permalink / raw)
To: Gustavo Padovan, linux-media
Cc: Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
On 10/20/2017 11:50 PM, Gustavo Padovan wrote:
> From: Gustavo Padovan <gustavo.padovan@collabora.com>
>
> Turn the reserved2 field into fence_fd that we will use to send
> an in-fence to the kernel and return an out-fence from the kernel to
> userspace.
>
> Two new flags were added, V4L2_BUF_FLAG_IN_FENCE, that should be used
> when sending a fence to the kernel to be waited on, and
> V4L2_BUF_FLAG_OUT_FENCE, to ask the kernel to give back an out-fence.
>
> v3:
> - make the out_fence refer to the current buffer (Hans)
>
> v2: add documentation
>
> Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
> ---
> Documentation/media/uapi/v4l/buffer.rst | 15 +++++++++++++++
> drivers/media/usb/cpia2/cpia2_v4l.c | 2 +-
> drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 4 ++--
> drivers/media/v4l2-core/videobuf2-v4l2.c | 2 +-
> include/uapi/linux/videodev2.h | 4 +++-
> 5 files changed, 22 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
> index ae6ee73f151c..99afc0837671 100644
> --- a/Documentation/media/uapi/v4l/buffer.rst
> +++ b/Documentation/media/uapi/v4l/buffer.rst
> @@ -648,6 +648,21 @@ Buffer Flags
> - Start Of Exposure. The buffer timestamp has been taken when the
> exposure of the frame has begun. This is only valid for the
> ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` buffer type.
> + * .. _`V4L2-BUF-FLAG-IN-FENCE`:
> +
> + - ``V4L2_BUF_FLAG_IN_FENCE``
> + - 0x00200000
> + - Ask V4L2 to wait on fence passed in ``fence_fd`` field. The buffer
> + won't be queued to the driver until the fence signals.
> +
> + * .. _`V4L2-BUF-FLAG-OUT-FENCE`:
> +
> + - ``V4L2_BUF_FLAG_OUT_FENCE``
> + - 0x00400000
> + - Request a fence to be attached to the buffer. One should listen for
> + the ``V4L2_EVENT_OUT_FENCE`` event to receive the buffer ``index``
> + and the ``out_fence_fd`` attached to the buffer. The event is
> + triggered at the moment the buffer is queued to the V4L2 driver.
>
>
>
> diff --git a/drivers/media/usb/cpia2/cpia2_v4l.c b/drivers/media/usb/cpia2/cpia2_v4l.c
> index 4666a642b2f6..d9d6b6a31751 100644
> --- a/drivers/media/usb/cpia2/cpia2_v4l.c
> +++ b/drivers/media/usb/cpia2/cpia2_v4l.c
> @@ -948,7 +948,7 @@ static int cpia2_dqbuf(struct file *file, void *fh, struct v4l2_buffer *buf)
> buf->sequence = cam->buffers[buf->index].seq;
> buf->m.offset = cam->buffers[buf->index].data - cam->frame_buffer;
> buf->length = cam->frame_size;
> - buf->reserved2 = 0;
> + buf->fence_fd = -1;
> buf->reserved = 0;
> memset(&buf->timecode, 0, sizeof(buf->timecode));
>
> diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> index 821f2aa299ae..3a31d318df2a 100644
> --- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> +++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> @@ -370,7 +370,7 @@ struct v4l2_buffer32 {
> __s32 fd;
> } m;
> __u32 length;
> - __u32 reserved2;
> + __s32 fence_fd;
> __u32 reserved;
> };
>
> @@ -533,7 +533,7 @@ static int put_v4l2_buffer32(struct v4l2_buffer *kp, struct v4l2_buffer32 __user
> put_user(kp->timestamp.tv_usec, &up->timestamp.tv_usec) ||
> copy_to_user(&up->timecode, &kp->timecode, sizeof(struct v4l2_timecode)) ||
> put_user(kp->sequence, &up->sequence) ||
> - put_user(kp->reserved2, &up->reserved2) ||
> + put_user(kp->fence_fd, &up->fence_fd) ||
> put_user(kp->reserved, &up->reserved) ||
> put_user(kp->length, &up->length))
> return -EFAULT;
> diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c b/drivers/media/v4l2-core/videobuf2-v4l2.c
> index 0c0669976bdc..110fb45fef6f 100644
> --- a/drivers/media/v4l2-core/videobuf2-v4l2.c
> +++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
> @@ -203,7 +203,7 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
> b->timestamp = ns_to_timeval(vb->timestamp);
> b->timecode = vbuf->timecode;
> b->sequence = vbuf->sequence;
> - b->reserved2 = 0;
> + b->fence_fd = -1;
> b->reserved = 0;
>
> if (q->is_multiplanar) {
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 2a432e8c18e3..8975fec4ab68 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -924,7 +924,7 @@ struct v4l2_buffer {
> __s32 fd;
> } m;
> __u32 length;
> - __u32 reserved2;
> + __s32 fence_fd;
I think this should become:
union {
__s32 fence_fd;
__u32 reserved2;
};
Currently applications set reserved2 to 0:
buf.reserved2 = 0;
But if we just replace reserved2 by fence_fd this will no longer compile and
Linus will start yelling at you (and more importantly, us :-) ).
Regards,
Hans
> __u32 reserved;
> };
>
> @@ -961,6 +961,8 @@ struct v4l2_buffer {
> #define V4L2_BUF_FLAG_TSTAMP_SRC_SOE 0x00010000
> /* mem2mem encoder/decoder */
> #define V4L2_BUF_FLAG_LAST 0x00100000
> +#define V4L2_BUF_FLAG_IN_FENCE 0x00200000
> +#define V4L2_BUF_FLAG_OUT_FENCE 0x00400000
>
> /**
> * struct v4l2_exportbuffer - export of video buffer as DMABUF file descriptor
>
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 17/17] [media] v4l: Document explicit synchronization behaviour
2017-10-20 21:50 ` [RFC v4 17/17] [media] v4l: Document explicit synchronization behaviour Gustavo Padovan
2017-10-27 10:08 ` Brian Starkey
@ 2017-11-03 7:53 ` Hans Verkuil
2017-11-03 13:32 ` Gustavo Padovan
1 sibling, 1 reply; 32+ messages in thread
From: Hans Verkuil @ 2017-11-03 7:53 UTC (permalink / raw)
To: Gustavo Padovan, linux-media
Cc: Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
On 10/20/2017 11:50 PM, Gustavo Padovan wrote:
> From: Gustavo Padovan <gustavo.padovan@collabora.com>
>
> Add section to VIDIOC_QBUF about it
>
> v3:
> - make the out_fence refer to the current buffer (Hans)
> - Note what happens when the IN_FENCE is not set (Hans)
>
> v2:
> - mention that fences are files (Hans)
> - rework for the new API
>
> Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
> ---
> Documentation/media/uapi/v4l/vidioc-qbuf.rst | 31 ++++++++++++++++++++++++++++
> 1 file changed, 31 insertions(+)
>
> diff --git a/Documentation/media/uapi/v4l/vidioc-qbuf.rst b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
> index 9e448a4aa3aa..a65a50578bad 100644
> --- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
> @@ -118,6 +118,37 @@ immediately with an ``EAGAIN`` error code when no buffer is available.
> The struct :c:type:`v4l2_buffer` structure is specified in
> :ref:`buffer`.
>
> +Explicit Synchronization
> +------------------------
> +
> +Explicit Synchronization allows us to control the synchronization of
> +shared buffers from userspace by passing fences to the kernel and/or
> +receiving them from it. Fences passed to the kernel are named in-fences and
> +the kernel should wait on them to signal before using the buffer, i.e., queueing
> +it to the driver. On the other side, the kernel can create out-fences for the
> +buffers it queues to the drivers. Out-fences signal when the driver is
> +finished with buffer, i.e., the buffer is ready. The fences are represented
> +as a file and passed as a file descriptor to userspace.
> +
> +The in-fences are communicated to the kernel at the ``VIDIOC_QBUF`` ioctl
> +using the ``V4L2_BUF_FLAG_IN_FENCE`` buffer
> +flags and the `fence_fd` field. If an in-fence needs to be passed to the kernel,
> +`fence_fd` should be set to the fence file descriptor number and the
> +``V4L2_BUF_FLAG_IN_FENCE`` should be set as well Setting one but not the other
Missing '.' after 'as well'.
To what value is fence_fd set when VIDIOC_QBUF returns? If you don't set the
IN_FENCE flag, what should userspace set fence_fd to? (I recommend 0).
> +will cause ``VIDIOC_QBUF`` to return with error.
> +
> +The fence_fd field (formely the reserved2 field) will be ignored if the
Drop the "(formely the reserved2 field)" part. We're not interested in the
history here.
> +``V4L2_BUF_FLAG_IN_FENCE`` is not set.
> +
> +To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE`` flag should
> +be set to ask for a fence to be attached to the buffer. To become aware of
> +the out-fence created one should listen for the ``V4L2_EVENT_OUT_FENCE`` event.
> +An event will be triggered for every buffer queued to the V4L2 driver with the
> +``V4L2_BUF_FLAG_OUT_FENCE``.
> +
> +At streamoff the out-fences will either signal normally if the drivers waits
drivers -> driver
> +for the operations on the buffers to finish or signal with error if the
error -> an error
> +driver cancels the pending operations.
>
> Return Value
> ============
>
What should be done if the driver doesn't set ordered_in_driver? How does userspace
know whether in and/or out fences are supported? I'm leaning towards a new capability
flag for QUERYCAPS.
What does VIDIOC_QUERYBUF return w.r.t. the fence flags and fence_fd?
Regards,
Hans
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 17/17] [media] v4l: Document explicit synchronization behaviour
2017-11-03 7:53 ` Hans Verkuil
@ 2017-11-03 13:32 ` Gustavo Padovan
0 siblings, 0 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-11-03 13:32 UTC (permalink / raw)
To: Hans Verkuil
Cc: linux-media, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
Alexandre Courbot, Sakari Ailus, Brian Starkey, linux-kernel,
Gustavo Padovan
Hi Hans,
2017-11-03 Hans Verkuil <hverkuil@xs4all.nl>:
> On 10/20/2017 11:50 PM, Gustavo Padovan wrote:
> > From: Gustavo Padovan <gustavo.padovan@collabora.com>
> >
> > Add section to VIDIOC_QBUF about it
> >
> > v3:
> > - make the out_fence refer to the current buffer (Hans)
> > - Note what happens when the IN_FENCE is not set (Hans)
> >
> > v2:
> > - mention that fences are files (Hans)
> > - rework for the new API
> >
> > Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
> > ---
> > Documentation/media/uapi/v4l/vidioc-qbuf.rst | 31 ++++++++++++++++++++++++++++
> > 1 file changed, 31 insertions(+)
> >
> > diff --git a/Documentation/media/uapi/v4l/vidioc-qbuf.rst b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
> > index 9e448a4aa3aa..a65a50578bad 100644
> > --- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst
> > +++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
> > @@ -118,6 +118,37 @@ immediately with an ``EAGAIN`` error code when no buffer is available.
> > The struct :c:type:`v4l2_buffer` structure is specified in
> > :ref:`buffer`.
> >
> > +Explicit Synchronization
> > +------------------------
> > +
> > +Explicit Synchronization allows us to control the synchronization of
> > +shared buffers from userspace by passing fences to the kernel and/or
> > +receiving them from it. Fences passed to the kernel are named in-fences and
> > +the kernel should wait on them to signal before using the buffer, i.e., queueing
> > +it to the driver. On the other side, the kernel can create out-fences for the
> > +buffers it queues to the drivers. Out-fences signal when the driver is
> > +finished with buffer, i.e., the buffer is ready. The fences are represented
> > +as a file and passed as a file descriptor to userspace.
> > +
> > +The in-fences are communicated to the kernel at the ``VIDIOC_QBUF`` ioctl
> > +using the ``V4L2_BUF_FLAG_IN_FENCE`` buffer
> > +flags and the `fence_fd` field. If an in-fence needs to be passed to the kernel,
> > +`fence_fd` should be set to the fence file descriptor number and the
> > +``V4L2_BUF_FLAG_IN_FENCE`` should be set as well Setting one but not the other
>
> Missing '.' after 'as well'.
>
> To what value is fence_fd set when VIDIOC_QBUF returns?
It should be -1 because we will be reusing the fence_fd field to return
the out_fence to userspace in the cases we don't need to use the
OUT_FENCE event. Like GPU drivers does with fences. That is the better
way to send the out fence back that I can think of at the moment.
> If you don't set the
> IN_FENCE flag, what should userspace set fence_fd to? (I recommend 0).
0 is still a valid fd, so the implementation is currently accepting -1
and 0.
>
> > +will cause ``VIDIOC_QBUF`` to return with error.
> > +
> > +The fence_fd field (formely the reserved2 field) will be ignored if the
>
> Drop the "(formely the reserved2 field)" part. We're not interested in the
> history here.
>
> > +``V4L2_BUF_FLAG_IN_FENCE`` is not set.
> > +
> > +To get an out-fence back from V4L2 the ``V4L2_BUF_FLAG_OUT_FENCE`` flag should
> > +be set to ask for a fence to be attached to the buffer. To become aware of
> > +the out-fence created one should listen for the ``V4L2_EVENT_OUT_FENCE`` event.
> > +An event will be triggered for every buffer queued to the V4L2 driver with the
> > +``V4L2_BUF_FLAG_OUT_FENCE``.
> > +
> > +At streamoff the out-fences will either signal normally if the drivers waits
>
> drivers -> driver
>
> > +for the operations on the buffers to finish or signal with error if the
>
> error -> an error
>
> > +driver cancels the pending operations.
> >
> > Return Value
> > ============
> >
>
> What should be done if the driver doesn't set ordered_in_driver? How does userspace
> know whether in and/or out fences are supported? I'm leaning towards a new capability
> flag for QUERYCAPS.
Yep. That is what we agreed last week in Prague.
> What does VIDIOC_QUERYBUF return w.r.t. the fence flags and fence_fd?
It does return the IN_FENCE flag if the fence didn't signal yet and the
OUT_FENCE one if the user set it on QBUF. The fence_fd is set to -1,
because the fd is specific to the pid using it.
Gustavo
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [RFC v4 13/17] [media] vb2: add in-fence support to QBUF
2017-10-25 14:49 ` Brian Starkey
@ 2017-11-10 12:38 ` Gustavo Padovan
0 siblings, 0 replies; 32+ messages in thread
From: Gustavo Padovan @ 2017-11-10 12:38 UTC (permalink / raw)
To: Brian Starkey
Cc: linux-media, Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan,
Pawel Osciak, Alexandre Courbot, Sakari Ailus, linux-kernel,
Gustavo Padovan
2017-10-25 Brian Starkey <brian.starkey@arm.com>:
> Hi Gustavo,
>
> On Fri, Oct 20, 2017 at 07:50:08PM -0200, Gustavo Padovan wrote:
> > From: Gustavo Padovan <gustavo.padovan@collabora.com>
> >
> > Receive in-fence from userspace and add support for waiting on them
> > before queueing the buffer to the driver. Buffers are only queued
> > to the driver once they are ready. A buffer is ready when its
> > in-fence signals.
> >
> > For queues that require vb2 to queue buffers to the v4l2 driver in same
> > order they are received from userspace we use fence_array to keep that
> > ordering. Basically we create a fence_array that contains both the current
> > fence and the fence from the previous buffer (which might be a fence array
> > as well). The base fence class for the fence_array becomes the new buffer
> > fence, waiting on that one guarantees that it won't be queued out of
> > order.
>
> The API sounds/looks good to me, makes sense to let driver opt in/out
> via the ordered_in_vb2 thing. Thanks for implementing it!
>
> >
> > v5: - use fence_array to keep buffers ordered in vb2 core when
> > needed (Brian Stark)
> > - keep backward compatibility on the reserved2 field (Brian Stark)
> > - protect fence callback removal with lock (Brian Stark)
>
> Brian Starkey, but close ;-)
>
> >
> > v4:
> > - Add a comment about dma_fence_add_callback() not returning a
> > error (Hans)
> > - Call dma_fence_put(vb->in_fence) if fence signaled (Hans)
> > - select SYNC_FILE under config VIDEOBUF2_CORE (Hans)
> > - Move dma_fence_is_signaled() check to __enqueue_in_driver() (Hans)
> > - Remove list_for_each_entry() in __vb2_core_qbuf() (Hans)
> > - Remove if (vb->state != VB2_BUF_STATE_QUEUED) from
> > vb2_start_streaming() (Hans)
> > - set IN_FENCE flags on __fill_v4l2_buffer (Hans)
> > - Queue buffers to the driver as soon as they are ready (Hans)
> > - call fill_user_buffer() after queuing the buffer (Hans)
> > - add err: label to clean up fence
> > - add dma_fence_wait() before calling vb2_start_streaming()
> >
> > v3: - document fence parameter
> > - remove ternary if at vb2_qbuf() return (Mauro)
> > - do not change if conditions behaviour (Mauro)
> >
> > v2:
> > - fix vb2_queue_or_prepare_buf() ret check
> > - remove check for VB2_MEMORY_DMABUF only (Javier)
> > - check num of ready buffers to start streaming
> > - when queueing, start from the first ready buffer
> > - handle queue cancel
> >
> > Signed-off-by: Gustavo Padovan <gustavo.padovan@collabora.com>
> > ---
> > drivers/media/v4l2-core/Kconfig | 1 +
> > drivers/media/v4l2-core/videobuf2-core.c | 179 ++++++++++++++++++++++++++++---
> > drivers/media/v4l2-core/videobuf2-v4l2.c | 29 ++++-
> > include/media/videobuf2-core.h | 17 ++-
> > 4 files changed, 208 insertions(+), 18 deletions(-)
> >
> > diff --git a/drivers/media/v4l2-core/Kconfig b/drivers/media/v4l2-core/Kconfig
> > index a35c33686abf..3f988c407c80 100644
> > --- a/drivers/media/v4l2-core/Kconfig
> > +++ b/drivers/media/v4l2-core/Kconfig
> > @@ -83,6 +83,7 @@ config VIDEOBUF_DVB
> > # Used by drivers that need Videobuf2 modules
> > config VIDEOBUF2_CORE
> > select DMA_SHARED_BUFFER
> > + select SYNC_FILE
> > tristate
> >
> > config VIDEOBUF2_MEMOPS
> > diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
> > index 60f8b582396a..78f369dba3e3 100644
> > --- a/drivers/media/v4l2-core/videobuf2-core.c
> > +++ b/drivers/media/v4l2-core/videobuf2-core.c
> > @@ -346,6 +346,7 @@ static int __vb2_queue_alloc(struct vb2_queue *q, enum vb2_memory memory,
> > vb->index = q->num_buffers + buffer;
> > vb->type = q->type;
> > vb->memory = memory;
> > + spin_lock_init(&vb->fence_cb_lock);
> > for (plane = 0; plane < num_planes; ++plane) {
> > vb->planes[plane].length = plane_sizes[plane];
> > vb->planes[plane].min_length = plane_sizes[plane];
> > @@ -1222,6 +1223,9 @@ static void __enqueue_in_driver(struct vb2_buffer *vb)
> > {
> > struct vb2_queue *q = vb->vb2_queue;
> >
> > + if (vb->in_fence && !dma_fence_is_signaled(vb->in_fence))
> > + return;
> > +
> > vb->state = VB2_BUF_STATE_ACTIVE;
> > atomic_inc(&q->owned_by_drv_count);
> >
> > @@ -1273,6 +1277,20 @@ static int __buf_prepare(struct vb2_buffer *vb, const void *pb)
> > return 0;
> > }
> >
> > +static int __get_num_ready_buffers(struct vb2_queue *q)
> > +{
> > + struct vb2_buffer *vb;
> > + int ready_count = 0;
> > +
> > + /* count num of buffers ready in front of the queued_list */
> > + list_for_each_entry(vb, &q->queued_list, queued_entry) {
> > + if (!vb->in_fence || dma_fence_is_signaled(vb->in_fence))
> > + ready_count++;
>
> I think there's still some races on vb->in_fence. Couldn't the
> callback on any of the buffers in the queued_list have their callback
> run at any moment?
>
> That could make this loop do bad things, and similarly the loop in
> vb2_start_streaming() which calls __enqueue_in_driver().
>
> > + }
> > +
> > + return ready_count;
> > +}
> > +
> > int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
> > {
> > struct vb2_buffer *vb;
> > @@ -1361,7 +1379,74 @@ static int vb2_start_streaming(struct vb2_queue *q)
> > return ret;
> > }
> >
> > -int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
> > +static struct dma_fence *__set_in_fence(struct vb2_queue *q,
> > + struct vb2_buffer *vb,
> > + struct dma_fence *fence)
> > +{
> > + if (q->last_fence && dma_fence_is_signaled(q->last_fence)) {
> > + dma_fence_put(q->last_fence);
> > + q->last_fence = NULL;
> > + }
> > +
> > + /* For drivers that require the ordering of buffers queued from
> > + * userspace to be the same it is queued to the driver - output
> > + * some m2m devices - we create a fence array with the fence from
> > + * the last queued buffer and this one, that way the fence for this
> > + * buffer can't signal before the last one.
> > + */
> > + if (q->is_output || q->ordered_in_vb2) {
>
> Just a stylistic thing - you could invert this conditional and return
> early, saving a level of indentation below.
>
> > + if (fence && q->last_fence) {
> > + struct dma_fence **fences;
> > + struct dma_fence_array *arr;
> > +
> > + fences = kcalloc(2, sizeof(*fences), GFP_KERNEL);
> > + if (!fences)
> > + return ERR_PTR(-ENOMEM);
> > +
> > + fences[0] = fence;
> > + fences[1] = q->last_fence;
> > +
> > + arr = dma_fence_array_create(2, fences,
> > + dma_fence_context_alloc(1),
> > + 1, false);
>
> I'm not sure if it's right to get a new fence_context every time here.
> The comment on dma_fence_context_counter sounds like the vb2_queue
> should have a single context of its own.
>
> > + if (!arr) {
> > + kfree(fences);
> > + return ERR_PTR(-ENOMEM);
> > + }
> > +
> > + fence = &arr->base;
> > + } else if (!fence && q->last_fence) {
> > + fence = dma_fence_get(q->last_fence);
> > + }
> > +
> > + q->last_fence = dma_fence_get(fence);
>
> Seems like the refcounting isn't quite right here.
>
> If you land in "else if (!fence && q->last_fence)", then "fence" gets
> a new reference to q->last_fence, and you take another one here -
> which looks like one too many.
>
> I _think_ the right thing to do is just give the previous reference on
> q->last_fence to "fence", without taking a new one:
>
> } else if (!fence && q->last_fence) {
> fence = q->last_fence;
> }
>
> That's analogous to the other path where fence is not-NULL, and the
> fence-array takes ownership of the existing references on fence and
> q->last_fence, without taking new references.
>
> Please double check my logic though!
>
> > + }
> > +
> > + return fence;
> > +}
> > +
> > +static void vb2_qbuf_fence_cb(struct dma_fence *f, struct dma_fence_cb *cb)
> > +{
> > + struct vb2_buffer *vb = container_of(cb, struct vb2_buffer, fence_cb);
> > + struct vb2_queue *q = vb->vb2_queue;
> > + unsigned long flags;
> > +
> > + spin_lock_irqsave(&vb->fence_cb_lock, flags);
> > + if (!vb->in_fence) {
> > + spin_unlock_irqrestore(&vb->fence_cb_lock, flags);
> > + return;
> > + }
> > +
> > + dma_fence_put(vb->in_fence);
> > + vb->in_fence = NULL;
> > + spin_unlock_irqrestore(&vb->fence_cb_lock, flags);
> > +
> > + if (q->start_streaming_called)
> > + __enqueue_in_driver(vb);
> > +}
> > +
> > +int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
> > + struct dma_fence *fence)
> > {
> > struct vb2_buffer *vb;
> > int ret;
> > @@ -1372,16 +1457,18 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
> > case VB2_BUF_STATE_DEQUEUED:
> > ret = __buf_prepare(vb, pb);
> > if (ret)
> > - return ret;
> > + goto err;
> > break;
> > case VB2_BUF_STATE_PREPARED:
> > break;
> > case VB2_BUF_STATE_PREPARING:
> > dprintk(1, "buffer still being prepared\n");
> > - return -EINVAL;
> > + ret = -EINVAL;
> > + goto err;
> > default:
> > dprintk(1, "invalid buffer state %d\n", vb->state);
> > - return -EINVAL;
> > + ret = -EINVAL;
> > + goto err;
> > }
> >
> > /*
> > @@ -1398,30 +1485,75 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
> >
> > trace_vb2_qbuf(q, vb);
> >
> > + vb->in_fence = __set_in_fence(q, vb, fence);
> > + if (IS_ERR(vb->in_fence)) {
> > + ret = PTR_ERR(vb->in_fence);
> > + goto err;
>
> Between here, __set_in_fence() and vb2_qbuf() there's a reference
> leaked on "fence" in the case of error. I'm not sure what semantics
> you'd like, but it should be clear whose responsibility it is to clear
> up "fence" if __set_in_fence() fails (at the moment, looks like no-one
> does).
>
> > + }
>
> ... and if __set_in_fence() succeeds, then vb->in_fence now owns the
> reference on "fence" that was passed in, so I guess you should set
> fence = NULL here.
>
> > +
> > + /*
> > + * If it is time to call vb2_start_streaming() wait for the fence
> > + * to signal first. Of course, this happens only once per streaming.
> > + * We want to run any step that might fail before we set the callback
> > + * to queue the fence when it signals.
> > + */
> > + if (fence && !q->start_streaming_called &&
>
> I think this should be vb->in_fence now, and the same for all further
> uses of "fence" in this function.
>
> > + __get_num_ready_buffers(q) == q->min_buffers_needed - 1)
> > + dma_fence_wait(fence, true);
> > +
> > /*
> > * If streamon has been called, and we haven't yet called
> > * start_streaming() since not enough buffers were queued, and
> > * we now have reached the minimum number of queued buffers,
> > * then we can finally call start_streaming().
> > - *
> > - * If already streaming, give the buffer to driver for processing.
> > - * If not, the buffer will be given to driver on next streamon.
> > */
> > if (q->streaming && !q->start_streaming_called &&
> > - q->queued_count >= q->min_buffers_needed) {
> > + __get_num_ready_buffers(q) >= q->min_buffers_needed) {
> > ret = vb2_start_streaming(q);
> > if (ret)
> > - return ret;
> > - } else if (q->start_streaming_called) {
> > - __enqueue_in_driver(vb);
> > + goto err;
> > + }
> > +
> > + /*
> > + * For explicit synchronization: If the fence didn't signal
> > + * yet we setup a callback to queue the buffer once the fence
> > + * signals, and then, return successfully. But if the fence
> > + * already signaled we lose the reference we held and queue the
> > + * buffer to the driver.
> > + */
> > + if (fence) {
> > + ret = dma_fence_add_callback(fence, &vb->fence_cb,
> > + vb2_qbuf_fence_cb);
> > + if (!ret)
> > + goto fill;
>
> Just spotted that you should probably handle the -EINVAL case here and
> jump to err.
>
> > +
> > + dma_fence_put(fence);
> > + vb->in_fence = NULL;
> > }
> >
> > +fill:
> > + /*
> > + * If already streaming and there is no fence to wait on
> > + * give the buffer to driver for processing.
> > + */
> > + if (q->start_streaming_called && !vb->in_fence)
> > + __enqueue_in_driver(vb);
>
> The callback could have run just before this, clearing vb->in_fence
> and ending up with __enqueue_in_driver() getting called twice.
>
> > +
> > /* Fill buffer information for the userspace */
> > if (pb)
> > call_void_bufop(q, fill_user_buffer, vb, pb);
> >
> > dprintk(2, "qbuf of buffer %d succeeded\n", vb->index);
> > return 0;
> > +
> > +err:
> > + if (vb->in_fence) {
> > + dma_fence_put(vb->in_fence);
> > + vb->in_fence = NULL;
> > + }
> > +
> > + return ret;
> > +
> > }
> > EXPORT_SYMBOL_GPL(vb2_core_qbuf);
> >
> > @@ -1632,6 +1764,8 @@ EXPORT_SYMBOL_GPL(vb2_core_dqbuf);
> > static void __vb2_queue_cancel(struct vb2_queue *q)
> > {
> > unsigned int i;
> > + struct vb2_buffer *vb;
> > + unsigned long flags;
> >
> > /*
> > * Tell driver to stop all transactions and release all queued
> > @@ -1659,6 +1793,21 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
> > q->queued_count = 0;
> > q->error = 0;
> >
> > + list_for_each_entry(vb, &q->queued_list, queued_entry) {
> > + spin_lock_irqsave(&vb->fence_cb_lock, flags);
> > + if (vb->in_fence) {
> > + dma_fence_remove_callback(vb->in_fence, &vb->fence_cb);
> > + dma_fence_put(vb->in_fence);
> > + vb->in_fence = NULL;
> > + }
> > + spin_unlock_irqrestore(&vb->fence_cb_lock, flags);
> > + }
> > +
> > + if (q->last_fence) {
> > + dma_fence_put(q->last_fence);
> > + q->last_fence = NULL;
> > + }
> > +
> > /*
> > * Remove all buffers from videobuf's list...
> > */
> > @@ -1720,7 +1869,7 @@ int vb2_core_streamon(struct vb2_queue *q, unsigned int type)
> > * Tell driver to start streaming provided sufficient buffers
> > * are available.
> > */
> > - if (q->queued_count >= q->min_buffers_needed) {
> > + if (__get_num_ready_buffers(q) >= q->min_buffers_needed) {
> > ret = v4l_vb2q_enable_media_source(q);
> > if (ret)
> > return ret;
> > @@ -2240,7 +2389,7 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
> > * Queue all buffers.
> > */
> > for (i = 0; i < q->num_buffers; i++) {
> > - ret = vb2_core_qbuf(q, i, NULL);
> > + ret = vb2_core_qbuf(q, i, NULL, NULL);
> > if (ret)
> > goto err_reqbufs;
> > fileio->bufs[i].queued = 1;
> > @@ -2419,7 +2568,7 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
> >
> > if (copy_timestamp)
> > b->timestamp = ktime_get_ns();
> > - ret = vb2_core_qbuf(q, index, NULL);
> > + ret = vb2_core_qbuf(q, index, NULL, NULL);
> > dprintk(5, "vb2_dbuf result: %d\n", ret);
> > if (ret)
> > return ret;
> > @@ -2522,7 +2671,7 @@ static int vb2_thread(void *data)
> > if (copy_timestamp)
> > vb->timestamp = ktime_get_ns();;
> > if (!threadio->stop)
> > - ret = vb2_core_qbuf(q, vb->index, NULL);
> > + ret = vb2_core_qbuf(q, vb->index, NULL, NULL);
> > call_void_qop(q, wait_prepare, q);
> > if (ret || threadio->stop)
> > break;
> > diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c b/drivers/media/v4l2-core/videobuf2-v4l2.c
> > index 110fb45fef6f..4c09ea007d90 100644
> > --- a/drivers/media/v4l2-core/videobuf2-v4l2.c
> > +++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
> > @@ -23,6 +23,7 @@
> > #include <linux/sched.h>
> > #include <linux/freezer.h>
> > #include <linux/kthread.h>
> > +#include <linux/sync_file.h>
> >
> > #include <media/v4l2-dev.h>
> > #include <media/v4l2-fh.h>
> > @@ -178,6 +179,12 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
> > return -EINVAL;
> > }
> >
> > + if ((b->fence_fd != 0 && b->fence_fd != -1) &&
> > + !(b->flags & V4L2_BUF_FLAG_IN_FENCE)) {
> > + dprintk(1, "%s: fence_fd set without IN_FENCE flag\n", opname);
> > + return -EINVAL;
> > + }
> > +
> > return __verify_planes_array(q->bufs[b->index], b);
> > }
> >
> > @@ -203,9 +210,14 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
> > b->timestamp = ns_to_timeval(vb->timestamp);
> > b->timecode = vbuf->timecode;
> > b->sequence = vbuf->sequence;
> > - b->fence_fd = -1;
> > b->reserved = 0;
> >
> > + b->fence_fd = -1;
> > + if (vb->in_fence)
> > + b->flags |= V4L2_BUF_FLAG_IN_FENCE;
> > + else
> > + b->flags &= ~V4L2_BUF_FLAG_IN_FENCE;
> > +
> > if (q->is_multiplanar) {
> > /*
> > * Fill in plane-related data if userspace provided an array
> > @@ -560,6 +572,7 @@ EXPORT_SYMBOL_GPL(vb2_create_bufs);
> >
> > int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
> > {
> > + struct dma_fence *fence = NULL;
> > int ret;
> >
> > if (vb2_fileio_is_active(q)) {
> > @@ -568,7 +581,19 @@ int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
> > }
> >
> > ret = vb2_queue_or_prepare_buf(q, b, "qbuf");
> > - return ret ? ret : vb2_core_qbuf(q, b->index, b);
> > + if (ret)
> > + return ret;
> > +
> > + if (b->flags & V4L2_BUF_FLAG_IN_FENCE) {
> > + fence = sync_file_get_fence(b->fence_fd);
>
> In DRM a fence_fd of -1 means "already signaled" - should that be
> allowed here too?
If it is already signaled we shouldn't even set the IN_FENCE
flag. There isn't much to gain from that. The current code already makes
sure to guarantee the ordering if a buffer don't have a fence.
Gustavo
^ permalink raw reply [flat|nested] 32+ messages in thread
end of thread, other threads:[~2017-11-10 12:38 UTC | newest]
Thread overview: 32+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-10-20 21:49 [RFC v4 00/17] V4L2 Explicit Synchronization support Gustavo Padovan
2017-10-20 21:49 ` [RFC v4 01/17] [media] v4l: create v4l2_event_subscribe_v4l2() Gustavo Padovan
2017-10-24 12:51 ` Hans Verkuil
2017-10-20 21:49 ` [RFC v4 02/17] [media] v4l: use v4l2_subscribe_event_v4l2() on vtables Gustavo Padovan
2017-10-20 21:49 ` [RFC v4 03/17] [media] v4l: use v4l2_subscribe_event_v4l2() on drivers Gustavo Padovan
2017-10-20 21:49 ` [RFC v4 04/17] WIP: [media] v4l2: add v4l2_event_queue_fh_with_cb() Gustavo Padovan
2017-11-03 7:31 ` Hans Verkuil
2017-10-20 21:50 ` [RFC v4 05/17] [media] v4l: add V4L2_EVENT_OUT_FENCE event Gustavo Padovan
2017-10-24 13:09 ` Hans Verkuil
2017-10-20 21:50 ` [RFC v4 06/17] [media] vb2: add .send_out_fence() to notify userspace of out_fence_fd Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 07/17] [media] vivid: assign the specific device to the vb2_queue->dev Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 08/17] [media] vb2: add 'ordered_in_driver' property to queues Gustavo Padovan
2017-10-24 13:11 ` Hans Verkuil
2017-10-20 21:50 ` [RFC v4 09/17] [media] vb2: add 'ordered_in_vb2' " Gustavo Padovan
2017-11-03 7:23 ` Hans Verkuil
2017-10-20 21:50 ` [RFC v4 10/17] [media] vivid: mark vivid queues as ordered_in_driver Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 11/17] [media] vb2: check earlier if stream can be started Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 12/17] [media] vb2: add explicit fence user API Gustavo Padovan
2017-11-03 7:47 ` Hans Verkuil
2017-10-20 21:50 ` [RFC v4 13/17] [media] vb2: add in-fence support to QBUF Gustavo Padovan
2017-10-25 14:49 ` Brian Starkey
2017-11-10 12:38 ` Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 14/17] [media] vb2: add videobuf2 dma-buf fence helpers Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 15/17] [media] vb2: add infrastructure to support out-fences Gustavo Padovan
2017-10-27 9:41 ` Brian Starkey
2017-10-20 21:50 ` [RFC v4 16/17] [media] vb2: add out-fence support to QBUF Gustavo Padovan
2017-10-27 10:01 ` Brian Starkey
2017-11-03 0:03 ` Gustavo Padovan
2017-10-20 21:50 ` [RFC v4 17/17] [media] v4l: Document explicit synchronization behaviour Gustavo Padovan
2017-10-27 10:08 ` Brian Starkey
2017-11-03 7:53 ` Hans Verkuil
2017-11-03 13:32 ` Gustavo Padovan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).