linux-media.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC v5 00/11] V4L2 Explicit Synchronization
@ 2017-11-15 17:10 Gustavo Padovan
  2017-11-15 17:10 ` [RFC v5 01/11] [media] v4l: add V4L2_CAP_ORDERED to the uapi Gustavo Padovan
                   ` (11 more replies)
  0 siblings, 12 replies; 48+ messages in thread
From: Gustavo Padovan @ 2017-11-15 17:10 UTC (permalink / raw)
  To: linux-media
  Cc: Hans Verkuil, Mauro Carvalho Chehab, Shuah Khan, Pawel Osciak,
	Alexandre Courbot, Sakari Ailus, Brian Starkey, Thierry Escande,
	linux-kernel, Gustavo Padovan

From: Gustavo Padovan <gustavo.padovan@collabora.com>

Hi,

After the comments received in the last patchset[1] and 
during the media summit [2] here is the new and improved version
of the patchset. The implementation is simpler, smaller and cover
a lot more cases.

If you look to the last patchset I got rid of a few things, the main
one is the OUT_FENCE event, one thing that we decided in Prague was
that, when using fences, we would keep ordering of all buffers queued
to vb2. That means they would be queued to the drivers in the same order
that the QBUF calls happen, just like it already happens when not using
fences. Fences can signal in whatever order, so we need this guarantee
here. Drivers can, however, not keep ordering when processing the
buffers.

But there is one conclusion of that that we didn't reached at the
summit, maybe because of the order we discussed things, and that is: we do
not need the OUT_FENCE event anymore, because now at the QBUF call time
we *always* know the order in which the buffers will be queued to the
v4l2 driver. So the out-fence fd is now returned using the fence_fd
field as a return argument, thus the event is not necessary anymore.

The fence_fd field is now used to comunicate both in-fences and
out-fences, just like we do for GPU drivers. We pass in-fences as input
arguments and get out-fences as return arguments on the QBUF call.
The approach is documented.

I also added a capability flag, V4L2_CAP_ORDERED, to tell userspace if
the v4l2 drivers keep the buffers ordered or not. 

We still have the 'ordered_in_driver' property for queues, but its
meaning has changed. When set videobuf2 will know that the driver can
keep the order of the buffers, thus videobuf2 can use the same fence
context for all out-fences. Fences inside the same context should signal
in order, so 'ordered_in_driver' is a optimization for that case.
When not set, a context for each out-fence is created.

So now explicit synchronization also works for drivers that do not keep
the ordering of buffers.

Another thing is that we do not allow videobuf2 to requeue buffers
internally when using fences, they have a fence associated to it and
we need to finish the job on them, i.e., signal the fence, even if an
error happened.

The rest of the changes are documented in each patch separated.

There a test app at:

https://gitlab.collabora.com/padovan/v4l2-fences-test

Among my next steps is to create a v4l2->drm test app using fences as a
PoC, and also look into how to support it in ChromeOS.

Open Questions
--------------

* Do drivers reorder buffers internally? How to handle that with fences?

* How to handle audio/video syncronization? Fences aren't enough, we  need 
  to know things like the start of capture timestamp.

Regards,

Gustavo
--

[1] https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1518928.html
[2] http://muistio.tieke.fi/p/media-summit-2017

Gustavo Padovan (10):
  [media] v4l: add V4L2_CAP_ORDERED to the uapi
  [media] vivid: add the V4L2_CAP_ORDERED capability
  [media] vb2: add 'ordered_in_driver' property to queues
  [media] vivid: mark vivid queues as ordered_in_driver
  [media] vb2: check earlier if stream can be started
  [media] vb2: add explicit fence user API
  [media] vb2: add in-fence support to QBUF
  [media] vb2: add infrastructure to support out-fences
  [media] vb2: add out-fence support to QBUF
  [media] v4l: Document explicit synchronization behavior

Javier Martinez Canillas (1):
  [media] vb2: add videobuf2 dma-buf fence helpers

 Documentation/media/uapi/v4l/buffer.rst          |  15 ++
 Documentation/media/uapi/v4l/vidioc-qbuf.rst     |  42 +++-
 Documentation/media/uapi/v4l/vidioc-querybuf.rst |   9 +-
 Documentation/media/uapi/v4l/vidioc-querycap.rst |   3 +
 drivers/media/platform/vivid/vivid-core.c        |  24 +-
 drivers/media/usb/cpia2/cpia2_v4l.c              |   2 +-
 drivers/media/v4l2-core/Kconfig                  |   1 +
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c    |   4 +-
 drivers/media/v4l2-core/videobuf2-core.c         | 274 +++++++++++++++++++++--
 drivers/media/v4l2-core/videobuf2-v4l2.c         |  48 +++-
 include/media/videobuf2-core.h                   |  44 +++-
 include/media/videobuf2-fence.h                  |  48 ++++
 include/uapi/linux/videodev2.h                   |   8 +-
 13 files changed, 485 insertions(+), 37 deletions(-)
 create mode 100644 include/media/videobuf2-fence.h

-- 
2.13.6

^ permalink raw reply	[flat|nested] 48+ messages in thread

end of thread, other threads:[~2017-11-30 18:53 UTC | newest]

Thread overview: 48+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-15 17:10 [RFC v5 00/11] V4L2 Explicit Synchronization Gustavo Padovan
2017-11-15 17:10 ` [RFC v5 01/11] [media] v4l: add V4L2_CAP_ORDERED to the uapi Gustavo Padovan
2017-11-17 11:57   ` Mauro Carvalho Chehab
2017-11-17 12:23     ` Gustavo Padovan
2017-11-15 17:10 ` [RFC v5 02/11] [media] vivid: add the V4L2_CAP_ORDERED capability Gustavo Padovan
2017-11-15 17:10 ` [RFC v5 03/11] [media] vb2: add 'ordered_in_driver' property to queues Gustavo Padovan
2017-11-17  5:56   ` Alexandre Courbot
2017-11-17 11:23     ` Gustavo Padovan
2017-11-17 12:15   ` Mauro Carvalho Chehab
2017-11-17 12:27     ` Gustavo Padovan
2017-11-15 17:10 ` [RFC v5 04/11] [media] vivid: mark vivid queues as ordered_in_driver Gustavo Padovan
2017-11-15 17:10 ` [RFC v5 05/11] [media] vb2: check earlier if stream can be started Gustavo Padovan
2017-11-15 17:10 ` [RFC v5 06/11] [media] vb2: add explicit fence user API Gustavo Padovan
2017-11-17 12:25   ` Mauro Carvalho Chehab
2017-11-17 13:29   ` Hans Verkuil
2017-11-17 13:53     ` Mauro Carvalho Chehab
2017-11-17 14:31       ` Hans Verkuil
2017-11-15 17:10 ` [RFC v5 07/11] [media] vb2: add in-fence support to QBUF Gustavo Padovan
2017-11-17  6:49   ` Alexandre Courbot
2017-11-17 13:00     ` Mauro Carvalho Chehab
2017-11-17 13:08       ` Gustavo Padovan
2017-11-17 13:19         ` Mauro Carvalho Chehab
2017-11-20 11:41           ` Brian Starkey
2017-11-17 13:01     ` Gustavo Padovan
2017-11-20  2:53       ` Alexandre Courbot
2017-11-17 12:53   ` Mauro Carvalho Chehab
2017-11-17 13:12     ` Gustavo Padovan
2017-11-17 13:47       ` Mauro Carvalho Chehab
2017-11-17 17:20         ` Gustavo Padovan
2017-11-17 14:15   ` Hans Verkuil
2017-11-17 17:40     ` Gustavo Padovan
2017-11-17 17:50       ` Gustavo Padovan
2017-11-18  9:30       ` Hans Verkuil
2017-11-15 17:10 ` [RFC v5 08/11] [media] vb2: add videobuf2 dma-buf fence helpers Gustavo Padovan
2017-11-17  7:02   ` Alexandre Courbot
2017-11-17  7:11     ` Alexandre Courbot
2017-11-17 11:27       ` Gustavo Padovan
2017-11-15 17:10 ` [RFC v5 09/11] [media] vb2: add infrastructure to support out-fences Gustavo Padovan
2017-11-17  7:19   ` Alexandre Courbot
2017-11-17  7:29     ` Alexandre Courbot
2017-11-17 11:30     ` Gustavo Padovan
2017-11-15 17:10 ` [RFC v5 10/11] [media] vb2: add out-fence support to QBUF Gustavo Padovan
2017-11-17  7:38   ` Alexandre Courbot
2017-11-17 11:48     ` Gustavo Padovan
2017-11-17 13:34   ` Hans Verkuil
2017-11-15 17:10 ` [RFC v5 11/11] [media] v4l: Document explicit synchronization behavior Gustavo Padovan
2017-11-20 10:19 ` [RFC v5 00/11] V4L2 Explicit Synchronization Smitha T Murthy
2017-11-30 18:53   ` Gustavo Padovan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).