From: Sakari Ailus <sakari.ailus@iki.fi>
To: Gustavo Padovan <gustavo.padovan@collabora.com>
Cc: Gustavo Padovan <gustavo@padovan.org>,
linux-media@vger.kernel.org, Hans Verkuil <hverkuil@xs4all.nl>,
Mauro Carvalho Chehab <mchehab@osg.samsung.com>,
Laurent Pinchart <laurent.pinchart@ideasonboard.com>,
Javier Martinez Canillas <javier@osg.samsung.com>,
linux-kernel@vger.kernel.org
Subject: Re: [RFC 00/10] V4L2 explicit synchronization support
Date: Wed, 5 Apr 2017 23:43:06 +0300 [thread overview]
Message-ID: <20170405204306.GB3265@valkosipuli.retiisi.org.uk> (raw)
In-Reply-To: <20170405152457.GD32294@joana>
Hi Gustavo,
On Wed, Apr 05, 2017 at 05:24:57PM +0200, Gustavo Padovan wrote:
> Hi Sakari,
>
> 2017-04-04 Sakari Ailus <sakari.ailus@iki.fi>:
>
> > Hi Gustavo,
> >
> > Thank you for the patchset. Please see my comments below.
> >
> > On Mon, Mar 13, 2017 at 04:20:25PM -0300, Gustavo Padovan wrote:
> > > From: Gustavo Padovan <gustavo.padovan@collabora.com>
> > >
> > > Hi,
> > >
> > > This RFC adds support for Explicit Synchronization of shared buffers in V4L2.
> > > It uses the Sync File Framework[1] as vector to communicate the fences
> > > between kernel and userspace.
> > >
> > > I'm sending this to start the discussion on the best approach to implement
> > > Explicit Synchronization, please check the TODO/OPEN section below.
> > >
> > > Explicit Synchronization allows us to control the synchronization of
> > > shared buffers from userspace by passing fences to the kernel and/or
> > > receiving them from the the kernel.
> > >
> > > Fences passed to the kernel are named in-fences and the kernel should wait
> > > them to signal before using the buffer. On the other side, the kernel creates
> > > out-fences for every buffer it receives from userspace. This fence is sent back
> > > to userspace and it will signal when the capture, for example, has finished.
> > >
> > > Signalling an out-fence in V4L2 would mean that the job on the buffer is done
> > > and the buffer can be used by other drivers.
> >
> > Shouldn't you be able to add two fences to the buffer, one in and one out?
> > I.e. you'd have the buffer passed from another device to a V4L2 device and
> > on to a third device.
> >
> > (Or, two fences per a plane, as you elaborated below.)
>
> The out one should be created by V4L2 in this case, sent to userspace
> and then sent to third device. Another options is what we've been
> calling future fences in DRM. Where we may have a syscall to create this
> out-fence for us and then we could pass both in and out fence to the
> device. But that can be supported later along with what this RFC
> proposes.
Please excuse my ignorance on fences.
I just wanted to make sure that case was also considered. struct v4l2_buffer
will run out of space soon so we'll need a replacement anyway. The timecode
field is still available for re-use...
>
>
> >
> > >
> > > Current RFC implementation
> > > --------------------------
> > >
> > > The current implementation is not intended to be more than a PoC to start
> > > the discussion on how Explicit Synchronization should be supported in V4L2.
> > >
> > > The first patch proposes an userspace API for fences, then on patch 2
> > > we prepare to the addition of in-fences in patch 3, by introducing the
> > > infrastructure on vb2 to wait on an in-fence signal before queueing the buffer
> > > in the driver.
> > >
> > > Patch 4 fix uvc v4l2 event handling and patch 5 configure q->dev for vivid
> > > drivers to enable to subscribe and dequeue events on it.
> > >
> > > Patches 6-7 enables support to notify BUF_QUEUED events, i.e., let userspace
> > > know that particular buffer was enqueued in the driver. This is needed,
> > > because we return the out-fence fd as an out argument in QBUF, but at the time
> > > it returns we don't know to which buffer the fence will be attached thus
> > > the BUF_QUEUED event tells which buffer is associated to the fence received in
> > > QBUF by userspace.
> > >
> > > Patches 8 and 9 add more fence infrastructure to support out-fences and finally
> > > patch 10 adds support to out-fences.
> > >
> > > TODO/OPEN:
> > > ----------
> > >
> > > * For this first implementation we will keep the ordering of the buffers queued
> > > in videobuf2, that means we will only enqueue buffer whose fence was signalled
> > > if that buffer is the first one in the queue. Otherwise it has to wait until it
> > > is the first one. This is not implmented yet. Later we could create a flag to
> > > allow unordered queing in the drivers from vb2 if needed.
> > >
> > > * Should we have out-fences per-buffer or per-plane? or both? In this RFC, for
> > > simplicity they are per-buffer, but Mauro and Javier raised the option of
> > > doing per-plane fences. That could benefit mem2mem and V4L2 <-> GPU operation
> > > at least on cases when we have Capture hw that releases the Y frame before the
> > > other frames for example. When using V4L2 per-plane out-fences to communicate
> > > with KMS they would need to be merged together as currently the DRM Plane
> > > interface only supports one fence per DRM Plane.
> > >
> > > In-fences should be per-buffer as the DRM only has per-buffer fences, but
> > > in case of mem2mem operations per-plane fences might be useful?
> > >
> > > So should we have both ways, per-plane and per-buffer, or just one of them
> > > for now?
> >
> > The data_offset field is only present in struct v4l2_plane, i.e. it is only
> > available through using the multi-planar API even if you just have a single
> > plane.
>
> I didn't get why you mentioned the data_offset field. :)
I think I meant to continue this but didn't end up writing it down. :-)
What I wanted to say that the multi-plane API is a super-set of the
single-plane API and there's already a case for not all the functionality
being available through the single-plane API. At least I'm ok with adding
the per-plane fences to the multi-plane API only.
The framework could possibly do more to support the single-plane API as an
application interface so that the applications using single-plane API only
would get that as a bonus. (Just thinking out loud. Out of scope of this
patchset definitely.)
--
Kind regards,
Sakari Ailus
e-mail: sakari.ailus@iki.fi XMPP: sailus@retiisi.org.uk
next prev parent reply other threads:[~2017-04-05 20:43 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-03-13 19:20 [RFC 00/10] V4L2 explicit synchronization support Gustavo Padovan
2017-03-13 19:20 ` [RFC 01/10] [media] vb2: add explicit fence user API Gustavo Padovan
2017-04-03 9:48 ` Philipp Zabel
2017-04-05 14:08 ` Gustavo Padovan
2017-03-13 19:20 ` [RFC 02/10] [media] vb2: split out queueing from vb_core_qbuf() Gustavo Padovan
2017-03-13 19:20 ` [RFC 03/10] [media] vb2: add in-fence support to QBUF Gustavo Padovan
2017-04-03 18:27 ` Javier Martinez Canillas
2017-03-13 19:20 ` [RFC 04/10] [media] uvc: enable subscriptions to other events Gustavo Padovan
2017-03-13 19:20 ` [RFC 05/10] [media] vivid: assign the specific device to the vb2_queue->dev Gustavo Padovan
2017-03-13 19:20 ` [RFC 06/10] [media] v4l: add V4L2_EVENT_BUF_QUEUED event Gustavo Padovan
2017-03-13 19:20 ` [RFC 07/10] [media] v4l: add support to BUF_QUEUED event Gustavo Padovan
2017-03-13 19:20 ` [RFC 08/10] [media] vb2: add videobuf2 dma-buf fence helpers Gustavo Padovan
2017-03-13 19:20 ` [RFC 09/10] [media] vb2: add infrastructure to support out-fences Gustavo Padovan
2017-03-13 19:20 ` [RFC 10/10] [media] vb2: add out-fence support to QBUF Gustavo Padovan
2017-04-03 11:16 ` [RFC 00/10] V4L2 explicit synchronization support Mauro Carvalho Chehab
2017-04-03 19:46 ` Javier Martinez Canillas
2017-04-03 20:48 ` Shuah Khan
2017-04-05 15:09 ` Gustavo Padovan
2017-04-05 17:12 ` Javier Martinez Canillas
2017-04-06 14:08 ` Gustavo Padovan
2017-04-06 14:35 ` Javier Martinez Canillas
2017-06-09 15:38 ` Nicolas Dufresne
2017-04-04 11:34 ` Sakari Ailus
2017-04-05 15:24 ` Gustavo Padovan
2017-04-05 20:43 ` Sakari Ailus [this message]
2017-05-25 0:31 ` Gustavo Padovan
2017-06-08 20:17 ` Mauro Carvalho Chehab
2017-06-08 21:36 ` Shuah Khan
2017-06-09 6:25 ` Gustavo Padovan
2017-06-09 16:09 ` Shuah Khan
2017-06-09 6:15 ` Gustavo Padovan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170405204306.GB3265@valkosipuli.retiisi.org.uk \
--to=sakari.ailus@iki.fi \
--cc=gustavo.padovan@collabora.com \
--cc=gustavo@padovan.org \
--cc=hverkuil@xs4all.nl \
--cc=javier@osg.samsung.com \
--cc=laurent.pinchart@ideasonboard.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=mchehab@osg.samsung.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).