From: Tomasz Figa <tfiga@chromium.org>
To: Helen Koike <helen.koike@collabora.com>
Cc: Hans Verkuil <hverkuil@xs4all.nl>,
Mauro Carvalho Chehab <mchehab@kernel.org>,
Hans Verkuil <hans.verkuil@cisco.com>,
Laurent Pinchart <laurent.pinchart@ideasonboard.com>,
Sakari Ailus <sakari.ailus@iki.fi>,
Linux Media Mailing List <linux-media@vger.kernel.org>,
Boris Brezillon <boris.brezillon@collabora.com>,
Hirokazu Honda <hiroh@chromium.org>,
Nicolas Dufresne <nicolas@ndufresne.ca>,
Brian Starkey <Brian.Starkey@arm.com>,
kernel@collabora.com, Neil Armstrong <narmstrong@baylibre.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Fritz Koenig <frkoenig@chromium.org>,
Maxime Jourdan <mjourdan@baylibre.com>,
Stanimir Varbanov <stanimir.varbanov@linaro.org>
Subject: Re: [PATCH v5 2/7] media: v4l2: Add extended buffer operations
Date: Mon, 21 Dec 2020 12:13:04 +0900 [thread overview]
Message-ID: <CAAFQd5Ds5DQ0V+c_Oapwg9CQ0ADkjtML6w6H5Ad4hwMz9Rg9YQ@mail.gmail.com> (raw)
In-Reply-To: <4fec6e91-a19b-b0be-d4b6-72a333451d9b@collabora.com>
On Thu, Dec 17, 2020 at 10:20 PM Helen Koike <helen.koike@collabora.com> wrote:
>
> Hi Tomasz,
>
> Thanks for your comments, I have a few questions below.
>
> On 12/16/20 12:13 AM, Tomasz Figa wrote:
> > On Tue, Dec 15, 2020 at 11:37 PM Helen Koike <helen.koike@collabora.com> wrote:
> >>
> >> Hi Tomasz,
> >>
> >> On 12/14/20 7:46 AM, Tomasz Figa wrote:
> >>> On Fri, Dec 4, 2020 at 4:52 AM Helen Koike <helen.koike@collabora.com> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> Please see my 2 points below (about v4l2_ext_buffer and another about timestamp).
> >>>>
> >>>> On 12/3/20 12:11 PM, Hans Verkuil wrote:
> >>>>> On 23/11/2020 18:40, Helen Koike wrote:
> >>>>>>
> >>>>>>
> >>>>>> On 11/23/20 12:46 PM, Tomasz Figa wrote:
> >>>>>>> On Tue, Nov 24, 2020 at 12:08 AM Helen Koike <helen.koike@collabora.com> wrote:
> >>>>>>>>
> >>>>>>>> Hi Hans,
> >>>>>>>>
> >>>>>>>> Thank you for your review.
> >>>>>>>>
> >>>>>>>> On 9/9/20 9:27 AM, Hans Verkuil wrote:
> >>>>>>>>> Hi Helen,
> >>>>>>>>>
> >>>>>>>>> Again I'm just reviewing the uAPI.
> >>>>>>>>>
> >>>>>>>>> On 04/08/2020 21:29, Helen Koike wrote:
[snip]
> >
> >>
> >> Output: userspace fills plane information, informing in which memory buffer each
> >> plane was placed (Or should this be pre-determined by the driver?)
> >>
> >> For MMAP
> >> -----------------------
> >> userspace performs EXT_CREATE_BUF ioctl to reserve a buffer "index" range in
> >> that mode, to be used in EXT_QBUF and EXT_DQBUF
> >>
> >> Should the API allow userspace to select how many memory buffers it wants?
> >> (maybe not)
> >
> > I think it does allow that - it accepts the v4l2_ext_format struct.
>
> hmmm, I thought v4l2_ext_format would describe color planes, and not memory planes.
> Should it describe memory planes instead? Since planes are defined by the pixelformat.
> But is this information relevant to ext_{set/get/try} format?
>
Good point. I ended up assuming the current convention, where giving
an M format would imply num_memory_planes == num_color_planes and
non-M format num_memory_planes == 1. Sounds like we might want
something like a flags field and that could have bits defined to
select that. I think it would actually be useful for S_FMT as well,
because that's what REQBUFS would use.
> >
> >>
> >> userspace performs EXT_QUERY_MMAP_BUF to get the mmap offset/cookie and length
> >> for each memory buffer.
> >>
> >> On EXT_QBUF, userspace doesn't need to fill membuf information. Should the
> >> mmap offset and length be filled by the kernel and returned to userspace here
> >> as well? I'm leaning towards: no.
> >
> > Yeah, based on my comment above, I think the answer should be no.
> >
> >>
> >> If the answer is no, then here is my proposal:
> >> ----------------------------------------------
> >>
> >> /* If MMAP, drivers decide how many memory buffers to allocate */
> >> int ioctl( int fd, VIDIOC_EXT_CREATE_BUFS, struct v4l2_ext_buffer *argp )
> >>
> >> /* Returns -EINVAL if not MMAP */
> >> int ioctl( int fd, VIDIOC_EXT_MMAP_QUERYBUF, struct v4l2_ext_mmap_querybuf *argp )
> >>
> >> /* userspace fills v4l2_ext_buffer.membufs if DMA-fd or Userptr, leave it zero for MMAP
> >> * Should userspace also fill v4l2_ext_buffer.planes?
> >> */
> >> int ioctl( int fd, VIDIOC_EXT_QBUF, struct v4l2_ext_buffer *argp )
> >>
> >> /* v4l2_ext_buffer.membufs is set to zero by the driver */
> >> int ioctl( int fd, VIDIOC_EXT_DBUF, struct v4l2_ext_buffer *argp )
> >>
> >> (I omitted reserved fields below)
> >>
> >> struct v4l2_ext_create_buffers {
> >> __u32 index;
> >> __u32 count;
> >> __u32 memory;
> >> __u32 capabilities;
> >> struct v4l2_ext_pix_format format;
> >> };
> >>
> >> struct v4l2_ext_mmap_membuf {
> >> __u32 offset;
> >> __u32 length;
> >> }
> >>
> >> struct v4l2_ext_mmap_querybuf {
> >> __u32 index;
> >> struct v4l2_ext_mmap_membuf membufs[VIDEO_MAX_PLANES];
> >> }
> >>
> >> struct v4l2_ext_membuf {
> >> __u32 memory;
> >> union {
> >> __u64 userptr;
> >> __s32 dmabuf_fd;
> >> } m;
> >> // Can't we just remove the union and "memory" field, and the non-zero
> >> // is the one we should use?
> >
> > I think that would lead to an equivalent result in this case. That
> > said, I'm not sure if there would be any significant enough benefit to
> > justify moving away from the current convention. Having the memory
> > field might also make the structure a bit less error prone, e.g.
> > resilient to missing memset().
> >
> >> };
> >>
> >> struct v4l2_ext_plane {
> >> __u32 membuf_index;
> >> __u32 offset;
> >> __u32 bytesused;
> >> };
> >>
> >> struct v4l2_ext_buffer {
> >> __u32 index;
> >> __u32 type;
> >> __u32 field;
> >> __u32 sequence;
> >> __u64 flags;
> >> __u64 timestamp;
> >> struct v4l2_ext_membuf membufs[VIDEO_MAX_PLANES];
> >> struct v4l2_ext_plane planes[VIDEO_MAX_PLANES];
> >
> > Do we actually need this split into membufs and planes here? After
> > all, all we want to pass to the kernel here is in what buffer the
> > plane is in.
>
> You are right, we don't.
>
> >
> > struct v4l2_ext_plane {
> > __u32 memory;
>
> Should we design the API to allow a buffer to contain multiple memory planes
> of different types? Lets say one memplane is DMA-fd, the other is userptr.
> If the answer is yes, then struct v4l2_ext_create_buffers requires some changes.
> If not, then there is no need a "memory" field per memory plane in a buffer.
>
That's a good question. I haven't seen any practical need to do that.
Moreover, I suspect that the API might be going towards the DMA-buf
centric model, with DMA-buf heaps getting upstream acceptance, so
maybe we would be fine moving the memory field to the buffer struct
indeed.
> > union {
> > __u32 membuf_index;
> > __u64 userptr;
> > __s32 dmabuf_fd;
> > } m;
> > __u32 offset;
> > __u32 bytesused;
>
> We also need userptr_length right?
Is it actually needed? The length of the plane is determined by the
current format. I can only see as it being an extra sanity check
before accessing the process memory, but is it necessary? I think I
want to hear others's opinion on this.
[snip]
Best regards,
Tomasz
next prev parent reply other threads:[~2020-12-21 5:03 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-04 19:29 [PATCH v5 0/7] media: v4l2: Add extended fmt and buffer ioctls Helen Koike
2020-08-04 19:29 ` [PATCH v5 1/7] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more) Helen Koike
2020-08-14 7:49 ` Alexandre Courbot
2020-11-19 16:23 ` Helen Koike
2020-09-09 11:41 ` Hans Verkuil
2020-09-14 2:14 ` Helen Koike
2020-10-02 19:49 ` Tomasz Figa
2020-11-14 14:21 ` Helen Koike
2020-11-19 5:45 ` Tomasz Figa
2020-11-19 10:08 ` Helen Koike
2020-11-19 13:43 ` Helen Koike
2020-12-14 10:19 ` Tomasz Figa
2020-08-04 19:29 ` [PATCH v5 2/7] media: v4l2: Add extended buffer operations Helen Koike
2020-08-14 7:49 ` Alexandre Courbot
2020-09-09 12:27 ` Hans Verkuil
2020-11-23 15:08 ` Helen Koike
2020-11-23 15:46 ` Tomasz Figa
2020-11-23 17:40 ` Helen Koike
2020-12-03 15:11 ` Hans Verkuil
2020-12-03 19:52 ` Helen Koike
2020-12-14 10:46 ` Tomasz Figa
2020-12-15 14:36 ` Helen Koike
2020-12-16 3:13 ` Tomasz Figa
2020-12-17 13:19 ` Helen Koike
2020-12-21 3:13 ` Tomasz Figa [this message]
2020-12-23 12:04 ` Helen Koike
2021-01-08 10:00 ` Tomasz Figa
2020-12-14 10:38 ` Tomasz Figa
2020-11-20 11:14 ` Tomasz Figa
2020-11-23 20:33 ` Helen Koike
2020-12-14 10:36 ` Tomasz Figa
2020-12-14 13:23 ` Helen Koike
2020-12-15 9:03 ` Tomasz Figa
2020-08-04 19:29 ` [PATCH v5 3/7] media: videobuf2: Expose helpers to implement the _ext_fmt and _ext_buf hooks Helen Koike
2020-12-14 8:52 ` Tomasz Figa
2020-12-14 12:29 ` Helen Koike
2020-08-04 19:29 ` [PATCH v5 4/7] media: mediabus: Add helpers to convert a ext_pix format to/from a mbus_fmt Helen Koike
2020-08-14 7:49 ` Alexandre Courbot
2020-08-04 19:29 ` [PATCH v5 5/7] media: vivid: Convert the capture and output drivers to EXT_FMT/EXT_BUF Helen Koike
2020-08-04 19:29 ` [PATCH v5 6/7] media: vimc: Implement the ext_fmt and ext_buf hooks Helen Koike
2020-08-04 19:29 ` [PATCH v5 7/7] media: docs: add documentation for the Extended API Helen Koike
2020-08-14 7:49 ` Alexandre Courbot
2020-11-19 10:28 ` Helen Koike
2020-11-20 11:06 ` Tomasz Figa
2020-11-20 12:24 ` Hans Verkuil
2020-11-20 12:40 ` Tomasz Figa
2020-11-20 13:20 ` Hans Verkuil
2021-01-14 18:04 ` Helen Koike
2020-08-04 19:34 ` [PATCH v5 0/7] media: v4l2: Add extended fmt and buffer ioctls Helen Koike
2020-08-14 7:49 ` Alexandre Courbot
2020-11-27 15:06 ` Helen Koike
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAAFQd5Ds5DQ0V+c_Oapwg9CQ0ADkjtML6w6H5Ad4hwMz9Rg9YQ@mail.gmail.com \
--to=tfiga@chromium.org \
--cc=Brian.Starkey@arm.com \
--cc=boris.brezillon@collabora.com \
--cc=frkoenig@chromium.org \
--cc=hans.verkuil@cisco.com \
--cc=helen.koike@collabora.com \
--cc=hiroh@chromium.org \
--cc=hverkuil@xs4all.nl \
--cc=kernel@collabora.com \
--cc=laurent.pinchart@ideasonboard.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=mchehab@kernel.org \
--cc=mjourdan@baylibre.com \
--cc=narmstrong@baylibre.com \
--cc=nicolas@ndufresne.ca \
--cc=sakari.ailus@iki.fi \
--cc=stanimir.varbanov@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).