From: Tomasz Figa <tfiga@chromium.org>
To: Hans Verkuil <hverkuil@xs4all.nl>
Cc: "Linux Media Mailing List" <linux-media@vger.kernel.org>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
"Stanimir Varbanov" <stanimir.varbanov@linaro.org>,
"Mauro Carvalho Chehab" <mchehab@kernel.org>,
"Pawel Osciak" <posciak@chromium.org>,
"Alexandre Courbot" <acourbot@chromium.org>,
kamil@wypas.org, a.hajda@samsung.com,
"Kyungmin Park" <kyungmin.park@samsung.com>,
jtp.park@samsung.com, "Philipp Zabel" <p.zabel@pengutronix.de>,
"Tiffany Lin (林慧珊)" <tiffany.lin@mediatek.com>,
"Andrew-CT Chen (陳智迪)" <andrew-ct.chen@mediatek.com>,
todor.tomov@linaro.org, nicolas@ndufresne.ca,
"Paul Kocialkowski" <paul.kocialkowski@bootlin.com>,
"Laurent Pinchart" <laurent.pinchart@ideasonboard.com>,
dave.stevenson@raspberrypi.org,
"Ezequiel Garcia" <ezequiel@collabora.com>,
"Maxime Jourdan" <maxi.jourdan@wanadoo.fr>
Subject: Re: [PATCH 1/2] media: docs-rst: Document memory-to-memory video decoder interface
Date: Wed, 19 Sep 2018 19:17:44 +0900 [thread overview]
Message-ID: <CAAFQd5Ba+8_wpCr2D2OUSRt-PbRUPk4MV1OxMzEetntL169fBA@mail.gmail.com> (raw)
In-Reply-To: <CAAFQd5BgGEBmd8gNGc-qqtUtLo=Mh8U+TVTWRsKYMv1LmeBQMA@mail.gmail.com>
Hi Hans,
On Thu, Jul 26, 2018 at 7:20 PM Tomasz Figa <tfiga@chromium.org> wrote:
>
> Hi Hans,
>
> On Wed, Jul 25, 2018 at 8:59 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> >
> > Hi Tomasz,
> >
> > Many, many thanks for working on this! It's a great document and when done
> > it will be very useful indeed.
> >
> > Review comments follow...
>
> Thanks for review!
>
> >
> > On 24/07/18 16:06, Tomasz Figa wrote:
[snip]
> > > +13. Allocate destination (raw format) buffers via :c:func:`VIDIOC_REQBUFS`
> > > + on the ``CAPTURE`` queue.
> > > +
> > > + * **Required fields:**
> > > +
> > > + ``count``
> > > + requested number of buffers to allocate; greater than zero
> > > +
> > > + ``type``
> > > + a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``
> > > +
> > > + ``memory``
> > > + follows standard semantics
> > > +
> > > + * **Return fields:**
> > > +
> > > + ``count``
> > > + adjusted to allocated number of buffers
> > > +
> > > + * The driver must adjust count to minimum of required number of
> > > + destination buffers for given format and stream configuration and the
> > > + count passed. The client must check this value after the ioctl
> > > + returns to get the number of buffers allocated.
> > > +
> > > + .. note::
> > > +
> > > + To allocate more than minimum number of buffers (for pipeline
> > > + depth), use G_CTRL(``V4L2_CID_MIN_BUFFERS_FOR_CAPTURE``) to
> > > + get minimum number of buffers required, and pass the obtained value
> > > + plus the number of additional buffers needed in count to
> > > + :c:func:`VIDIOC_REQBUFS`.
> >
> >
> > I think we should mention here the option of using VIDIOC_CREATE_BUFS in order
> > to allocate buffers larger than the current CAPTURE format in order to accommodate
> > future resolution changes.
>
> Ack.
>
I'm about to add a paragraph to describe this, but there is one detail
to iron out.
The VIDIOC_CREATE_BUFS ioctl accepts a v4l2_format struct. Userspace
needs to fill in this struct and the specs says that
"Usually this will be done using the VIDIOC_TRY_FMT or VIDIOC_G_FMT
ioctls to ensure that the requested format is supported by the
driver."
However, in case of a decoder, those calls would fixup the format to
match the currently parsed stream, which would likely resolve to the
current coded resolution (~hardware alignment). How do we get a format
for the desired maximum resolution?
[snip].
> > > +
> > > + * The driver is also allowed to and may not return all decoded frames
[snip]
> > > + queued but not decode before the seek sequence was initiated. For
> >
> > Very confusing sentence. I think you mean this:
> >
> > The driver may not return all decoded frames that where ready for
> > dequeueing from before the seek sequence was initiated.
> >
> > Is this really true? Once decoded frames are marked as buffer_done by the
> > driver there is no reason for them to be removed. Or you mean something else
> > here, e.g. the frames are decoded, but the buffers not yet given back to vb2.
> >
>
> Exactly "the frames are decoded, but the buffers not yet given back to
> vb2", for example, if reordering takes place. However, if one stops
> streaming before dequeuing all buffers, they are implicitly returned
> (reset to the state after REQBUFS) and can't be dequeued anymore, so
> the frames are lost, even if the driver returned them. I guess the
> sentence was really unfortunate indeed.
>
Actually, that's not the only case.
The documentation is written from userspace point of view. Queuing an
OUTPUT buffer is not equivalent to having it decoded (and a CAPTURE
buffer given back to vb2). If the userspace queues a buffer and then
stops streaming, the buffer might have been still waiting in the
queue, for decoding of previous buffers to finish.
So basically by "queued frames" I meant "OUTPUT buffers queued by
userspace and not sent to the hardware yet" and by "decoded frames" I
meant "CAPTURE buffers containing matching frames given back to vb2".
How about rewording like this:
* The ``VIDIOC_STREAMOFF`` operation discards any remaining queued
``OUTPUT`` buffers, which means that not all of the ``OUTPUT`` buffers
queued before the seek may have matching ``CAPTURE`` buffers produced.
For example, [...]
> > > + example, given an ``OUTPUT`` queue sequence: QBUF(A), QBUF(B),
> > > + STREAMOFF(OUT), STREAMON(OUT), QBUF(G), QBUF(H), any of the
> > > + following results on the ``CAPTURE`` queue is allowed: {A’, B’, G’,
> > > + H’}, {A’, G’, H’}, {G’, H’}.
> > > +
> > > + .. note::
> > > +
> > > + To achieve instantaneous seek, the client may restart streaming on
> > > + ``CAPTURE`` queue to discard decoded, but not yet dequeued buffers.
Best regards,
Tomasz
next prev parent reply other threads:[~2018-09-19 10:23 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-24 14:06 [PATCH 0/2] Document memory-to-memory video codec interfaces Tomasz Figa
2018-07-24 14:06 ` [PATCH 1/2] media: docs-rst: Document memory-to-memory video decoder interface Tomasz Figa
2018-07-25 11:58 ` Hans Verkuil
2018-07-26 10:20 ` Tomasz Figa
2018-07-26 10:36 ` Philipp Zabel
2018-08-07 6:55 ` Tomasz Figa
2018-07-26 10:57 ` Hans Verkuil
2018-08-07 7:05 ` Tomasz Figa
2018-08-07 7:37 ` Hans Verkuil
2018-08-08 2:55 ` Tomasz Figa
2018-08-21 11:29 ` Stanimir Varbanov
2018-08-27 4:09 ` Tomasz Figa
2018-10-15 10:13 ` Tomasz Figa
2018-10-16 1:09 ` Nicolas Dufresne
2018-08-07 7:13 ` Hans Verkuil
2018-08-07 19:11 ` Maxime Jourdan
2018-08-08 3:07 ` Tomasz Figa
2018-08-08 7:19 ` Maxime Jourdan
2018-08-08 3:11 ` Tomasz Figa
2018-08-08 6:43 ` Hans Verkuil
2018-08-08 6:54 ` Ian Arkver
2018-09-19 10:17 ` Tomasz Figa [this message]
2018-10-08 12:22 ` Hans Verkuil
2018-10-09 4:23 ` Tomasz Figa
2018-10-09 6:39 ` Hans Verkuil
2018-07-30 12:52 ` Hans Verkuil
2018-08-07 7:08 ` Tomasz Figa
2018-08-08 2:46 ` Tomasz Figa
2018-08-20 13:04 ` Philipp Zabel
2018-08-20 13:12 ` Tomasz Figa
2018-08-20 14:13 ` Philipp Zabel
2018-08-20 14:27 ` Tomasz Figa
2018-08-20 15:33 ` Philipp Zabel
2018-08-27 4:03 ` Tomasz Figa
2018-08-31 8:26 ` Alexandre Courbot
2018-09-05 5:45 ` Tomasz Figa
2018-10-17 13:34 ` Laurent Pinchart
2018-10-18 10:03 ` Tomasz Figa
2018-10-18 11:22 ` Laurent Pinchart
2018-10-20 8:52 ` Tomasz Figa
2018-10-21 9:23 ` Laurent Pinchart
2018-10-22 6:19 ` Tomasz Figa
2018-10-20 10:24 ` Tomasz Figa
2018-10-21 9:26 ` Laurent Pinchart
2018-10-20 15:39 ` Tomasz Figa
2018-07-24 14:06 ` [PATCH 2/2] media: docs-rst: Document memory-to-memory video encoder interface Tomasz Figa
2018-07-25 13:41 ` Philipp Zabel
2018-08-07 6:07 ` Tomasz Figa
2018-07-25 13:57 ` Hans Verkuil
2018-08-07 6:54 ` Tomasz Figa
2018-08-07 7:25 ` Hans Verkuil
2018-10-16 7:36 ` Tomasz Figa
2018-10-16 13:49 ` Hans Verkuil
2018-10-22 4:50 ` Tomasz Figa
2018-09-07 20:17 ` Ezequiel Garcia
2018-09-10 3:34 ` Tomasz Figa
2018-10-17 15:19 ` Laurent Pinchart
2018-10-22 6:12 ` Tomasz Figa
2018-07-25 13:28 ` [PATCH 0/2] Document memory-to-memory video codec interfaces Philipp Zabel
2018-07-25 13:35 ` Tomasz Figa
2018-09-10 9:13 ` Hans Verkuil
2018-09-11 3:52 ` Tomasz Figa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAAFQd5Ba+8_wpCr2D2OUSRt-PbRUPk4MV1OxMzEetntL169fBA@mail.gmail.com \
--to=tfiga@chromium.org \
--cc=a.hajda@samsung.com \
--cc=acourbot@chromium.org \
--cc=andrew-ct.chen@mediatek.com \
--cc=dave.stevenson@raspberrypi.org \
--cc=ezequiel@collabora.com \
--cc=hverkuil@xs4all.nl \
--cc=jtp.park@samsung.com \
--cc=kamil@wypas.org \
--cc=kyungmin.park@samsung.com \
--cc=laurent.pinchart@ideasonboard.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=maxi.jourdan@wanadoo.fr \
--cc=mchehab@kernel.org \
--cc=nicolas@ndufresne.ca \
--cc=p.zabel@pengutronix.de \
--cc=paul.kocialkowski@bootlin.com \
--cc=posciak@chromium.org \
--cc=stanimir.varbanov@linaro.org \
--cc=tiffany.lin@mediatek.com \
--cc=todor.tomov@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).