From: Ian Arkver <ian.arkver.dev@gmail.com>
To: Hans Verkuil <hverkuil@xs4all.nl>, Tomasz Figa <tfiga@chromium.org>
Cc: "Linux Media Mailing List" <linux-media@vger.kernel.org>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
"Stanimir Varbanov" <stanimir.varbanov@linaro.org>,
"Mauro Carvalho Chehab" <mchehab@kernel.org>,
"Pawel Osciak" <posciak@chromium.org>,
"Alexandre Courbot" <acourbot@chromium.org>,
kamil@wypas.org, a.hajda@samsung.com,
"Kyungmin Park" <kyungmin.park@samsung.com>,
jtp.park@samsung.com, "Philipp Zabel" <p.zabel@pengutronix.de>,
"Tiffany Lin (林慧珊)" <tiffany.lin@mediatek.com>,
"Andrew-CT Chen (陳智迪)" <andrew-ct.chen@mediatek.com>,
todor.tomov@linaro.org, nicolas@ndufresne.ca,
"Paul Kocialkowski" <paul.kocialkowski@bootlin.com>,
"Laurent Pinchart" <laurent.pinchart@ideasonboard.com>,
dave.stevenson@raspberrypi.org,
"Ezequiel Garcia" <ezequiel@collabora.com>
Subject: Re: [PATCH 1/2] media: docs-rst: Document memory-to-memory video decoder interface
Date: Wed, 8 Aug 2018 07:54:54 +0100 [thread overview]
Message-ID: <2348260b-aa1f-a1e1-cf3e-a155c5aca20d@gmail.com> (raw)
In-Reply-To: <fa0b9a5d-a2c6-85af-694c-0bd7bedb4bb3@xs4all.nl>
Hi Hans,
On 08/08/18 07:43, Hans Verkuil wrote:
> On 08/08/2018 05:11 AM, Tomasz Figa wrote:
>> On Tue, Aug 7, 2018 at 4:13 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>>
>>> On 07/26/2018 12:20 PM, Tomasz Figa wrote:
>>>> Hi Hans,
>>>>
>>>> On Wed, Jul 25, 2018 at 8:59 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>>>>> +
>>>>>> +14. Call :c:func:`VIDIOC_STREAMON` to initiate decoding frames.
>>>>>> +
>>>>>> +Decoding
>>>>>> +========
>>>>>> +
>>>>>> +This state is reached after a successful initialization sequence. In this
>>>>>> +state, client queues and dequeues buffers to both queues via
>>>>>> +:c:func:`VIDIOC_QBUF` and :c:func:`VIDIOC_DQBUF`, following standard
>>>>>> +semantics.
>>>>>> +
>>>>>> +Both queues operate independently, following standard behavior of V4L2
>>>>>> +buffer queues and memory-to-memory devices. In addition, the order of
>>>>>> +decoded frames dequeued from ``CAPTURE`` queue may differ from the order of
>>>>>> +queuing coded frames to ``OUTPUT`` queue, due to properties of selected
>>>>>> +coded format, e.g. frame reordering. The client must not assume any direct
>>>>>> +relationship between ``CAPTURE`` and ``OUTPUT`` buffers, other than
>>>>>> +reported by :c:type:`v4l2_buffer` ``timestamp`` field.
>>>>>
>>>>> Is there a relationship between capture and output buffers w.r.t. the timestamp
>>>>> field? I am not aware that there is one.
>>>>
>>>> I believe the decoder was expected to copy the timestamp of matching
>>>> OUTPUT buffer to respective CAPTURE buffer. Both s5p-mfc and coda seem
>>>> to be implementing it this way. I guess it might be a good idea to
>>>> specify this more explicitly.
>>>
>>> What about an output buffer producing multiple capture buffers? Or the case
>>> where the encoded bitstream of a frame starts at one output buffer and ends
>>> at another? What happens if you have B frames and the order of the capture
>>> buffers is different from the output buffers?
>>>
>>> In other words, for codecs there is no clear 1-to-1 relationship between an
>>> output buffer and a capture buffer. And we never defined what the 'copy timestamp'
>>> behavior should be in that case or if it even makes sense.
>>
>> You're perfectly right. There is no 1:1 relationship, but it doesn't
>> prevent copying timestamps. It just makes it possible for multiple
>> CAPTURE buffers to have the same timestamp or some OUTPUT timestamps
>> not to be found in any CAPTURE buffer.
>
> We need to document the behavior. Basically there are three different
> corner cases that need documenting:
>
> 1) one OUTPUT buffer generates multiple CAPTURE buffers
> 2) multiple OUTPUT buffers generate one CAPTURE buffer
> 3) the decoding order differs from the presentation order (i.e. the
> CAPTURE buffers are out-of-order compared to the OUTPUT buffers).
>
> For 1) I assume that we just copy the same OUTPUT timestamp to multiple
> CAPTURE buffers.
I'm not sure how this interface would handle something like a temporal
scalability layer, but conceivably this assumption might be invalid in
that case.
Regards,
Ian.
>
> For 2) we need to specify if the CAPTURE timestamp is copied from the first
> or last OUTPUT buffer used in creating the capture buffer. Using the last
> OUTPUT buffer makes more sense to me.
>
> And 3) implies that timestamps can be out-of-order. This needs to be
> very carefully documented since it is very unexpected.
>
> This should probably be a separate patch, adding text to the v4l2_buffer
> documentation (esp. the V4L2_BUF_FLAG_TIMESTAMP_COPY documentation).
>
> Regards,
>
> Hans
>
next prev parent reply other threads:[~2018-08-08 6:55 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-24 14:06 [PATCH 0/2] Document memory-to-memory video codec interfaces Tomasz Figa
2018-07-24 14:06 ` [PATCH 1/2] media: docs-rst: Document memory-to-memory video decoder interface Tomasz Figa
2018-07-25 11:58 ` Hans Verkuil
2018-07-26 10:20 ` Tomasz Figa
2018-07-26 10:36 ` Philipp Zabel
2018-08-07 6:55 ` Tomasz Figa
2018-07-26 10:57 ` Hans Verkuil
2018-08-07 7:05 ` Tomasz Figa
2018-08-07 7:37 ` Hans Verkuil
2018-08-08 2:55 ` Tomasz Figa
2018-08-21 11:29 ` Stanimir Varbanov
2018-08-27 4:09 ` Tomasz Figa
2018-10-15 10:13 ` Tomasz Figa
2018-10-16 1:09 ` Nicolas Dufresne
2018-08-07 7:13 ` Hans Verkuil
2018-08-07 19:11 ` Maxime Jourdan
2018-08-08 3:07 ` Tomasz Figa
2018-08-08 7:19 ` Maxime Jourdan
2018-08-08 3:11 ` Tomasz Figa
2018-08-08 6:43 ` Hans Verkuil
2018-08-08 6:54 ` Ian Arkver [this message]
2018-09-19 10:17 ` Tomasz Figa
2018-10-08 12:22 ` Hans Verkuil
2018-10-09 4:23 ` Tomasz Figa
2018-10-09 6:39 ` Hans Verkuil
2018-07-30 12:52 ` Hans Verkuil
2018-08-07 7:08 ` Tomasz Figa
2018-08-08 2:46 ` Tomasz Figa
2018-08-20 13:04 ` Philipp Zabel
2018-08-20 13:12 ` Tomasz Figa
2018-08-20 14:13 ` Philipp Zabel
2018-08-20 14:27 ` Tomasz Figa
2018-08-20 15:33 ` Philipp Zabel
2018-08-27 4:03 ` Tomasz Figa
2018-08-31 8:26 ` Alexandre Courbot
2018-09-05 5:45 ` Tomasz Figa
2018-10-17 13:34 ` Laurent Pinchart
2018-10-18 10:03 ` Tomasz Figa
2018-10-18 11:22 ` Laurent Pinchart
2018-10-20 8:52 ` Tomasz Figa
2018-10-21 9:23 ` Laurent Pinchart
2018-10-22 6:19 ` Tomasz Figa
2018-10-20 10:24 ` Tomasz Figa
2018-10-21 9:26 ` Laurent Pinchart
2018-10-20 15:39 ` Tomasz Figa
2018-07-24 14:06 ` [PATCH 2/2] media: docs-rst: Document memory-to-memory video encoder interface Tomasz Figa
2018-07-25 13:41 ` Philipp Zabel
2018-08-07 6:07 ` Tomasz Figa
2018-07-25 13:57 ` Hans Verkuil
2018-08-07 6:54 ` Tomasz Figa
2018-08-07 7:25 ` Hans Verkuil
2018-10-16 7:36 ` Tomasz Figa
2018-10-16 13:49 ` Hans Verkuil
2018-10-22 4:50 ` Tomasz Figa
2018-09-07 20:17 ` Ezequiel Garcia
2018-09-10 3:34 ` Tomasz Figa
2018-10-17 15:19 ` Laurent Pinchart
2018-10-22 6:12 ` Tomasz Figa
2018-07-25 13:28 ` [PATCH 0/2] Document memory-to-memory video codec interfaces Philipp Zabel
2018-07-25 13:35 ` Tomasz Figa
2018-09-10 9:13 ` Hans Verkuil
2018-09-11 3:52 ` Tomasz Figa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2348260b-aa1f-a1e1-cf3e-a155c5aca20d@gmail.com \
--to=ian.arkver.dev@gmail.com \
--cc=a.hajda@samsung.com \
--cc=acourbot@chromium.org \
--cc=andrew-ct.chen@mediatek.com \
--cc=dave.stevenson@raspberrypi.org \
--cc=ezequiel@collabora.com \
--cc=hverkuil@xs4all.nl \
--cc=jtp.park@samsung.com \
--cc=kamil@wypas.org \
--cc=kyungmin.park@samsung.com \
--cc=laurent.pinchart@ideasonboard.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=mchehab@kernel.org \
--cc=nicolas@ndufresne.ca \
--cc=p.zabel@pengutronix.de \
--cc=paul.kocialkowski@bootlin.com \
--cc=posciak@chromium.org \
--cc=stanimir.varbanov@linaro.org \
--cc=tfiga@chromium.org \
--cc=tiffany.lin@mediatek.com \
--cc=todor.tomov@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).