From: Ayaka <ayaka@soulik.info>
To: Nicolas Dufresne <nicolas@ndufresne.ca>
Cc: "Alexandre Courbot" <acourbot@chromium.org>,
"Paul Kocialkowski" <paul.kocialkowski@bootlin.com>,
"Randy Li" <randy.li@rock-chips.com>,
"Jernej Škrabec" <jernej.skrabec@gmail.com>,
"Linux Media Mailing List" <linux-media@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
devel@driverdev.osuosl.org, linux-arm-kernel@lists.infradead.org,
"Mauro Carvalho Chehab" <mchehab@kernel.org>,
"Maxime Ripard" <maxime.ripard@bootlin.com>,
"Hans Verkuil" <hverkuil@xs4all.nl>,
"Ezequiel Garcia" <ezequiel@collabora.com>,
"Tomasz Figa" <tfiga@chromium.org>,
"Thomas Petazzoni" <thomas.petazzoni@bootlin.com>,
linux-rockchip@lists.infradead.org
Subject: Re: [linux-sunxi] [PATCH v2 1/2] media: v4l: Add definitions for the HEVC slice format and controls
Date: Wed, 30 Jan 2019 15:03:51 +0800 [thread overview]
Message-ID: <3E2E3087-B2C7-48A9-B465-C4BDFA1250A7@soulik.info> (raw)
In-Reply-To: <2442b23d3b0a00c1bd441298a712888e016acf92.camel@ndufresne.ca>
Sent from my iPad
> On Jan 30, 2019, at 5:41 AM, Nicolas Dufresne <nicolas@ndufresne.ca> wrote:
>
>> Le mardi 29 janvier 2019 à 16:44 +0900, Alexandre Courbot a écrit :
>> On Fri, Jan 25, 2019 at 10:04 PM Paul Kocialkowski
>> <paul.kocialkowski@bootlin.com> wrote:
>>> Hi,
>>>
>>>> On Thu, 2019-01-24 at 20:23 +0800, Ayaka wrote:
>>>> Sent from my iPad
>>>>
>>>>> On Jan 24, 2019, at 6:27 PM, Paul Kocialkowski <paul.kocialkowski@bootlin.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>>> On Thu, 2019-01-10 at 21:32 +0800, ayaka wrote:
>>>>>> I forget a important thing, for the rkvdec and rk hevc decoder, it would
>>>>>> requests cabac table, scaling list, picture parameter set and reference
>>>>>> picture storing in one or various of DMA buffers. I am not talking about
>>>>>> the data been parsed, the decoder would requests a raw data.
>>>>>>
>>>>>> For the pps and rps, it is possible to reuse the slice header, just let
>>>>>> the decoder know the offset from the bitstream bufer, I would suggest to
>>>>>> add three properties(with sps) for them. But I think we need a method to
>>>>>> mark a OUTPUT side buffer for those aux data.
>>>>>
>>>>> I'm quite confused about the hardware implementation then. From what
>>>>> you're saying, it seems that it takes the raw bitstream elements rather
>>>>> than parsed elements. Is it really a stateless implementation?
>>>>>
>>>>> The stateless implementation was designed with the idea that only the
>>>>> raw slice data should be passed in bitstream form to the decoder. For
>>>>> H.264, it seems that some decoders also need the slice header in raw
>>>>> bitstream form (because they take the full slice NAL unit), see the
>>>>> discussions in this thread:
>>>>> media: docs-rst: Document m2m stateless video decoder interface
>>>>
>>>> Stateless just mean it won’t track the previous result, but I don’t
>>>> think you can define what a date the hardware would need. Even you
>>>> just build a dpb for the decoder, it is still stateless, but parsing
>>>> less or more data from the bitstream doesn’t stop a decoder become a
>>>> stateless decoder.
>>>
>>> Yes fair enough, the format in which the hardware decoder takes the
>>> bitstream parameters does not make it stateless or stateful per-se.
>>> It's just that stateless decoders should have no particular reason for
>>> parsing the bitstream on their own since the hardware can be designed
>>> with registers for each relevant bitstream element to configure the
>>> decoding pipeline. That's how GPU-based decoder implementations are
>>> implemented (VAAPI/VDPAU/NVDEC, etc).
>>>
>>> So the format we have agreed on so far for the stateless interface is
>>> to pass parsed elements via v4l2 control structures.
>>>
>>> If the hardware can only work by parsing the bitstream itself, I'm not
>>> sure what the best solution would be. Reconstructing the bitstream in
>>> the kernel is a pretty bad option, but so is parsing in the kernel or
>>> having the data both in parsed and raw forms. Do you see another
>>> possibility?
>>
>> Is reconstructing the bitstream so bad? The v4l2 controls provide a
>> generic interface to an encoded format which the driver needs to
>> convert into a sequence that the hardware can understand. Typically
>> this is done by populating hardware-specific structures. Can't we
>> consider that in this specific instance, the hardware-specific
>> structure just happens to be identical to the original bitstream
>> format?
>
> At maximum allowed bitrate for let's say HEVC (940MB/s iirc), yes, it
Lucky, most of hardware won’t be able to processing such a big buffer.
General speaking, the register is 24bits for stream length in bytes.
> would be really really bad. In GStreamer project we have discussed for
> a while (but have never done anything about) adding the ability through
> a bitmask to select which part of the stream need to be parsed, as
> parsing itself was causing some overhead. Maybe similar thing applies,
> though as per our new design, it's the fourcc that dictate the driver
> behaviour, we'd need yet another fourcc for drivers that wants the full
> bitstream (which seems odd if you have already parsed everything, I
> think this need some clarification).
>
>>
>> I agree that this is not strictly optimal for that particular
>> hardware, but such is the cost of abstractions, and in this specific
>> case I don't believe the cost would be particularly high?
next prev parent reply other threads:[~2019-01-30 7:04 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-23 13:02 [PATCH v2 0/2] HEVC/H.265 stateless support for V4L2 and Cedrus Paul Kocialkowski
2018-11-23 13:02 ` [PATCH v2 1/2] media: v4l: Add definitions for the HEVC slice format and controls Paul Kocialkowski
2018-12-05 13:18 ` Hans Verkuil
2018-12-05 20:59 ` [linux-sunxi] " Jernej Škrabec
2018-12-12 12:51 ` Paul Kocialkowski
2019-01-07 3:49 ` Randy Li
2019-01-07 9:57 ` Paul Kocialkowski
2019-01-08 1:16 ` Ayaka
2019-01-08 8:38 ` Paul Kocialkowski
2019-01-08 10:00 ` Ayaka
2019-01-10 13:32 ` ayaka
2019-01-24 10:27 ` Paul Kocialkowski
2019-01-24 12:23 ` Ayaka
2019-01-25 13:04 ` Paul Kocialkowski
2019-01-29 7:44 ` Alexandre Courbot
2019-01-29 8:09 ` Maxime Ripard
2019-01-29 9:39 ` Tomasz Figa
2019-01-29 21:41 ` Nicolas Dufresne
2019-01-30 2:28 ` Alexandre Courbot
2019-01-30 3:35 ` Tomasz Figa
2019-01-30 6:27 ` Ayaka
2019-01-30 7:17 ` Tomasz Figa
2019-01-30 9:54 ` Ayaka
2019-01-30 7:57 ` Maxime Ripard
2019-01-30 7:03 ` Ayaka [this message]
2019-01-24 10:36 ` Paul Kocialkowski
2019-01-24 12:19 ` Ayaka
2018-11-23 13:02 ` [PATCH v2 2/2] media: cedrus: Add HEVC/H.265 decoding support Paul Kocialkowski
2018-11-27 8:21 ` Maxime Ripard
2019-01-24 13:10 ` Paul Kocialkowski
2019-01-25 10:10 ` Maxime Ripard
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3E2E3087-B2C7-48A9-B465-C4BDFA1250A7@soulik.info \
--to=ayaka@soulik.info \
--cc=acourbot@chromium.org \
--cc=devel@driverdev.osuosl.org \
--cc=ezequiel@collabora.com \
--cc=hverkuil@xs4all.nl \
--cc=jernej.skrabec@gmail.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-rockchip@lists.infradead.org \
--cc=maxime.ripard@bootlin.com \
--cc=mchehab@kernel.org \
--cc=nicolas@ndufresne.ca \
--cc=paul.kocialkowski@bootlin.com \
--cc=randy.li@rock-chips.com \
--cc=tfiga@chromium.org \
--cc=thomas.petazzoni@bootlin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).