From: Hans Verkuil <firstname.lastname@example.org> To: Tomasz Figa <email@example.com> Cc: Linux Media Mailing List <firstname.lastname@example.org>, Nicolas Dufresne <email@example.com>, Dave Stevenson <firstname.lastname@example.org>, Boris Brezillon <email@example.com>, Paul Kocialkowski <firstname.lastname@example.org>, Stanimir Varbanov <email@example.com>, Philipp Zabel <firstname.lastname@example.org>, Ezequiel Garcia <email@example.com>, Michael Tretter <firstname.lastname@example.org>, Sylwester Nawrocki <email@example.com> Subject: Re: [RFC] Stateful codecs and requirements for compressed formats Date: Wed, 10 Jul 2019 11:14:17 +0200 [thread overview] Message-ID: <firstname.lastname@example.org> (raw) In-Reply-To: <CAAFQd5BqUS201QP4KHzmnKi5r+3P_KAa=L9CF3=zyQKypNyVuw@mail.gmail.com> On 7/3/19 10:32 AM, Tomasz Figa wrote: > Hi Hans, > > On Fri, Jun 28, 2019 at 11:34 PM Hans Verkuil <email@example.com> wrote: >> >> Hi all, >> >> I hope I Cc-ed everyone with a stake in this issue. >> >> One recurring question is how a stateful encoder fills buffers and how a stateful >> decoder consumes buffers. >> >> The most generic case is that an encoder produces a bitstream and just fills each >> CAPTURE buffer to the brim before continuing with the next buffer. >> >> I don't think there are drivers that do this, I believe that all drivers just >> output a single compressed frame. For interlaced formats I understand it is either >> one compressed field per buffer, or two compressed fields per buffer (this is >> what I heard, I don't know if this is true). >> >> In any case, I don't think this is specified anywhere. Please correct me if I am >> wrong. >> >> The latest stateful codec spec is here: >> >> https://hverkuil.home.xs4all.nl/codec-api/uapi/v4l/dev-mem2mem.html >> >> Assuming what I described above is indeed the case, then I think this should >> be documented. I don't know enough if a flag is needed somewhere to describe >> the behavior for interlaced formats, or can we leave this open and have userspace >> detect this? >> > > From Chromium perspective, we don't have any use case for encoding > interlaced contents, so we'll be okay with whatever the interested > parties decide on. :) > >> >> For decoders it is more complicated. The stateful decoder spec is written with >> the assumption that userspace can just fill each OUTPUT buffer to the brim with >> the compressed bitstream. I.e., no need to split at frame or other boundaries. >> >> See section 220.127.116.11 in the spec. >> >> But I understand that various HW decoders *do* have limitations. I would really >> like to know about those, since that needs to be exposed to userspace somehow. > > AFAIK mtk-vcodec needs H.264 SPS and PPS to be split into their own > separate buffers. I believe it also needs 1 buffer to contain exactly > 1 frame and 1 frame to be fully contained inside 1 buffer. > > Venus also needed 1 buffer to contain exactly 1 frame and 1 frame to > be fully contained inside 1 buffer. It used to have some specific > requirements regarding SPS and PPS too, but I think that was fixed in > the firmware. > >> >> Specifically, the venus decoder needs to know the resolution of the coded video >> beforehand > > I don't think that's true for venus. It does parsing and can detect > the resolution. > > However that's probably the case for coda... > >> and it expects a single frame per buffer (how does that work for >> interlaced formats?). >> >> Such requirements mean that some userspace parsing is still required, so these >> decoders are not completely stateful. >> >> Can every codec author give information about their decoder/encoder? >> >> I'll start off with my virtual codec driver: >> >> vicodec: the decoder fully parses the bitstream. The encoder produces a single >> compressed frame per buffer. This driver doesn't yet support interlaced formats, >> but when that is added it will encode one field per buffer. >> >> Let's see what the results are. > > s5p-mfc: > decoder: fully parses the bitstream, > encoder: produces a single frame per buffer (haven't tested interlaced stuff) > > mtk-vcodec: > decoder: expects separate buffers for SPS, PPS and full frames > (including some random stuff like SEIMessage), Do you mean that the SPS/PPS etc. should all be in separate buffers? I.e. you can't combine SPS and PPS in a single buffer? Regards, Hans > encoder: produces a single frame per buffer (haven't tested interlaced stuff) > > Best regards, > Tomasz >
next prev parent reply other threads:[~2019-07-10 9:14 UTC|newest] Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-06-28 14:34 Hans Verkuil 2019-06-28 15:21 ` Dave Stevenson 2019-06-28 15:48 ` Nicolas Dufresne 2019-06-29 10:02 ` Dave Stevenson 2019-06-29 12:55 ` Nicolas Dufresne 2019-06-28 16:18 ` Nicolas Dufresne 2019-06-28 18:09 ` Nicolas Dufresne 2019-07-03 8:46 ` Tomasz Figa 2019-07-03 17:43 ` Nicolas Dufresne 2019-07-10 8:43 ` Hans Verkuil 2019-07-11 1:40 ` Nicolas Dufresne 2019-07-03 8:32 ` Tomasz Figa 2019-07-03 14:46 ` Philipp Zabel 2019-07-03 17:46 ` Nicolas Dufresne 2019-07-10 9:14 ` Hans Verkuil [this message] 2019-07-11 12:49 ` Tomasz Figa 2019-07-11 1:42 ` Nicolas Dufresne 2019-07-11 12:47 ` Tomasz Figa
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --subject='Re: [RFC] Stateful codecs and requirements for compressed formats' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).