linux-media.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Hans Verkuil <hverkuil@xs4all.nl>
To: Linux Media Mailing List <linux-media@vger.kernel.org>,
	Nicolas Dufresne <nicolas@ndufresne.ca>,
	Dave Stevenson <dave.stevenson@raspberrypi.org>,
	Boris Brezillon <boris.brezillon@collabora.com>,
	Paul Kocialkowski <paul.kocialkowski@bootlin.com>,
	Stanimir Varbanov <stanimir.varbanov@linaro.org>,
	Philipp Zabel <p.zabel@pengutronix.de>,
	Ezequiel Garcia <ezequiel@collabora.com>,
	Michael Tretter <m.tretter@pengutronix.de>,
	Tomasz Figa <tfiga@chromium.org>,
	Sylwester Nawrocki <snawrocki@kernel.org>
Subject: [RFC] Stateful codecs and requirements for compressed formats
Date: Fri, 28 Jun 2019 16:34:20 +0200	[thread overview]
Message-ID: <530f28e9-f686-6222-c6cc-9a5207b151f7@xs4all.nl> (raw)

Hi all,

I hope I Cc-ed everyone with a stake in this issue.

One recurring question is how a stateful encoder fills buffers and how a stateful
decoder consumes buffers.

The most generic case is that an encoder produces a bitstream and just fills each
CAPTURE buffer to the brim before continuing with the next buffer.

I don't think there are drivers that do this, I believe that all drivers just
output a single compressed frame. For interlaced formats I understand it is either
one compressed field per buffer, or two compressed fields per buffer (this is
what I heard, I don't know if this is true).

In any case, I don't think this is specified anywhere. Please correct me if I am
wrong.

The latest stateful codec spec is here:

https://hverkuil.home.xs4all.nl/codec-api/uapi/v4l/dev-mem2mem.html

Assuming what I described above is indeed the case, then I think this should
be documented. I don't know enough if a flag is needed somewhere to describe
the behavior for interlaced formats, or can we leave this open and have userspace
detect this?


For decoders it is more complicated. The stateful decoder spec is written with
the assumption that userspace can just fill each OUTPUT buffer to the brim with
the compressed bitstream. I.e., no need to split at frame or other boundaries.

See section 4.5.1.7 in the spec.

But I understand that various HW decoders *do* have limitations. I would really
like to know about those, since that needs to be exposed to userspace somehow.

Specifically, the venus decoder needs to know the resolution of the coded video
beforehand and it expects a single frame per buffer (how does that work for
interlaced formats?).

Such requirements mean that some userspace parsing is still required, so these
decoders are not completely stateful.

Can every codec author give information about their decoder/encoder?

I'll start off with my virtual codec driver:

vicodec: the decoder fully parses the bitstream. The encoder produces a single
compressed frame per buffer. This driver doesn't yet support interlaced formats,
but when that is added it will encode one field per buffer.

Let's see what the results are.

Regards,

	Hans

             reply	other threads:[~2019-06-28 14:34 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-06-28 14:34 Hans Verkuil [this message]
2019-06-28 15:21 ` Dave Stevenson
2019-06-28 15:48   ` Nicolas Dufresne
2019-06-29 10:02     ` Dave Stevenson
2019-06-29 12:55       ` Nicolas Dufresne
2019-06-28 16:18 ` Nicolas Dufresne
2019-06-28 18:09 ` Nicolas Dufresne
2019-07-03  8:46   ` Tomasz Figa
2019-07-03 17:43     ` Nicolas Dufresne
2019-07-10  8:43   ` Hans Verkuil
2019-07-11  1:40     ` Nicolas Dufresne
2019-07-03  8:32 ` Tomasz Figa
2019-07-03 14:46   ` Philipp Zabel
2019-07-03 17:46   ` Nicolas Dufresne
2019-07-10  9:14   ` Hans Verkuil
2019-07-11 12:49     ` Tomasz Figa
2019-07-11  1:42   ` Nicolas Dufresne
2019-07-11 12:47     ` Tomasz Figa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=530f28e9-f686-6222-c6cc-9a5207b151f7@xs4all.nl \
    --to=hverkuil@xs4all.nl \
    --cc=boris.brezillon@collabora.com \
    --cc=dave.stevenson@raspberrypi.org \
    --cc=ezequiel@collabora.com \
    --cc=linux-media@vger.kernel.org \
    --cc=m.tretter@pengutronix.de \
    --cc=nicolas@ndufresne.ca \
    --cc=p.zabel@pengutronix.de \
    --cc=paul.kocialkowski@bootlin.com \
    --cc=snawrocki@kernel.org \
    --cc=stanimir.varbanov@linaro.org \
    --cc=tfiga@chromium.org \
    --subject='Re: [RFC] Stateful codecs and requirements for compressed formats' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).