Re: Stateless Encoding uAPI Discussion and Proposal

From: Nicolas Dufresne <nicolas.dufresne@collabora.com>
To: Michael Grzeschik <mgr@pengutronix.de>,
	Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Cc: linux-kernel@vger.kernel.org, linux-media@vger.kernel.org,
	"Hans Verkuil" <hverkuil@xs4all.nl>,
	"Sakari Ailus" <sakari.ailus@iki.fi>,
	"Andrzej Pietrasiewicz" <andrzej.p@collabora.com>,
	"Michael Tretter" <m.tretter@pengutronix.de>,
	"Jernej Škrabec" <jernej.skrabec@gmail.com>,
	"Chen-Yu Tsai" <wens@csie.org>,
	"Samuel Holland" <samuel@sholland.org>,
	"Thomas Petazzoni" <thomas.petazzoni@bootlin.com>
Subject: Re: Stateless Encoding uAPI Discussion and Proposal
Date: Mon, 24 Jul 2023 10:03:12 -0400	[thread overview]
Message-ID: <c6a222be5eee962581cf5dcb9a1473cf45ff303c.camel@collabora.com> (raw)
In-Reply-To: <20230721181951.GL12001@pengutronix.de>

Le vendredi 21 juillet 2023 à 20:19 +0200, Michael Grzeschik a écrit :
> > As a result, we cannot expect that any given encoder is able to produce frames
> > for any set of headers. Reporting related constraints and limitations (beyond
> > profile/level) seems quite difficult and error-prone.
> > 
> > So it seems that keeping header generation in-kernel only (close to where the
> > hardware is actually configured) is the safest approach.
> 
> For the case with the rkvenc, the headers are also not created by the
> kernel driver. Instead we use the gst_h264_bit_writer_sps/pps functions
> that are part of the codecparsers module.

One level of granularity we can add is split headers (like SPS/PPS) and
slice/frame headers. It remains that in some cases, like HEVC, when the slice
header is byte aligned, it can be nice to be able to handle it at application
side in order to avoid limiting SVC support (and other creative features) by our
API/abstraction limitations. I think a certain level of "per CODEC" reasoning is
also needed. Just like, I would not want to have to ask the kernel to generate
user data SEI and other in-band data.

regards,
Nicolas