Hi everyone! Just to let you know. I have just pushed a Branch that includes some first steps to make the h264-stateless encoder working in Gstreamer. The work is based on the VP8 Stateless Encoder patches Benjamin Gaignard created. https://gitlab.freedesktop.org/mgrzeschik/gstreamer/-/commits/1.22/topic/h264-stateless-encoder The codec this is used with, is the rkvenc that can be found on rockchip rk3568. I will send an RFC driver for that in the next weeks after my vacation. On Tue, Jul 11, 2023 at 07:12:41PM +0200, Paul Kocialkowski wrote: >After various discussions following Andrzej's talk at EOSS, feedback from the >Media Summit (which I could not attend unfortunately) and various direct >discussions, I have compiled some thoughts and ideas about stateless encoders >support with various proposals. This is the result of a few years of interest >in the topic, after working on a PoC for the Hantro H1 using the hantro driver, >which turned out to have numerous design issues. > >I am now working on a H.264 encoder driver for Allwinner platforms (currently >focusing on the V3/V3s), which already provides some usable bitstream and will >be published soon. > >This is a very long email where I've tried to split things into distinct topics >and explain a few concepts to make sure everyone is on the same page. > ># Bitstream Headers > >Stateless encoders typically do not generate all the bitstream headers and >sometimes no header at all (e.g. Allwinner encoder does not even produce slice >headers). There's often some hardware block that makes bit-level writing to the >destination buffer easier (deals with alignment, etc). > >The values of the bitstream headers must be in line with how the compressed >data bitstream is generated and generally follow the codec specification. >Some encoders might allow configuring all the fields found in the headers, >others may only allow configuring a few or have specific constraints regarding >which values are allowed. > >As a result, we cannot expect that any given encoder is able to produce frames >for any set of headers. Reporting related constraints and limitations (beyond >profile/level) seems quite difficult and error-prone. > >So it seems that keeping header generation in-kernel only (close to where the >hardware is actually configured) is the safest approach. For the case with the rkvenc, the headers are also not created by the kernel driver. Instead we use the gst_h264_bit_writer_sps/pps functions that are part of the codecparsers module. ># Codec Features > >Codecs have many variable features that can be enabled or not and specific >configuration fields that can take various values. There is usually some >top-level indication of profile/level that restricts what can be used. > >This is a very similar situation to stateful encoding, where codec-specific >controls are used to report and set profile/level and configure these aspects. >A particularly nice thing about it is that we can reuse these existing controls >and add new ones in the future for features that are not yet covered. > >This approach feels more flexible than designing new structures with a selected >set of parameters (that could match the existing controls) for each codec. I back the Idea of generic profiles instead of explicit configuration from the usersapace point of view. The parameterization works like this: Read the sane default parameter set from the driver. Modify the parameters based on the userspace decisions. - (currently hardcoded and not based on any user input) Write the updated parameters back to the driver. ># Reference and Reconstruction Management ># Frame Types ># Rate Control ># Regions of Interest Since the first shot of the rkvenc is I-Frame only code, these other topics are currently undefined and unimplemented in the Gstreamer stack. Regards, Michael -- Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |