Re: [RFC PATCH v6 02/11] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)

From: Nicolas Dufresne <nicolas@ndufresne.ca>
To: Tomasz Figa <tfiga@chromium.org>, Hsia-Jun Li <Randy.Li@synaptics.com>
Cc: mchehab@kernel.org, hans.verkuil@cisco.com,
	laurent.pinchart@ideasonboard.com, sakari.ailus@iki.fi,
	boris.brezillon@collabora.com, hiroh@chromium.org,
	Brian.Starkey@arm.com, kernel@collabora.com,
	narmstrong@baylibre.com, linux-kernel@vger.kernel.org,
	frkoenig@chromium.org, stanimir.varbanov@linaro.org,
	Hans Verkuil <hverkuil@xs4all.nl>,
	linux-media@vger.kernel.org
Subject: Re: [RFC PATCH v6 02/11] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
Date: Tue, 15 Nov 2022 11:03:26 -0500	[thread overview]
Message-ID: <df2a19b7fd0238808f00973b2cb5f4f478c38ef9.camel@ndufresne.ca> (raw)
In-Reply-To: <CAAFQd5DD4QgDs-Dff_SB2tNT3d9Hs8HG0rkQFPV+6vgvKqg+qA@mail.gmail.com>

Le vendredi 11 novembre 2022 à 17:52 +0900, Tomasz Figa a écrit :
> > > Memory allocation doesn't sound to me like it is related to formats or
> > > modifiers in any way. I agree with Nicolas that if we want to allow
> > > the userspace to specify if the memory should be contiguous or not,
> > > that should be a separate flag and actually I'd probably see it in
> > > REQBUF_EXT and CREATE_BUFS_EXT, rather than as a part of the format.
> > > 
> > I agree with that. But here is a problem, if there was a display
> > device(DRM) that only supports contiguous planes in a frame buffer.
> > How do we be aware of that?
> 
> That's why I think the MMAP mode is not scalable and shouldn't be
> expanded anymore. Both V4L2 and DRM devices should describe their
> constraints to the userspace and then the userspace should allocate
> accordingly from the right DMA-buf heap. (Or as Android and ChromeOS
> do, just have a central allocator library that understands the
> constraints, so there is no need to query the drivers.)

Just pointing out, they "hardcode" the constraints, they don't truly understand
them. Also, the Android/ChromeOS implementation is not a great playground, as it
completely ignores the constrait already exposed by V4L2 API (the sizeimage and
bytesperline found in the FMT structure). You would not have to implement Hantro
and Rockchip motion vector size calculation there if you'd simply use the
sizeimage.

Nicolas