Re: [RFC 5/7] media: uapi: Add generic serial metadata mbus formats

[Sakari Ailus <sakari.ailus@linux.intel.com>]

Hi Hans,

On Thu, Jun 08, 2023 at 10:35:38AM +0200, Hans Verkuil wrote:
> On 05/05/2023 23:52, Sakari Ailus wrote:
> > Add generic serial metadata mbus formats. These formats describe data
> > width and packing but not the content itself. The reason for specifying
> > such formats is that the formats as such are fairly device specific but
> > they are still handled by CSI-2 receiver drivers that should not be aware
> > of device specific formats. What makes generic metadata formats possible
> > is that these formats are parsed by software only, after capturing the
> > data to system memory.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  .../media/v4l/subdev-formats.rst              | 257 ++++++++++++++++++
> >  include/uapi/linux/media-bus-format.h         |   9 +
> >  2 files changed, 266 insertions(+)
> > 
> > diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > index a3a35eeed708..1492fff58426 100644
> > --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > @@ -8234,3 +8234,260 @@ The following table lists the existing metadata formats.
> >  	both sides of the link and the bus format is a fixed
> >  	metadata format that is not configurable from userspace.
> >  	Width and height will be set to 0 for this format.
> > +
> > +Generic Serial Metadata Formats
> > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +
> > +Generic serial metadata formats are used on serial busses where the actual data
> > +content is more or less device specific but the data is transmitted and received
> > +by multiple devices that do not process the data in any way, simply writing
> > +it to system memory for processing in software at the end of the pipeline.
> > +
> > +The more specific variant describing the actual data is used on the internal
> > +source pad of the originating sub-device.
> 
> As Laurent said as well, the mention of 'internal source pad' is confusing.
> 
> Aren't there two possibilities here? Either the entity has just a source pad
> that delivers the metadata, and that source pad has the precise metadata
> format. Or the entity supports routing and then there is a sink pad connected
> to an internal source that is routed to a source pad. In that case that sink
> pad has the precise metadata format.
> 
> Apologies if I am wrong, it's hard to keep track of all the possibilities,
> especially if you do not use this regularly.

It's required the generic metadata format is present only on internal pads,
otherwise link validation will fail --- the next device won't be aware of
the specific mbus code.

> 
> > +
> > +"b" in an array cell signifies a byte of data, followed by the number of byte
> > +and finally the bit number in subscript. "p" indicates a padding bit.
> > +
> > +.. _media-bus-format-generic-meta:
> > +
> > +.. cssclass: longtable
> > +
> > +.. flat-table:: Generic Serial Metadata Formats
> > +    :header-rows:  2
> > +    :stub-columns: 0
> > +
> > +    * - Identifier
> > +      - Code
> > +      -
> > +      - :cspan:`23` Data organization
> > +    * -
> > +      -
> > +      - Bit
> > +      - 23
> > +      - 22
> > +      - 21
> > +      - 20
> > +      - 19
> > +      - 18
> > +      - 17
> > +      - 16
> > +      - 15
> > +      - 14
> > +      - 13
> > +      - 12
> > +      - 11
> > +      - 10
> > +      - 9
> > +      - 8
> > +      - 7
> > +      - 6
> > +      - 5
> > +      - 4
> > +      - 3
> > +      - 2
> > +      - 1
> > +      - 0
> > +    * .. _MEDIA-BUS-FMT-META-1X8-8:
> > +
> > +      - MEDIA_BUS_FMT_META_1X8_8
> > +      - 0x8001
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      - b0\ :sub:`7`
> > +      - b0\ :sub:`6`
> > +      - b0\ :sub:`5`
> > +      - b0\ :sub:`4`
> > +      - b0\ :sub:`3`
> > +      - b0\ :sub:`2`
> > +      - b0\ :sub:`1`
> > +      - b0\ :sub:`0`
> > +    * .. _MEDIA-BUS-FMT-META-1X8-10:
> > +
> > +      - MEDIA_BUS_FMT_META_1X8_10
> > +      - 0x8002
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      - b0\ :sub:`7`
> > +      - b0\ :sub:`6`
> > +      - b0\ :sub:`5`
> > +      - b0\ :sub:`4`
> > +      - b0\ :sub:`3`
> > +      - b0\ :sub:`2`
> > +      - b0\ :sub:`1`
> > +      - b0\ :sub:`0`
> > +      - p
> > +      - p
> > +    * .. _MEDIA-BUS-FMT-META-1X8-12:
> > +
> > +      - MEDIA_BUS_FMT_META_1X8_12
> > +      - 0x8003
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      - b0\ :sub:`7`
> > +      - b0\ :sub:`6`
> > +      - b0\ :sub:`5`
> > +      - b0\ :sub:`4`
> > +      - b0\ :sub:`3`
> > +      - b0\ :sub:`2`
> > +      - b0\ :sub:`1`
> > +      - b0\ :sub:`0`
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +    * .. _MEDIA-BUS-FMT-META-1X8-14:
> > +
> > +      - MEDIA_BUS_FMT_META_1X8_14
> > +      - 0x8004
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      - b0\ :sub:`7`
> > +      - b0\ :sub:`6`
> > +      - b0\ :sub:`5`
> > +      - b0\ :sub:`4`
> > +      - b0\ :sub:`3`
> > +      - b0\ :sub:`2`
> > +      - b0\ :sub:`1`
> > +      - b0\ :sub:`0`
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +    * .. _MEDIA-BUS-FMT-META-1X8-16:
> > +
> > +      - MEDIA_BUS_FMT_META_1X8_16
> > +      - 0x8005
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      - b0\ :sub:`7`
> > +      - b0\ :sub:`6`
> > +      - b0\ :sub:`5`
> > +      - b0\ :sub:`4`
> > +      - b0\ :sub:`3`
> > +      - b0\ :sub:`2`
> > +      - b0\ :sub:`1`
> > +      - b0\ :sub:`0`
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +    * .. _MEDIA-BUS-FMT-META-1X8-20:
> > +
> > +      - MEDIA_BUS_FMT_META_1X8_20
> > +      - 0x8007
> > +      -
> > +      -
> > +      -
> > +      -
> > +      -
> > +      - b0\ :sub:`7`
> > +      - b0\ :sub:`6`
> > +      - b0\ :sub:`5`
> > +      - b0\ :sub:`4`
> > +      - b0\ :sub:`3`
> > +      - b0\ :sub:`2`
> > +      - b0\ :sub:`1`
> > +      - b0\ :sub:`0`
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +    * .. _MEDIA-BUS-FMT-META-1X8-24:
> > +
> > +      - MEDIA_BUS_FMT_META_1X8_24
> > +      - 0x8009
> > +      -
> > +      - b0\ :sub:`7`
> > +      - b0\ :sub:`6`
> > +      - b0\ :sub:`5`
> > +      - b0\ :sub:`4`
> > +      - b0\ :sub:`3`
> > +      - b0\ :sub:`2`
> > +      - b0\ :sub:`1`
> > +      - b0\ :sub:`0`
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > +      - p
> > diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h
> > index a03c543cb072..722463523bbd 100644
> > --- a/include/uapi/linux/media-bus-format.h
> > +++ b/include/uapi/linux/media-bus-format.h
> > @@ -173,4 +173,13 @@
> >   */
> >  #define MEDIA_BUS_FMT_METADATA_FIXED		0x7001
> >  
> > +/* Generic line based metadata formats for serial buses. Next is 0x800b. */
> > +#define MEDIA_BUS_FMT_META_1X8_8		0x8001
> > +#define MEDIA_BUS_FMT_META_1X8_10		0x8002
> > +#define MEDIA_BUS_FMT_META_1X8_12		0x8003
> > +#define MEDIA_BUS_FMT_META_1X8_14		0x8004
> > +#define MEDIA_BUS_FMT_META_1X8_16		0x8005
> 
> You skip 0x8006 here in case there will be a _18 variant?

I removed duplicate entries but forgot to change the values. I'll address
that for v2.

There are different packings defined for these formats, sensor generally
using the pixel data equivalent format. With 16 and more bits per sample,
multiple octets of metadata can be transported in that sample, and those
formats are equivalent to the formats at half (or third) of the sample
size. Say, alternative format for 16 bpp image data to 1X8_16 is 1X8_8.

-- 
Kind regards,

Sakari Ailus