14-bit pixel/media bus formats for i.MX8QM/QXP CSI

14-bit pixel/media bus formats for i.MX8QM/QXP CSI

[Daniel Glöckner]

Hello,

the i.MX8QM and i.MX8QXP contain MIPI CSI-2 controllers that forward the
received data on a parallel bus to the Image Sensing Interface (ISI) of
the chip. If the data on the MIPI bus is in any of the six RAW formats
defined for CSI-2, the CSI controller will shift the values so that the
msb is always in bit 13. This was most likely done to allow following
hardware to process the data as RAW14 regardless of the actual RAW format.
Unfortunately the ISI is not able to shift the bits back before writing it
to memory. RAW8 data therefore has to be saved in two bytes per sample with
two unused bits at the top and six unused bits at the bottom.

The drivers for the hardware are still being developed in NXP's repository
at CodeAurora. We have extended them to support greyscale and Bayer
cameras. Now all we need are stable defines for the pixel and media bus
formats for use in libraries and applications and documentation for people
to know their meaning.

To keep the number of needed formats low, we would like to ignore that
there might be unused bits at the bottom. Then we can use the existing
MEDIA_BUS_FMT_S{BGGR,GBRG,GRBG,RGGB}14_1X14 formats between the CSI
controller and the ISI and just have to add a MEDIA_BUS_FMT_Y14_1X14
format. For the pixel formats we would add V4L2_PIX_FMT_Y14 and rebase
e38d5f254ad8 from Sakari's packed12-postponed branch for Bayer.

Now the questions:

 - Is it acceptable to use MEDIA_BUS_FMT_Y14_1X14 in this case instead of
   using MEDIA_BUS_FMT_Y12_1X14_PADLO, MEDIA_BUS_FMT_Y10_1X14_PADLO,
   MEDIA_BUS_FMT_Y8_1X14_PADLO, MEDIA_BUS_FMT_Y7_1X14_PADLO,
   MEDIA_BUS_FMT_Y6_1X14_PADLO? Another 20 _PADLO formats would have to
   be added for Bayer.

 - Given the history of Sakari's packed12-postponed branch, is there a
   chance to get these definitions merged into mainline although the
   driver is not? I fear that NXP's drivers will not hit mainline for a
   long time.

 - Sakari, do you mind me adding the same license header to your
   pixfmt-y14.rst that is used by all other pixfmt-y*.rst files?

Best regards,

  Daniel

Re: 14-bit pixel/media bus formats for i.MX8QM/QXP CSI

[Sakari Ailus]

Hi Daniel,

On Fri, Jan 24, 2020 at 09:36:25PM +0100, Daniel Glöckner wrote:
> Hello,
> 
> the i.MX8QM and i.MX8QXP contain MIPI CSI-2 controllers that forward the
> received data on a parallel bus to the Image Sensing Interface (ISI) of
> the chip. If the data on the MIPI bus is in any of the six RAW formats
> defined for CSI-2, the CSI controller will shift the values so that the
> msb is always in bit 13. This was most likely done to allow following
> hardware to process the data as RAW14 regardless of the actual RAW format.
> Unfortunately the ISI is not able to shift the bits back before writing it
> to memory. RAW8 data therefore has to be saved in two bytes per sample with
> two unused bits at the top and six unused bits at the bottom.

That's a rather peculiar implementation, indeed.

> 
> The drivers for the hardware are still being developed in NXP's repository
> at CodeAurora. We have extended them to support greyscale and Bayer
> cameras. Now all we need are stable defines for the pixel and media bus
> formats for use in libraries and applications and documentation for people
> to know their meaning.
> 
> To keep the number of needed formats low, we would like to ignore that
> there might be unused bits at the bottom. Then we can use the existing
> MEDIA_BUS_FMT_S{BGGR,GBRG,GRBG,RGGB}14_1X14 formats between the CSI
> controller and the ISI and just have to add a MEDIA_BUS_FMT_Y14_1X14
> format. For the pixel formats we would add V4L2_PIX_FMT_Y14 and rebase
> e38d5f254ad8 from Sakari's packed12-postponed branch for Bayer.
> 
> Now the questions:
> 
>  - Is it acceptable to use MEDIA_BUS_FMT_Y14_1X14 in this case instead of
>    using MEDIA_BUS_FMT_Y12_1X14_PADLO, MEDIA_BUS_FMT_Y10_1X14_PADLO,
>    MEDIA_BUS_FMT_Y8_1X14_PADLO, MEDIA_BUS_FMT_Y7_1X14_PADLO,
>    MEDIA_BUS_FMT_Y6_1X14_PADLO? Another 20 _PADLO formats would have to
>    be added for Bayer.

I think I'd say yes, you could do this, *if* you're fully certain you'll
*never* see this CSI-2 receiver paired with any other hardware than the
ISI, which is the case for instance if it's part of the same device. As if
there is hardware that can make use of the information on how many bits are
actually used there, you'd need to expose that information on the uAPI as
well. Changing that would be an uAPI change, something that should be
avoided if at all possible.

> 
>  - Given the history of Sakari's packed12-postponed branch, is there a
>    chance to get these definitions merged into mainline although the
>    driver is not? I fear that NXP's drivers will not hit mainline for a
>    long time.

Cc'ing Hans.

We have a practice not merging the format definitions (or other API
additions) before there are users. That's partly because often API
additions merged before the user have ended never being used because it
turned out to be something else that the driver actually needed, or the
driver was never merged.

That said, I don't foresee that problem here: these (14-bit raw Bayer
pixel) formats are pretty standard stuff albeit still rare for 14 bits is
more than you usually need, but we started off with 8 bits per pixel and 12
isn't uncommon nowadays.

I'd be leaning towards merging them --- it's just a question of time when
they'll be needed somewhere else.

> 
>  - Sakari, do you mind me adding the same license header to your
>    pixfmt-y14.rst that is used by all other pixfmt-y*.rst files?

Feel free to do so.

-- 
Kind regards,

Sakari Ailus
sakari.ailus@linux.intel.com

Re: 14-bit pixel/media bus formats for i.MX8QM/QXP CSI

[Daniel Glöckner]

Hi Sakari,

Am 01/27/20 um 11:42 schrieb Sakari Ailus:
> On Fri, Jan 24, 2020 at 09:36:25PM +0100, Daniel Glöckner wrote:
>> the i.MX8QM and i.MX8QXP contain MIPI CSI-2 controllers that forward the
>> received data on a parallel bus to the Image Sensing Interface (ISI) of
>> the chip. If the data on the MIPI bus is in any of the six RAW formats
>> defined for CSI-2, the CSI controller will shift the values so that the
>> msb is always in bit 13. This was most likely done to allow following
>> hardware to process the data as RAW14 regardless of the actual RAW format.
>> Unfortunately the ISI is not able to shift the bits back before writing it
>> to memory. RAW8 data therefore has to be saved in two bytes per sample with
>> two unused bits at the top and six unused bits at the bottom.

>>  - Is it acceptable to use MEDIA_BUS_FMT_Y14_1X14 in this case instead of
>>    using MEDIA_BUS_FMT_Y12_1X14_PADLO, MEDIA_BUS_FMT_Y10_1X14_PADLO,
>>    MEDIA_BUS_FMT_Y8_1X14_PADLO, MEDIA_BUS_FMT_Y7_1X14_PADLO,
>>    MEDIA_BUS_FMT_Y6_1X14_PADLO? Another 20 _PADLO formats would have to
>>    be added for Bayer.
> 
> I think I'd say yes, you could do this, *if* you're fully certain you'll
> *never* see this CSI-2 receiver paired with any other hardware than the
> ISI, which is the case for instance if it's part of the same device. As if
> there is hardware that can make use of the information on how many bits are
> actually used there, you'd need to expose that information on the uAPI as
> well. Changing that would be an uAPI change, something that should be
> avoided if at all possible.

Given that the NXP revealed that the MIPI DSI controller in the i.MX8 is from
Northwest Logic and the block diagram for the MIPI CSI controller in NXP's
reference manual looks like it was drawn by the same artist who drew the block
diagram in Northwest Logic's DSI product brief, I suspect that the CSI
controller is also an IP core from Northwest Logic. I believe the one currently
advertized on their website is the successor. I don't have access to Northwest
Logic's datasheet, so I can't compare the register layouts.

Looking at the driver from NXP we can probably rule out that the driver will be
used with anything else than NXP hardware. Half of the register accesses in that
driver go to additional logic specific to the i.MX8 (CSR registers) and there is
communication with the chip-internal Cortex-M4 to reset the CSI controller. NXP
placed it into drivers/media/platform/imx8/.

Best regards,

  Daniel

Re: 14-bit pixel/media bus formats for i.MX8QM/QXP CSI

[Sakari Ailus]

Hi Daniel,

On Mon, Jan 27, 2020 at 12:26:20PM +0100, Daniel Glöckner wrote:
> Hi Sakari,
> 
> Am 01/27/20 um 11:42 schrieb Sakari Ailus:
> > On Fri, Jan 24, 2020 at 09:36:25PM +0100, Daniel Glöckner wrote:
> >> the i.MX8QM and i.MX8QXP contain MIPI CSI-2 controllers that forward the
> >> received data on a parallel bus to the Image Sensing Interface (ISI) of
> >> the chip. If the data on the MIPI bus is in any of the six RAW formats
> >> defined for CSI-2, the CSI controller will shift the values so that the
> >> msb is always in bit 13. This was most likely done to allow following
> >> hardware to process the data as RAW14 regardless of the actual RAW format.
> >> Unfortunately the ISI is not able to shift the bits back before writing it
> >> to memory. RAW8 data therefore has to be saved in two bytes per sample with
> >> two unused bits at the top and six unused bits at the bottom.
> 
> >>  - Is it acceptable to use MEDIA_BUS_FMT_Y14_1X14 in this case instead of
> >>    using MEDIA_BUS_FMT_Y12_1X14_PADLO, MEDIA_BUS_FMT_Y10_1X14_PADLO,
> >>    MEDIA_BUS_FMT_Y8_1X14_PADLO, MEDIA_BUS_FMT_Y7_1X14_PADLO,
> >>    MEDIA_BUS_FMT_Y6_1X14_PADLO? Another 20 _PADLO formats would have to
> >>    be added for Bayer.
> > 
> > I think I'd say yes, you could do this, *if* you're fully certain you'll
> > *never* see this CSI-2 receiver paired with any other hardware than the
> > ISI, which is the case for instance if it's part of the same device. As if
> > there is hardware that can make use of the information on how many bits are
> > actually used there, you'd need to expose that information on the uAPI as
> > well. Changing that would be an uAPI change, something that should be
> > avoided if at all possible.
> 
> Given that the NXP revealed that the MIPI DSI controller in the i.MX8 is from
> Northwest Logic and the block diagram for the MIPI CSI controller in NXP's
> reference manual looks like it was drawn by the same artist who drew the block
> diagram in Northwest Logic's DSI product brief, I suspect that the CSI
> controller is also an IP core from Northwest Logic. I believe the one currently
> advertized on their website is the successor. I don't have access to Northwest
> Logic's datasheet, so I can't compare the register layouts.
> 
> Looking at the driver from NXP we can probably rule out that the driver will be
> used with anything else than NXP hardware. Half of the register accesses in that
> driver go to additional logic specific to the i.MX8 (CSR registers) and there is

What do those registers do, roughly?

> communication with the chip-internal Cortex-M4 to reset the CSI controller. NXP

Ouch.

> placed it into drivers/media/platform/imx8/.

Should I assume this CSI-2 receiver will have its own compatible string
specific to these two SoCs?

-- 
Regards,

Sakari Ailus
sakari.ailus@linux.intel.com

Re: 14-bit pixel/media bus formats for i.MX8QM/QXP CSI

[Daniel Glöckner]

Hello Sakari,

sorry for the late reply.

Am 01/27/20 um 12:46 schrieb Sakari Ailus:
> On Mon, Jan 27, 2020 at 12:26:20PM +0100, Daniel Glöckner wrote:
>> Looking at the driver from NXP we can probably rule out that the driver will be
>> used with anything else than NXP hardware. Half of the register accesses in that
>> driver go to additional logic specific to the i.MX8 (CSR registers) and there is
> 
> What do those registers do, roughly?

Enable signals between the ISI and the CSI controller, change sync polarity,
provide PHY status bits, settings for interlaced mode, power management, etc.

>> placed it into drivers/media/platform/imx8/.
> 
> Should I assume this CSI-2 receiver will have its own compatible string
> specific to these two SoCs?

It is fsl,mxc-mipi-csi2.

And they wrote a second driver used on other SoCs (i.MX8MQ) that, according to
its first commit message, should eventually be merged with the first driver.
Its compatible string is simply fsl,mxc-mipi-csi2_yav. That driver accesses
just a single GPR register which contains a subset of the bits found in the
CSR registers of the i.MX8QXP.

Best regards,

  Daniel

-- 
Besuchen Sie uns auf der Embedded World 2020 in Nürnberg!
-> Halle 4, Stand 368

Dipl.-Math. Daniel Glöckner, emlix GmbH, http://www.emlix.com
Fon +49 551 30664-0, Fax +49 551 30664-11,
Gothaer Platz 3, 37083 Göttingen, Germany
Sitz der Gesellschaft: Göttingen, Amtsgericht Göttingen HR B 3160
Geschäftsführung: Heike Jordan, Dr. Uwe Kracke
Ust-IdNr.: DE 205 198 055

emlix - your embedded linux partner

Re: 14-bit pixel/media bus formats for i.MX8QM/QXP CSI

[Hans Verkuil]

On 1/27/20 11:42 AM, Sakari Ailus wrote:
> Hi Daniel,
> 
> On Fri, Jan 24, 2020 at 09:36:25PM +0100, Daniel Glöckner wrote:
>> Hello,
>>
>> the i.MX8QM and i.MX8QXP contain MIPI CSI-2 controllers that forward the
>> received data on a parallel bus to the Image Sensing Interface (ISI) of
>> the chip. If the data on the MIPI bus is in any of the six RAW formats
>> defined for CSI-2, the CSI controller will shift the values so that the
>> msb is always in bit 13. This was most likely done to allow following
>> hardware to process the data as RAW14 regardless of the actual RAW format.
>> Unfortunately the ISI is not able to shift the bits back before writing it
>> to memory. RAW8 data therefore has to be saved in two bytes per sample with
>> two unused bits at the top and six unused bits at the bottom.
> 
> That's a rather peculiar implementation, indeed.
> 
>>
>> The drivers for the hardware are still being developed in NXP's repository
>> at CodeAurora. We have extended them to support greyscale and Bayer
>> cameras. Now all we need are stable defines for the pixel and media bus
>> formats for use in libraries and applications and documentation for people
>> to know their meaning.
>>
>> To keep the number of needed formats low, we would like to ignore that
>> there might be unused bits at the bottom. Then we can use the existing
>> MEDIA_BUS_FMT_S{BGGR,GBRG,GRBG,RGGB}14_1X14 formats between the CSI
>> controller and the ISI and just have to add a MEDIA_BUS_FMT_Y14_1X14
>> format. For the pixel formats we would add V4L2_PIX_FMT_Y14 and rebase
>> e38d5f254ad8 from Sakari's packed12-postponed branch for Bayer.
>>
>> Now the questions:
>>
>>  - Is it acceptable to use MEDIA_BUS_FMT_Y14_1X14 in this case instead of
>>    using MEDIA_BUS_FMT_Y12_1X14_PADLO, MEDIA_BUS_FMT_Y10_1X14_PADLO,
>>    MEDIA_BUS_FMT_Y8_1X14_PADLO, MEDIA_BUS_FMT_Y7_1X14_PADLO,
>>    MEDIA_BUS_FMT_Y6_1X14_PADLO? Another 20 _PADLO formats would have to
>>    be added for Bayer.
> 
> I think I'd say yes, you could do this, *if* you're fully certain you'll
> *never* see this CSI-2 receiver paired with any other hardware than the
> ISI, which is the case for instance if it's part of the same device. As if
> there is hardware that can make use of the information on how many bits are
> actually used there, you'd need to expose that information on the uAPI as
> well. Changing that would be an uAPI change, something that should be
> avoided if at all possible.
> 
>>
>>  - Given the history of Sakari's packed12-postponed branch, is there a
>>    chance to get these definitions merged into mainline although the
>>    driver is not? I fear that NXP's drivers will not hit mainline for a
>>    long time.
> 
> Cc'ing Hans.
> 
> We have a practice not merging the format definitions (or other API
> additions) before there are users. That's partly because often API
> additions merged before the user have ended never being used because it
> turned out to be something else that the driver actually needed, or the
> driver was never merged.
> 
> That said, I don't foresee that problem here: these (14-bit raw Bayer
> pixel) formats are pretty standard stuff albeit still rare for 14 bits is
> more than you usually need, but we started off with 8 bits per pixel and 12
> isn't uncommon nowadays.
> 
> I'd be leaning towards merging them --- it's just a question of time when
> they'll be needed somewhere else.

I see no problem with adding MEDIA_BUS_FMT_Y14_1X14.

Regards,

	Hans

> 
>>
>>  - Sakari, do you mind me adding the same license header to your
>>    pixfmt-y14.rst that is used by all other pixfmt-y*.rst files?
> 
> Feel free to do so.
> 

Re: 14-bit pixel/media bus formats for i.MX8QM/QXP CSI

[Laurent Pinchart]

Hi Daniel,

On Fri, Jan 24, 2020 at 09:36:25PM +0100, Daniel Glöckner wrote:
> Hello,
> 
> the i.MX8QM and i.MX8QXP contain MIPI CSI-2 controllers that forward the
> received data on a parallel bus to the Image Sensing Interface (ISI) of
> the chip. If the data on the MIPI bus is in any of the six RAW formats
> defined for CSI-2, the CSI controller will shift the values so that the
> msb is always in bit 13. This was most likely done to allow following
> hardware to process the data as RAW14 regardless of the actual RAW format.
> Unfortunately the ISI is not able to shift the bits back before writing it
> to memory. RAW8 data therefore has to be saved in two bytes per sample with
> two unused bits at the top and six unused bits at the bottom.
> 
> The drivers for the hardware are still being developed in NXP's repository
> at CodeAurora.

Highjacking the thread a little bit, for my information (and thus
without any commitment of any kind), do you know what the upstreaming
plan (and in particular the upstreaming timeline) is ?

> We have extended them to support greyscale and Bayer
> cameras. Now all we need are stable defines for the pixel and media bus
> formats for use in libraries and applications and documentation for people
> to know their meaning.
> 
> To keep the number of needed formats low, we would like to ignore that
> there might be unused bits at the bottom. Then we can use the existing
> MEDIA_BUS_FMT_S{BGGR,GBRG,GRBG,RGGB}14_1X14 formats between the CSI
> controller and the ISI and just have to add a MEDIA_BUS_FMT_Y14_1X14
> format. For the pixel formats we would add V4L2_PIX_FMT_Y14 and rebase
> e38d5f254ad8 from Sakari's packed12-postponed branch for Bayer.
> 
> Now the questions:
> 
>  - Is it acceptable to use MEDIA_BUS_FMT_Y14_1X14 in this case instead of
>    using MEDIA_BUS_FMT_Y12_1X14_PADLO, MEDIA_BUS_FMT_Y10_1X14_PADLO,
>    MEDIA_BUS_FMT_Y8_1X14_PADLO, MEDIA_BUS_FMT_Y7_1X14_PADLO,
>    MEDIA_BUS_FMT_Y6_1X14_PADLO? Another 20 _PADLO formats would have to
>    be added for Bayer.
> 
>  - Given the history of Sakari's packed12-postponed branch, is there a
>    chance to get these definitions merged into mainline although the
>    driver is not? I fear that NXP's drivers will not hit mainline for a
>    long time.
> 
>  - Sakari, do you mind me adding the same license header to your
>    pixfmt-y14.rst that is used by all other pixfmt-y*.rst files?

-- 
Regards,

Laurent Pinchart

Re: 14-bit pixel/media bus formats for i.MX8QM/QXP CSI

[Daniel Glöckner]

Hello Laurent,

Am 02/18/20 um 02:34 schrieb Laurent Pinchart:
> On Fri, Jan 24, 2020 at 09:36:25PM +0100, Daniel Glöckner wrote:
>> the i.MX8QM and i.MX8QXP contain MIPI CSI-2 controllers that forward the
>> received data on a parallel bus to the Image Sensing Interface (ISI) of
>> the chip.

>> The drivers for the hardware are still being developed in NXP's repository
>> at CodeAurora.
> 
> Highjacking the thread a little bit, for my information (and thus
> without any commitment of any kind), do you know what the upstreaming
> plan (and in particular the upstreaming timeline) is ?

sorry, I have no information about that and I am not in direct contact
with NXP.

Best regards,

  Daniel


-- 
Besuchen Sie uns auf der Embedded World 2020 in Nürnberg!
-> Halle 4, Stand 368

Dipl.-Math. Daniel Glöckner, emlix GmbH, http://www.emlix.com
Fon +49 551 30664-0, Fax +49 551 30664-11,
Gothaer Platz 3, 37083 Göttingen, Germany
Sitz der Gesellschaft: Göttingen, Amtsgericht Göttingen HR B 3160
Geschäftsführung: Heike Jordan, Dr. Uwe Kracke
Ust-IdNr.: DE 205 198 055

emlix - your embedded linux partner

[PATCH 1/3] media: v4l: Add 14-bit raw bayer pixel formats

[Daniel Glöckner]

From: Sakari Ailus <sakari.ailus@linux.intel.com>

The formats added by this patch are:

	V4L2_PIX_FMT_SBGGR14
	V4L2_PIX_FMT_SGBRG14
	V4L2_PIX_FMT_SGRBG14
	V4L2_PIX_FMT_SRGGB14

Signed-off-by: Jouni Ukkonen <jouni.ukkonen@intel.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
[dg@emlix.com: rebased onto current media_tree]
Signed-off-by: Daniel Glöckner <dg@emlix.com>
---
 Documentation/media/uapi/v4l/pixfmt-bayer.rst |  1 +
 .../media/uapi/v4l/pixfmt-srggb14.rst         | 82 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c          |  4 +
 include/uapi/linux/videodev2.h                |  4 +
 4 files changed, 91 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-srggb14.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-bayer.rst b/Documentation/media/uapi/v4l/pixfmt-bayer.rst
index cfa2f4e3e114..807ab34ba93b 100644
--- a/Documentation/media/uapi/v4l/pixfmt-bayer.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-bayer.rst
@@ -34,5 +34,6 @@ orders. See also `the Wikipedia article on Bayer filter
     pixfmt-srggb10-ipu3
     pixfmt-srggb12
     pixfmt-srggb12p
+    pixfmt-srggb14
     pixfmt-srggb14p
     pixfmt-srggb16
diff --git a/Documentation/media/uapi/v4l/pixfmt-srggb14.rst b/Documentation/media/uapi/v4l/pixfmt-srggb14.rst
new file mode 100644
index 000000000000..3420d4d1825e
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-srggb14.rst
@@ -0,0 +1,82 @@
+.. Permission is granted to copy, distribute and/or modify this
+.. document under the terms of the GNU Free Documentation License,
+.. Version 1.1 or any later version published by the Free Software
+.. Foundation, with no Invariant Sections, no Front-Cover Texts
+.. and no Back-Cover Texts. A copy of the license is included at
+.. Documentation/media/uapi/fdl-appendix.rst.
+..
+.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
+
+.. _V4L2-PIX-FMT-SRGGB14:
+.. _v4l2-pix-fmt-sbggr14:
+.. _v4l2-pix-fmt-sgbrg14:
+.. _v4l2-pix-fmt-sgrbg14:
+
+
+***************************************************************************************************************************
+V4L2_PIX_FMT_SRGGB14 ('RG14'), V4L2_PIX_FMT_SGRBG14 ('GR14'), V4L2_PIX_FMT_SGBRG14 ('GB14'), V4L2_PIX_FMT_SBGGR14 ('BG14'),
+***************************************************************************************************************************
+
+
+14-bit Bayer formats expanded to 16 bits
+
+
+Description
+===========
+
+These four pixel formats are raw sRGB / Bayer formats with 14 bits per
+colour. Each sample is stored in a 16-bit word, with two unused high
+bits filled with zeros. Each n-pixel row contains n/2 green samples
+and n/2 blue or red samples, with alternating red and blue rows. Bytes
+are stored in memory in little endian order. They are conventionally
+described as GRGR... BGBG..., RGRG... GBGB..., etc. Below is an
+example of a small V4L2_PIX_FMT_SBGGR14 image:
+
+**Byte Order.**
+Each cell is one byte, the two most significant bits in the high bytes are
+zero.
+
+
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+    :widths:       2 1 1 1 1 1 1 1 1
+
+
+    * - start + 0:
+      - B\ :sub:`00low`
+      - B\ :sub:`00high`
+      - G\ :sub:`01low`
+      - G\ :sub:`01high`
+      - B\ :sub:`02low`
+      - B\ :sub:`02high`
+      - G\ :sub:`03low`
+      - G\ :sub:`03high`
+    * - start + 8:
+      - G\ :sub:`10low`
+      - G\ :sub:`10high`
+      - R\ :sub:`11low`
+      - R\ :sub:`11high`
+      - G\ :sub:`12low`
+      - G\ :sub:`12high`
+      - R\ :sub:`13low`
+      - R\ :sub:`13high`
+    * - start + 16:
+      - B\ :sub:`20low`
+      - B\ :sub:`20high`
+      - G\ :sub:`21low`
+      - G\ :sub:`21high`
+      - B\ :sub:`22low`
+      - B\ :sub:`22high`
+      - G\ :sub:`23low`
+      - G\ :sub:`23high`
+    * - start + 24:
+      - G\ :sub:`30low`
+      - G\ :sub:`30high`
+      - R\ :sub:`31low`
+      - R\ :sub:`31high`
+      - G\ :sub:`32low`
+      - G\ :sub:`32high`
+      - R\ :sub:`33low`
+      - R\ :sub:`33high`
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index aaf83e254272..2558ffe4e3cb 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1306,6 +1306,10 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_SGBRG12P:	descr = "12-bit Bayer GBGB/RGRG Packed"; break;
 	case V4L2_PIX_FMT_SGRBG12P:	descr = "12-bit Bayer GRGR/BGBG Packed"; break;
 	case V4L2_PIX_FMT_SRGGB12P:	descr = "12-bit Bayer RGRG/GBGB Packed"; break;
+	case V4L2_PIX_FMT_SBGGR14:	descr = "14-bit Bayer BGBG/GRGR"; break;
+	case V4L2_PIX_FMT_SGBRG14:	descr = "14-bit Bayer GBGB/RGRG"; break;
+	case V4L2_PIX_FMT_SGRBG14:	descr = "14-bit Bayer GRGR/BGBG"; break;
+	case V4L2_PIX_FMT_SRGGB14:	descr = "14-bit Bayer RGRG/GBGB"; break;
 	case V4L2_PIX_FMT_SBGGR14P:	descr = "14-bit Bayer BGBG/GRGR Packed"; break;
 	case V4L2_PIX_FMT_SGBRG14P:	descr = "14-bit Bayer GBGB/RGRG Packed"; break;
 	case V4L2_PIX_FMT_SGRBG14P:	descr = "14-bit Bayer GRGR/BGBG Packed"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 5f9357dcb060..8429e5ab2d94 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -662,6 +662,10 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_SGBRG12P v4l2_fourcc('p', 'G', 'C', 'C')
 #define V4L2_PIX_FMT_SGRBG12P v4l2_fourcc('p', 'g', 'C', 'C')
 #define V4L2_PIX_FMT_SRGGB12P v4l2_fourcc('p', 'R', 'C', 'C')
+#define V4L2_PIX_FMT_SBGGR14 v4l2_fourcc('B', 'G', '1', '4') /* 14  BGBG.. GRGR.. */
+#define V4L2_PIX_FMT_SGBRG14 v4l2_fourcc('G', 'B', '1', '4') /* 14  GBGB.. RGRG.. */
+#define V4L2_PIX_FMT_SGRBG14 v4l2_fourcc('G', 'R', '1', '4') /* 14  GRGR.. BGBG.. */
+#define V4L2_PIX_FMT_SRGGB14 v4l2_fourcc('R', 'G', '1', '4') /* 14  RGRG.. GBGB.. */
 	/* 14bit raw bayer packed, 7 bytes for every 4 pixels */
 #define V4L2_PIX_FMT_SBGGR14P v4l2_fourcc('p', 'B', 'E', 'E')
 #define V4L2_PIX_FMT_SGBRG14P v4l2_fourcc('p', 'G', 'E', 'E')
-- 
2.17.1

Re: [PATCH 1/3] media: v4l: Add 14-bit raw bayer pixel formats

[Sakari Ailus]

Hi Daniel,

On Mon, Feb 24, 2020 at 06:52:20PM +0100, Daniel Glöckner wrote:
> From: Sakari Ailus <sakari.ailus@linux.intel.com>
> 
> The formats added by this patch are:
> 
> 	V4L2_PIX_FMT_SBGGR14
> 	V4L2_PIX_FMT_SGBRG14
> 	V4L2_PIX_FMT_SGRBG14
> 	V4L2_PIX_FMT_SRGGB14
> 
> Signed-off-by: Jouni Ukkonen <jouni.ukkonen@intel.com>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
> [dg@emlix.com: rebased onto current media_tree]
> Signed-off-by: Daniel Glöckner <dg@emlix.com>

For the three patches:

Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>

-- 
Sakari Ailus

[PATCH 2/3] media: v4l: Add 14-bit raw greyscale pixel format

[Daniel Glöckner]

The new format is called V4L2_PIX_FMT_Y14. Like V4L2_PIX_FMT_Y10 and
V4L2_PIX_FMT_Y12 it is stored in two bytes per pixel but has only two
unused bits at the top.

Signed-off-by: Daniel Glöckner <dg@emlix.com>
---
 Documentation/media/uapi/v4l/pixfmt-y14.rst  | 72 ++++++++++++++++++++
 Documentation/media/uapi/v4l/yuv-formats.rst |  1 +
 drivers/media/v4l2-core/v4l2-ioctl.c         |  1 +
 include/uapi/linux/videodev2.h               |  1 +
 4 files changed, 75 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-y14.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-y14.rst b/Documentation/media/uapi/v4l/pixfmt-y14.rst
new file mode 100644
index 000000000000..5c260f8da088
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-y14.rst
@@ -0,0 +1,72 @@
+.. Permission is granted to copy, distribute and/or modify this
+.. document under the terms of the GNU Free Documentation License,
+.. Version 1.1 or any later version published by the Free Software
+.. Foundation, with no Invariant Sections, no Front-Cover Texts
+.. and no Back-Cover Texts. A copy of the license is included at
+.. Documentation/media/uapi/fdl-appendix.rst.
+..
+.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
+
+.. _V4L2-PIX-FMT-Y14:
+
+*************************
+V4L2_PIX_FMT_Y14 ('Y14 ')
+*************************
+
+
+Grey-scale image
+
+
+Description
+===========
+
+This is a grey-scale image with a depth of 14 bits per pixel. Pixels are
+stored in 16-bit words with unused high bits padded with 0. The least
+significant byte is stored at lower memory addresses (little-endian).
+
+**Byte Order.**
+Each cell is one byte.
+
+
+
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - Y'\ :sub:`00low`
+      - Y'\ :sub:`00high`
+      - Y'\ :sub:`01low`
+      - Y'\ :sub:`01high`
+      - Y'\ :sub:`02low`
+      - Y'\ :sub:`02high`
+      - Y'\ :sub:`03low`
+      - Y'\ :sub:`03high`
+    * - start + 8:
+      - Y'\ :sub:`10low`
+      - Y'\ :sub:`10high`
+      - Y'\ :sub:`11low`
+      - Y'\ :sub:`11high`
+      - Y'\ :sub:`12low`
+      - Y'\ :sub:`12high`
+      - Y'\ :sub:`13low`
+      - Y'\ :sub:`13high`
+    * - start + 16:
+      - Y'\ :sub:`20low`
+      - Y'\ :sub:`20high`
+      - Y'\ :sub:`21low`
+      - Y'\ :sub:`21high`
+      - Y'\ :sub:`22low`
+      - Y'\ :sub:`22high`
+      - Y'\ :sub:`23low`
+      - Y'\ :sub:`23high`
+    * - start + 24:
+      - Y'\ :sub:`30low`
+      - Y'\ :sub:`30high`
+      - Y'\ :sub:`31low`
+      - Y'\ :sub:`31high`
+      - Y'\ :sub:`32low`
+      - Y'\ :sub:`32high`
+      - Y'\ :sub:`33low`
+      - Y'\ :sub:`33high`
diff --git a/Documentation/media/uapi/v4l/yuv-formats.rst b/Documentation/media/uapi/v4l/yuv-formats.rst
index 867470e5f9e1..3b259e31b7a1 100644
--- a/Documentation/media/uapi/v4l/yuv-formats.rst
+++ b/Documentation/media/uapi/v4l/yuv-formats.rst
@@ -35,6 +35,7 @@ to brightness information.
     pixfmt-grey
     pixfmt-y10
     pixfmt-y12
+    pixfmt-y14
     pixfmt-y10b
     pixfmt-y10p
     pixfmt-y16
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 2558ffe4e3cb..7fdb4e87325f 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1222,6 +1222,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_Y6:		descr = "6-bit Greyscale"; break;
 	case V4L2_PIX_FMT_Y10:		descr = "10-bit Greyscale"; break;
 	case V4L2_PIX_FMT_Y12:		descr = "12-bit Greyscale"; break;
+	case V4L2_PIX_FMT_Y14:		descr = "14-bit Greyscale"; break;
 	case V4L2_PIX_FMT_Y16:		descr = "16-bit Greyscale"; break;
 	case V4L2_PIX_FMT_Y16_BE:	descr = "16-bit Greyscale BE"; break;
 	case V4L2_PIX_FMT_Y10BPACK:	descr = "10-bit Greyscale (Packed)"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 8429e5ab2d94..5c54183c2630 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -565,6 +565,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_Y6      v4l2_fourcc('Y', '0', '6', ' ') /*  6  Greyscale     */
 #define V4L2_PIX_FMT_Y10     v4l2_fourcc('Y', '1', '0', ' ') /* 10  Greyscale     */
 #define V4L2_PIX_FMT_Y12     v4l2_fourcc('Y', '1', '2', ' ') /* 12  Greyscale     */
+#define V4L2_PIX_FMT_Y14     v4l2_fourcc('Y', '1', '4', ' ') /* 14  Greyscale     */
 #define V4L2_PIX_FMT_Y16     v4l2_fourcc('Y', '1', '6', ' ') /* 16  Greyscale     */
 #define V4L2_PIX_FMT_Y16_BE  v4l2_fourcc_be('Y', '1', '6', ' ') /* 16  Greyscale BE  */
 
-- 
2.17.1

[PATCH 3/3] media: v4l: Add 1X14 14-bit greyscale media bus code definition

[Daniel Glöckner]

The code is called MEDIA_BUS_FMT_Y14_1X14 and behaves just like
MEDIA_BUS_FMT_Y12_1X12 with two more bits.

Signed-off-by: Daniel Glöckner <dg@emlix.com>
---
 .../media/uapi/v4l/subdev-formats.rst         | 37 +++++++++++++++++++
 include/uapi/linux/media-bus-format.h         |  3 +-
 2 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/Documentation/media/uapi/v4l/subdev-formats.rst b/Documentation/media/uapi/v4l/subdev-formats.rst
index 15e11f27b4c8..17bfb2beaa6a 100644
--- a/Documentation/media/uapi/v4l/subdev-formats.rst
+++ b/Documentation/media/uapi/v4l/subdev-formats.rst
@@ -5792,6 +5792,43 @@ the following codes.
       - u\ :sub:`2`
       - u\ :sub:`1`
       - u\ :sub:`0`
+    * .. _MEDIA-BUS-FMT-Y14-1X14:
+
+      - MEDIA_BUS_FMT_Y14_1X14
+      - 0x202d
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      - y\ :sub:`13`
+      - y\ :sub:`12`
+      - y\ :sub:`11`
+      - y\ :sub:`10`
+      - y\ :sub:`9`
+      - y\ :sub:`8`
+      - y\ :sub:`7`
+      - y\ :sub:`6`
+      - y\ :sub:`5`
+      - y\ :sub:`4`
+      - y\ :sub:`3`
+      - y\ :sub:`2`
+      - y\ :sub:`1`
+      - y\ :sub:`0`
     * .. _MEDIA-BUS-FMT-UYVY8-1X16:
 
       - MEDIA_BUS_FMT_UYVY8_1X16
diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h
index 16c1fa2d89a4..84fa53ffb13f 100644
--- a/include/uapi/linux/media-bus-format.h
+++ b/include/uapi/linux/media-bus-format.h
@@ -64,7 +64,7 @@
 #define MEDIA_BUS_FMT_RGB121212_1X36		0x1019
 #define MEDIA_BUS_FMT_RGB161616_1X48		0x101a
 
-/* YUV (including grey) - next is	0x202d */
+/* YUV (including grey) - next is	0x202e */
 #define MEDIA_BUS_FMT_Y8_1X8			0x2001
 #define MEDIA_BUS_FMT_UV8_1X8			0x2015
 #define MEDIA_BUS_FMT_UYVY8_1_5X8		0x2002
@@ -86,6 +86,7 @@
 #define MEDIA_BUS_FMT_VYUY12_2X12		0x201d
 #define MEDIA_BUS_FMT_YUYV12_2X12		0x201e
 #define MEDIA_BUS_FMT_YVYU12_2X12		0x201f
+#define MEDIA_BUS_FMT_Y14_1X14			0x202d
 #define MEDIA_BUS_FMT_UYVY8_1X16		0x200f
 #define MEDIA_BUS_FMT_VYUY8_1X16		0x2010
 #define MEDIA_BUS_FMT_YUYV8_1X16		0x2011
-- 
2.17.1